Persistent Configuration Repository

All configurations for SRTC components should be stored in the Persistent Configuration Repository. The purpose is to store all possible configurations for each Deployment Set centrally and in a persistent manner. The user normally prepares all necessary configuration in the Persistent Configuration Repository rather than interacting with the Runtime Configuration Repository. The configuration values for the active Deployment Set are then copied from the Persistent Configuration Repository to the Runtime Configuration Repository automatically by the RTC Supervisor.

Modifying the Configuration

The user should typically use rtctkConfigTool to interact with the configuration in the Persistent Configuration Repository. See section Configuration Tool for a general description of how to use the tool. Alternatively, if a local CII configuration is being used, i.e. cii.config://local URI scheme, the underlying YAML files stored on disk can be accessed directly to prepare the configuration. The YAML file format was chosen to make direct manipulation easier. In addition, external FITS files can be used for large vectors and matrices. A URI to the external FITS file can then be embedded into the YAML file for a given datapoint.

The following subsections provide brief examples that are more specific to manipulating the Persistent Configuration Repository with rtctkConfigTool.

Datapoint Paths

Configuration parameters are stored in a tree hierarchy as leaf nodes in the Persistent Configuration Repository. This can also be thought of as a folder-like structure, similar to a filesystem. The nodes from the root of the tree to a particular leaf node form the components of a path. By adding the ‘/’ character as the path separator between each component, this forms a datapoint path string.

The current canonical structure for the path is one of the following:

/<deployment>/defaults/<component>/{static,dynamic}/<parameter>
/<deployment>/modes/<name>/<option>/<component>/dynamic/<parameter>

Where <deployment> is the name of a Deployment Set;
<name> is the name of a mode;
<option> is the name of one of the possible mode options;
<component> will typically be the SRTC component instance name;
<parameter> can represent a hierarchical sub-path with multiple sub-folders if a deeper hierarchy of configuration parameters is desired for a particular SRTC component, besides the basic grouping into static and dynamic parameters.

Path components must contain only lowercase alphanumeric characters or the underscore, i.e. characters from the set [a-z0-9_].

Datapoint Reading

The following is an example of reading a parameter called flatimg, which is a matrix, into a local file called flatimg.fits for the component datatask1:

$ rtctkConfigTool read persistent /depl1/defaults/datatask1/static/flatimg flatimg.fits

In this example, the repository is located at /home/eltdev/persistent_repo, i.e. this is the base directory.

Datapoint Writing

The following is an example of writing the contents of a local FITS file called flatimg.fits, which contains a matrix of floating-point doubles, into a parameter called flatimg for the component datatask1:

$ rtctkConfigTool write persistent /depl1/defaults/datatask1/static/flatimg flatimg.fits

It is possible to subsequently manipulate the data directly in the local file system, in this case, under the location /home/eltdev/persistent_repo.

If the datapoint does not exist, it is created automatically. Otherwise it is updated with the new value.

Datapoint Querying

Use the following command to list all datapoints and sub-paths under a certain path, in this case /datatask1/static:

$ rtctkConfigTool list persistent /depl1/defaults/datatask1/static

The data type for a particular datapoint can be queried with the following command:

$ rtctkConfigTool get persistent /depl1/defaults/datatask1/static/flatimg --type

Note

Since the --persistent-repo-endpoint option is not used in the above examples, this would cause the command to use the URI as given in the configuration for rtctkConfigTool itself or from any environment variable overrides.

Configuration Layout

The Persistent Configuration Repository is structured into one or more Deployment Sets. Each Deployment Set is placed into its own top-level sub-folder. This is further divided into two folders:

defaults: contains a hierarchy of datapoints with default values.
modes: contains datapoint values for one or more modes that each have one or more options that override the dynamic default values.

Therefore, the path structure for the default values follows:

/<deployment>/defaults/<component>/{static,dynamic}/<parameter>

The hierarchy under the defaults sub-folder is copied as is to the root of the Runtime Configuration Repository. Therefore, a datapoint /<deployment>/defaults/<component>/{static,dynamic}/<parameter> will be placed in /<component>/{static,dynamic}/<parameter> when the Runtime Configuration Repository is populated, assuming <deployment> is the current active Deployment Set.

The path structure for modes follows:

/<deployment>/modes/<name>/<option>/<component>/dynamic/<parameter>

Where each sub-folder under /<deployment>/modes is a separate mode. The mode name is given by the sub-folder name, i.e. <name>. Each mode should then have one or more options indicated by further sub-folders. Again, the name of the option corresponds to the sub-folder name, i.e. <option>. The hierarchy under the /<deployment>/modes/<name>/<option>/ path will be copied as is to the Runtime Configuration Repository if that mode option is selected, replacing any initial default values loaded from /<deployment>/defaults/.

Note

Only dynamic datapoints are allowed for modes. Static datapoints should not be provided.

The set of datapoints being configured by each option for a mode must match, i.e. meaning they must be the same for each option of a given mode.

These requirements are checked by the RTC Supervisor validation logic.

For each mode, the default option that should be loaded initially by RTC Supervisor is indicated by the following datapoints:

/<deployment>/modes/<name>/default_option

The value for each of these datapoints should be a string indicating one of the <option> sub-folders corresponding to the mode, i.e. one of /<deployment>/modes/<name>/<option>. If the default option is not set for the mode, then the mode is not activated when initially populating the Runtime Configuration Repository, i.e. no default values are replaced by that mode.

The default Deployment Set can optionally be indicated by the datapoint with the following path:

/default_deployment

This value is used by the RTC Supervisor as a fallback solution in case it cannot read /deployment/active_deployment from the Runtime Configuration Repository. The value of /default_deployment must be a string indicating the name of one of the top level Deployment Set folders, i.e. one of the <deployment> folder names.

Server Alias Table

Warning

The server alias table is now deprecated.

To speed up and optimize data transfer to and from the legagy CII OLDB based Runtime Repository, a mechanism was introduced that allows to pin individual datapoints to dedicated Ext. Redis instances that are deployed locally on specific SRTC nodes. The main idea of this mechanism is to improve latency and throughput by hosting large datapoints containing e.g. control matrices locally on the SRTC in close proximity to the producing and consuming processes.

Note

This feature can only be used if the legacy CII OLDB backend is select and configured such that the External Redis instances are configured, running, and known to the system. For correct CII OLDB configuration please refer to the CII OLDB documentation (especially the part about Ext. Redis configuration using the redisServersExternal setting).

The pinning of Runtime Configuration Repository datapoints to specific Redis instances can be configured by providing an optional server_alias_table for each deployment set in the Persistent Configuration Repository. The table maps a list of Persistent Configuration Repository datapoint paths to specific Ext. Redis instances, where each Ext. Redis instance is represented by a unique Server Alias string. During system initialisation the RTC Supervisor will then use this information to create the datapoints in the right location.

The Server Alias Tables support a form of pattern matching where individual datapoint leaf nodes are first enumerated explicitly, afterwards, more general datapoint paths are added to cover the remaining set of datapoints without having to enumerate them all one by one.

/server_alias_table:
    ops:
        defaults: !cfg.type::matrix2d_string
        [["/ops/defaults/common/dynamic/wfs_1/avg_slopes"  , "ext_redis_2"],
         ["/ops/defaults/common/dynamic/wfs_1/avg_modes"   , "ext_redis_2"],
         ["/ops/defaults/"                                 , "ext_redis_1"]]
    test:
        defaults: !cfg.type::matrix2d_string
        [["/test/defaults/"                                , "ext_redis_test"]]

The example above shows two different server alias tables for deployment sets ops and test. In ops datapoints avg_slopes and avg_modes are explicitly pinned to ext_redis_2 while all other datapoints are pinned to ext_redis_1. For deployment set test the allocation of datapoints is different, here all datapoints are pinned to ext_redis_test.

Note

The server alias tables may only contain datapoints from the defaults branch of the configuration. It is not allowed to change the server alias setting depending on the active operational mode. If such a need arises then a new deployment set should be created instead.

Service Discovery Endpoint

The currently supported URI schemes for the service discovery parameter persistent_repo_endpoint are:

cii.config: This will select the repository adapter that interfaces with the CII configuration service. Only local is supported for the hostname part of the URI. For example cii.config://local/rtc/config.

Data Access Using the API

Note

SRTC components should not access the Persistent Configuration Repository directly. Only a limited subset of components and tools should use the API to modify the Persistent Configuration Repository. See the section Modifying the Configuration for details about the typical manner to update the Persistent Configuration Repository.

The API for accessing datapoints is similar to the Runtime Configuration Repository. The differences are that the Persistent Configuration Repository does not support the datapoint subscription API nor the server alias setting, and the interface class to use is the PersistentRepoIf, which is fetched from the Service Discovery as follows:

auto oldb = m_services.Get<PersistentRepoIf>();

Refer to the Data Access sub-section of Runtime Configuration Repository for details about the methods for creating, reading, writing, deleting and querying datapoints.

Supported Data Types

The same data types are supported as for the Runtime Configuration Repository. Refer to the Supported Data Types sub-section of Runtime Configuration Repository for further details.

File Format

The YAML file format when using the Persistent Configuration Repository adapter with cii.config://local URI prefix (local config) is based on the CII configuration service.

A detailed description of the syntax and options available can be found in the CII Configuration manual. Here we only highlight important information that is specific to the RTC Toolkit.

Layout and Directory Structure

The Persistent Configuration Repository is mapped to a number of individual YAML documents in the CII configuration service. These files are encoded as Deployment Sets and placed into separate sub-directories. Specifically, the files are kept in the following directory structure on the filesystem:

<repodir>/<deployment>/defaults/<component>.yaml - The initial default datapoints for a Deployment Set are placed in a YAML file per component under a defaults directory.
<repodir>/<deployment>/modes/<name>/<option>/<component>.yaml - The datapoints related to a particular mode are placed in a YAML file per component under the indicated directory structure.
<repodir>/globals.yaml - Any datapoints that do not map to the above document locations are placed in this file.

Where <repodir> is simply the top-level filesystem directory where the repository is located. This would normally be configured through the CFGPATH environment variable. See the CII configuration service documentation for more details.

Each file under one of the defaults/ or modes/ sub-directories typically represents the configuration for a component instance. This correspondence depends on how the component’s logic is written and is therefore not enforced. However, if feasible, it is encouraged to maintain this correspondence in the code for simplicity.

Within each YAML file a hierarchical structure of YAML maps (dictionaries) is used for the remaining logical path components of a datapoint’s path. The leaf nodes then store the datapoint’s value and type information. For example, the datapoint path /depl1/defaults/datatask1/static/param1 maps to the file <repodir>/depl1/defaults/datatask1.yaml with the following contents:

static:
    param1: !cfg.type:string "abc"

As can be seen, the last path components of a datapoint’s logical path are exactly the dictionary key names within the YAML file.

For a more complete example of the mapping between datapoint paths and the values in the YAML files, consider the following datapoints:

/depl1/defaults/datatask1/static/param1
/depl1/defaults/datatask1/dynamic/param2
/depl1/defaults/datatask1/dynamic/matrix
/depl1/modes/mode1/opt1/datatask1/dynamic/param2
/depl1/modes/mode1/opt2/datatask1/dynamic/param2
/depl1/modes/mode1/default_option
/depl1/defaults/datatask2/static/settings/param_a
/depl1/defaults/datatask2/static/settings/param_b
/default_deployment

These would correspond to the following files and directories:

<repodir>/depl1/defaults/datatask1.yaml
<repodir>/depl1/defaults/datatask2.yaml
<repodir>/depl1/modes/mode1/opt1/datatask1.yaml
<repodir>/depl1/modes/mode1/opt2/datatask1.yaml
<repodir>/globals.yaml

The contents of these files would be as follows:

Listing 2 <repodir>/depl1/defaults/datatask1.yaml

 static:
     param1: !cfg.type:string "abc"
 dynamic:
     param2: !cfg.type:int64 0
     matrix: !cfg.type:matrix2d_single [[1.0, 2.0], [3.0, 4.0]]

Listing 3 <repodir>/depl1/defaults/datatask2.yaml

 static:
     settings:
         param_a: !cfg.type:int32 80
         param_b: !cfg.type:double 2.125

Listing 4 <repodir>/depl1/modes/mode1/opt1/datatask1.yaml

 dynamic:
     param2: !cfg.type:int64 1024

Listing 5 <repodir>/depl1/modes/mode1/opt2/datatask1.yaml

 dynamic:
     param2: !cfg.type:int64 2048

Listing 6 <repodir>/globals.yaml

 depl1:
     modes:
         mode1:
             default_option: !cfg.type:string "opt1"
 default_deployment: !cfg.type:string "depl1"

Note

The structure of the Persistent Configuration Repository can be checked with the rtctkConfigTool. See for example the Validate command section for details.

External URIs

It is possible to encode vectors or matrices as FITS files and refer to them via an external URI in the YAML, instead of embedding them into the YAML file directly. This can be particularly useful for large vectors or matrices, which can be handled more conveniently as FITS files.

Note

Only datapoints for numerical vectors and matrices are supported using external URIs.

For numerical vectors and matrices that are larger than the threshold of 16 elements, the API automatically writes these to FITS files instead of embedding them in the YAML files. Any vector or matrix smaller or equal to the threshold is written to the YAML file instead.

To change a datapoint from being encoded directly in the YAML, simply replace the type tag with the special type !cfg.type:uri and value with the external URI string. The uri type must be declared by adding the definition !cfg.typedef uri(string): once to the top of the file. The following is an example of the needed changes:

!cfg.typedef uri(string):

dynamic:
    matrix: !cfg.type:uri "file:matrix.fits"

The URI must use the file: scheme and must refer to a FITS file in the local filesystem.

The path can be a relative or absolute filesystem path. If a relative path is used then it is relative to the location of the YAML file in which the external URI is encoded. For example, if the YAML file’s path is /home/eltdev/repo/depl1/defaults/datatask1.yaml and the encoded external URI in the YAML is file:matrix.fits, then the path of the FITS file must be /home/eltdev/repo/depl1/defaults/matrix.fits. Alternatively, if the encoded external URI is file:subdir/matrix.fits, then the path of the FITS file must be /home/eltdev/repo/depl1/defaults/subdir/matrix.fits.

The format of the FITS file must be a 1-D or 2-D image in the primary HDU. Appropriate encoding of the NAXIS, BITPIX, BZERO and BSCALE keywords will determine the datapoint type according to the following table:

C++ Type	NAXIS	BITPIX	BZERO	BSCALE	BOOLITEM
`std::vector<bool>`	1	8	-128	1	“true”
`std::vector<int8_t>`	1	8	-128	1	(none)
`std::vector<uint8_t>`	1	8	(none)	(none)	(none)
`std::vector<int16_t>`	1	16	(none)	(none)	(none)
`std::vector<uint16_t>`	1	16	32768	1	(none)
`std::vector<int32_t>`	1	32	(none)	(none)	(none)
`std::vector<uint32_t>`	1	32	2147483648	1	(none)
`std::vector<int64_t>`	1	64	(none)	(none)	(none)
`std::vector<uint64_t>`	1	64	9223372036854775808	1	(none)
`std::vector<float>`	1	-32	(none)	(none)	(none)
`std::vector<double>`	1	-64	(none)	(none)	(none)
`MatrixBuffer<bool>`	2	8	-128	1	“true”
`MatrixBuffer<int8_t>`	2	8	-128	1	(none)
`MatrixBuffer<uint8_t>`	2	8	(none)	(none)	(none)
`MatrixBuffer<int16_t>`	2	16	(none)	(none)	(none)
`MatrixBuffer<uint16_t>`	2	16	32768	1	(none)
`MatrixBuffer<int32_t>`	2	32	(none)	(none)	(none)
`MatrixBuffer<uint32_t>`	2	32	2147483648	1	(none)
`MatrixBuffer<int64_t>`	2	64	(none)	(none)	(none)
`MatrixBuffer<uint64_t>`	2	64	9223372036854775808	1	(none)
`MatrixBuffer<float>`	2	-32	(none)	(none)	(none)
`MatrixBuffer<double>`	2	-64	(none)	(none)	(none)

Entries indicated as (none) in the above table mean that the keyword does not need to be present in the FITS file header.

Limitations and Known Issues

When using the Persistent Configuration Repository backed by the CII configuration service, i.e. using the cii.config://local URI scheme, the following limitations apply:

The adapter for the CII configuration service still needs optimisation and may not currently be very performant.
It is not possible to use a datapoint path that is also the directory name for another datapoint, i.e. a datapoint path cannot be the exact string prefix of another datapoint’s path. For example, the following two paths cannot be used at the same time: /depl1/defaults/dt1/static/sub and /depl1/defaults/dt1/static/sub/.
There is no locking mechanism implemented in the CII configuration service. Therefore, users must avoid having one or more processes attempting to read from the Persistent Configuration Repository while it is being modified.