Standard Adapters

The CCF shall be considered as an SDK, used to develop DCS solutions. It is therefore considered that for specific CCF instances, it will be necessary to develop one or more adapters of various types. This is typically Communication Adapters, to support a specific camera properties.

However, some adapters considered of common interest, will be provided by the CCF Package. It may be possible to use these, or to use them as base classes for new adapters, adjusted to the specific use cases.

In general it is recommended to attempt to use camera models already in use and to attempt to re-use the Standard Adapter provided by the CCF Package. Developers are encouraged to contact ESO to obtain information about how to reuse existing components rather than implementing new ones.

If developers identify adapters that might be of general interest, they are encouraged to design/develop these in a way that they can be re-used and to propose these for inclusion in the CCF Package.

The Standard Adapters of the three categories 1. Communication Adapters, 2. Processing Recipe Adapters, and 3. Data Publisher Adapters, are described in the following sections.

Standard Communication Adapters

In this release, the following Standard Communication Adapters are provided:

GenICam Communication Adapter - “ifw::ccf::protocols::genicam::ComAdptGenICam”: The recommended adapter for GenICam / GigE Vision cameras. It is a bridge onto the camcom GenICam adapter, keeping the camera SDK confined inside camcom.

Aravis SDK Communication Adapter - “ifw::ccf::protocols::aravis::ComAdptAravis”: Deprecated — based on the Aravis SDK, implementing the GigE Vision protocol. Superseded by the GenICam Communication Adapter (see below).

RTMS Communication Adapter - “ifw::ccf::protocols::rtms::ComAdptRtms”: Communication Adapter capable of receiving image frames via the RTMS protocol. This adapter does not provide features for controlling the image source (camera).

ENVision Communication Adapter - “ifw::ccf::protocols::envision_llnetio::ComAdptEnvision”: This adapter provides an interface for controlling the camera and retrieving status information, based on OPC UA. The data is received via the RTMS protocol.

Simulation Communication Adapter - “ifw::ccf::control::ComAdptSim”: This provides in-application simulation. For now only play-back of an existing FITS file, typically a FITS cube, is provided.

These adapters are described in more detail in the following sections.

GenICam Communication Adapter

The GenICam Communication Adapter (“ifw::ccf::protocols::genicam::ComAdptGenICam”) is the recommended way to communicate with GenICam / GigE Vision cameras. It supersedes the (deprecated) Aravis adapter.

It is implemented as a thin bridge: it implements the CCF Communication Adapter contract (“ccf::common::ComAdptBase”) by delegating to camcom’s GenICam adapter (“camcom::genicam::GenICamAdapter”). The camera SDK (Aravis) stays confined inside camcom; the CCF adapter only translates between the CCF and camcom parameter/frame representations. This keeps the camera-protocol handling in one shared, separately-tested place (camcom) rather than re-implemented per framework.

The control application is “ccfCtrlGenICam”.

The device address may be given in any of the following forms (in precedence order):

a host name, resolved via DNS (recommended — stable across DHCP), e.g. “tccd001.hq.eso.org”;

a literal IP address, e.g. “192.0.2.10”;

the GenICam device id (vendor-model-serial), e.g. “Allied Vision Technologies-Mako G-158B (12750)-50-0536990174”.

An explicit address selects the right device when more than one camera is on the network. A literal IP is used as-is; a host name is resolved to an IP (IPv4 preferred); a value that is neither (e.g. a device id) is passed through to the SDK.

A device configuration entry looks as follows:

devices:                   [
  {
    <standard parameters removed>
    protocol:              'GigE Vision',
    mapping:               'config/ifw/ccf/protocols/genicam/genicam.mapping.yaml',
    sdk:                   'GenICam',
    address:               'tccd001.hq.eso.org',   # host name, IP, or device id
    sim_address:           'InternalSimulator',
    properties:            [
      { name: 'PixelFormat', value: 'Mono12', write_on_init: true }
    ]
  }
]

The Camera Name Mapping (“genicam.mapping.yaml”) maps the CCF internal parameter names to the camera-native GenICam (SFNC) names (e.g. “expo.time” -> “ExposureTime”, “expo.bin_x” -> “BinningHorizontal”). The provided production mapping uses the current SFNC names; adjust it per camera model as needed.

A complete example deployment (config + init setup) for a real AVT Mako G-158B camera is provided in the CCF Package under “protocols/genicam/exe/resource/config/ifw/ccf/protocols/genicam/” (see also “Installation & Deployment”).

The camcom layer logs through the ifw-fnd “FND*” logging API; “ccfCtrlGenICam” installs the FND-to-log4cplus bridge (provided by ifw-core) at startup so that camcom’s diagnostics flow into the CCF log4cplus log under the “camcom” logger (configurable in the log properties file), alongside the rest of the CCF logs.

Aravis (GigE Vision) Communication Adapter

Warning

The Aravis Communication Adapter (“ccfCtrlAravis”) is DEPRECATED and will be removed in a future release. Use the GenICam Communication Adapter (“ccfCtrlGenICam”, see above) instead. “ccfCtrlAravis” prints a deprecation warning at startup.

This adapter is based on the GenICam standard and provides support for the GigE Vision communication protocol. Even though Aravis is a GenICam implementation, the CCF Aravis Communication Adapter only supports the GigE Vision protocol. Moreover, the present implementation is not a ‘true’ generic GigE Vision adapter; there are a few camera specific features implemented in the adapter and in general, there will usually always be differences between the various cameras to deal with

The Camera Name Mapping feature provided by CCF, may be capable of coping with at least simple differences in the parameter namespaces.

To use the CCF Aravis Communication Adapter to communicate with a camera with a GigE Vision interface, a first quick test may be carried out:

Connect and configure the camera according to the camera user’s guide.

Launch the example Aravis Communication Adapter executable, “ccfCtrlAravis”. The IP address in the configuration shall be set to the allocated IP address of the camera (refer to “Installation & Deployment” for information about how to execute a CCF Control instance). The “server.simulation” parameter in the configuration should be set to “false”.

Bring CCF Control to “Operational::Idle” by submitting an “stdif::Init” and “stdif::Enable” request.

If this was successful, start the image acquisition by issuing a “dcsif::Start” request.

If executing these commands are successfully executed, it could be attempted to display frames in the DDT Viewer (see “Installation & Deployment”).

If issues are encountered, which seem related to the invalid names, it may be attempted to provide a camera specific Name Mapping; an example can be found within the CCF Package: “config/ccf/protocols/aravis/avt.mapping.yaml”.

If this does not work, it will probably be necessary to implement a dedicated Communication Adapter for the camera in question.

Minor modifications of general interest may be accepted for the CCF Aravis Standard Communication Adapter.

RTMS Communication Adapter

The Communication Adapter can be used to receive RTMS packages from the connected network and have these published, e.g. in FITS files or in the DDT.

The adapter does not provide support for controlling the image source, nor to read the status from it. I.e., it’s a passive subscriber to the data stream.

In order to use this adapter, the specific properties “IpAddress”, “Port”, “NetworkInterface”, “DataType” and “QueueSize”, must be specified in the configuration, e.g.:

devices:                   [
  {
    name:                  'TestSimCamera1',
    <standard parameters removed>
    properties:            [
      {
        name:              'IpAddress',
        value:             '127.0.0.1'
      },
      {
        name:              'Port',
        value:             '59000'
      },
      {
        name:              'NetworkInterface',
        value:             ''
      },
      {
        name:              'DataType',
        value:             'Int16'
      },
      {
        name:              'QueueSize',
        value:             '10'
      }
    ]
  }
]

Please consult the documentation for the RTMS protocol for specific information about this.

ENVision Communication Adapter

The ENVision Communication Adapter uses internally the RTMS Communication Adapter to receive the data.

In addition to receiving the data, it also provides control and status querying of the associated device, via an OPC UA interface.

This RTMS/OPC UA interface is referred to as “ENVision”.

No formal (released) ICD document exists yet, documenting the protocol.

The ENVision Protocol is defined primarily for the “Software Frame Grabber” toolkit (“ifw-fgf”), which is used to integrate SW frame grabber solutions for specific cameras.

The adapter class is “ifw::ccf::protocols::envision_llnetio::ComAdptEnvision” and the control application is “ccfCtrlEnvision_llnetio”.

Note

Two builds of the ENVision adapter currently exist: the “_llnetio” build (adapter “ifw::ccf::protocols::envision_llnetio::ComAdptEnvision”, application “ccfCtrlEnvision_llnetio”), which is the one to use, and an older build (“ifw::ccf::protocols::envision::ComAdptEnvision”, “ccfCtrlEnvision”) retained only for backward compatibility. The older build will be removed in IFW27, at which point the “_llnetio” build becomes the sole ENVision adapter and its name loses the “_llnetio” suffix (application “ccfCtrlEnvision”, adapter namespace “ifw::ccf::protocols::envision”). New deployments should target the “_llnetio” build so the later rename is the only change required.

The configuration for the ENVision Communication Adapter is as follows:

devices:                   [
  {
    <standard parameters removed>
    protocol:              'ENVision',
    mapping:               'config/ifw/ccf/protocols/envision_llnetio/envision.mapping.yaml',
    sdk:                   'OPCUA/RTMS',
    address:               'opc.tcp://127.0.0.1:4840',
    sim_address:           'InternalSimulator',
    properties:            [
      {
        name:              'OpcUaObject',
        value:             'Andor'
      },
      {
        name:              'OpcUaNs',
        value:             '2'
      },
      {
        name:              'IpAddress',
        value:             '127.0.0.1'
      },
      {
        name:              'Port',
        value:             '59000'
      },
      {
        name:              'NetworkInterface',
        value:             ''
      },
      {
        name:              'DataType',
        value:             'Int16'
      },
      {
        name:              'QueueSize',
        value:             '10'
      }
    ]
  }
]

Simulation Communication Adapter

This adapter provides in-application simulation. This means that the Acquisition Thread, executing the Communication Adapter “Receive” method, may have a higher CPU load compared to if the simulation is implemented in an external process.

The simulation provided in this release, is based on playing back the image frame contained in a FITS image cube.

The simulation respects the exposure time, but frame rate and windows parameters are not taken into account in this release. This will be added in a future release as well as other types of simulation, based on generating artificial data. Latter means it is not necessary to provide a FITS file to be used as input for the simulation, but predictable simulated images can be generated for test purposes.

A small tool, “ccfFitsCubeGen”, is provided to generate a simple FITS cube to be used for the simulator.

Standard Processing Recipe Adapters

The following Standard Processing Recipes are provided:

Centroiding Recipe (“ifw::ccf::stdrecipe::RecipeCentroid”).

Average Recipe (“ifw::ccf::stdrecipe::RecipeAverage”).

The provided Standard Recipes are based on the ICS ODP Component, which again is based on CPL.

Centroiding Recipe

The Centroiding Recipe finds the coordinates of an object, located in the center of the given images, within a certain window. To actually find the centroid, it uses internally the function “clipm_centroiding_gauss()”.

The Configuration Parameters are as follows (example):

proc1.recipe1.adapter:       "ifw::ccf::stdrecipe::RecipeCentroid"
proc1.recipe1.name:          "TestCentroidRecipe"

The Setup Parameters are as follows (example):

ProcThread1:
  enabled:                       true

  recipe_1_1_centroid:
    enabled:                     true
    max_centre_error:            0.5
    max_sigma_error:             3
    robustness:                  7

Parameter	Description
enabled (Boolean)	Enables/disables the recipe.
max_centre_error (Double)	Maximum error from the center to consider the search for the centroid as successful.
max_sigma_error (Double)	Maximum sigma error of the fitting of the Gaussian.
robustness (Integer)	Number of iterations for obtaining the centroid accurately.

Note, the size of the central window in which the location of the centroiding is executed, is computed as (lower left, upper right) corners of window: “(((width / 4), (width / 4) + (width / 2)), (((height / 4), (height / 4) + (height / 2)))”.

The OLDB status keys are of the form: “<server ID>/<CCF instance name>/status/pipelines/<pipeline name>/recipes/<recipe name>.<key>”.

The keys are:

Parameter	Description
sigma_error_x/y (Double)	Output sigma error in X/Y.
centre_error_x/y (Double)	Output centre error X/Y.
centre_intensity (Double)	Intensity (value) of centre pixel.
centre_x/y (Double)	X/Y coordinate of centre pixel.
error (Boolean)	If true, indicates that the centroid searching failed.
error_msg (String)	Error message (if available), in case an error occurred.
max_centre_error (Double)	Maximum centre error specified in the configuration. If the error is above, this will lead to failure of recipe.
max_sigma_error (Double)	Maximum sigma error specified in the configuration. If the error is above, this will lead to failure of recipe.
last_update (Double)	Time since epoch of last update.
probe_window (String)	Status of probe window used ((x1, y1), (x2, y2)).
robustness (Integer)	Maximum number of retries.

Average Recipe

The Average Recipe (“ifw::ccf::stdrecipe::RecipeAverage”) computes a running average of the incoming image frames over a sliding window. When the window is full, the oldest frame is evicted as each new frame arrives, so the published frame is the average of the last “window_size” frames.

The Setup Parameters are as follows (example):

ProcThread2:
  enabled:                       true

  recipe_2_1_average:
    enabled:                     true
    window_size:                 31

Parameter	Description
enabled (Boolean)	Enables/disables the recipe.
window_size (Integer)	Number of most-recent frames over which the average is computed (sliding window). The averaging starts producing a full-window result once this many frames have been received.

Standard Data Publisher Adapters

In this release, three Standard Data Publisher Adapters are provided:

FITS File Publisher.

DDT Publisher.

RTMS Publisher.

These are described in the following.

FITS File Publisher

An example of the parameters for the FITS Publisher is shown in the following:

ProcThread1:
  enabled:                       true

  FitsPublisher:
    enabled:                     true
    basename:                    'PubFitsTest1'
    max_size:                    100      # MB
    nb_of_frames:                7
    format:                      'Single' # Single, Cube, CubeX, MEF
    overwrite:                   true
    rec_mode:                    'All'    # All, I:<nb>, P:<period> - not supported.
    add_timestamps_tbl:          'None'   # None, Binary, ASCII
    nb_of_hdr_blocks:            1

The specific Setup Parameters provided for the adapter are:

Parameter	Description
basename (String)	Basename used for generating the output filenames. The names are of the form: “<proc#.pub#.basename>[_<ISO 8601>].fits”, where the ISO is prepended to make the name unique when “proc#.pub#.overwrite” is false.
max_size (Integer)	Indicates the maximum volume in MB of the output data generated during a Recording Session.
nb_of_frames (Integer)	Indicates the maximum number of image frames to be handled during a Recording Session.
format (String)	Format of the Output Data Product files. Valid options are: Single, Cube, CubeX, MEF. Single: one frame per FITS file. Cube: all frames of the session in one 3-D FITS cube. CubeX: cube with each frame in its own image extension. MEF: multi-extension FITS (one extension per frame).
overwrite (Boolean)	If set to “true”, the same output filename is used, which is basically the specified “proc#.pub#.basename”. This is used for test purposes to avoid generating a lot of files on disk.
rec_mode (String)	The recording mode. Valid options are: 1: All: All frames are recorded. 2: I:<#>: Every ‘#’th frame is taken. 3: P:<period>: A frame every <period> second is recorded. In this release, only “1: All” is supported.
add_timestamps_tbl (String)	Whether to append a per-frame timestamps table to the data product, and in which form. Valid options are: None, Binary, ASCII.
nb_of_hdr_blocks (Integer)	Number of FITS header blocks to reserve in the primary header (to leave room for metadata added later).

The target directory for the generated output files is defined as “$DATAROOT/dcs/<cfg:server_id>”.

The FITS Data Publisher is not responsible for any ‘cleaning up’ such that all output files generated, must be removed by the client/user. When ‘overwrite mode’ is used, however, the same file is always written.

DDT Publisher

The DDT Publisher publishes data into the DDT data handling infrastructure to allow for distributed access and for displaying in the DDT image widgets.

When using the CCF DDT Publisher, it is important to ensure that the properly configured instance of the DDT Broker is running, otherwise CCF Control will not start.

The DDT system as such, is not documented here. Refer to the DDT user’s manual for further information.

An example of the parameters for the DDT Publisher is shown in the following:

ProcThread1:
  enabled:                       true

  pub_1_1_ddt:
    enabled:                     true
    ddt_id:                      'CcfTest11'
    ddt_broker:                  'zpb.rr://127.0.0.1:12011/broker/Broker1'
    ddt_max_rate:                1000

The specific Setup Parameters provided for the adapter are:

Parameter	Description
ddt_id (Integer)	Name of the DDT stream used for publishing the data.
ddt_broker (Integer)	The URI of the DDT Broker to use when publishing. Example: “zpb.rr://127.0.0.1:11011/broker/Broker1”.
ddt_max_rate (Double)	Can be used to limit the rate with which data is published in the DDT system, to avoid overloading the DDT and the execution host.

RTMS Publisher

The RTMS Publisher publishes data on the specified network in the MUDPI/RTMS protocol. This is defined for transferring the data (realtime) from the WFS camera to the Realtime Computer, usually via a deterministic network.

The MUDPI and RTMS protocols are documented elsewhere and the format and functioning will not be described in detail in this manual.

An example of the parameters for the RTMS Publisher is shown in the following:

ProcThread1:
  enabled:                       true

  RtmsPublisher:
    enabled:                     true
    id:                          'RtmsPub'
    max_rate:                    220
    bpp:                         2
    interface:                   ''
    address:                     '127.0.0.1'
    port:                        59000
    topic_id:                    10
    sample_id:                   500
    pkt_send_delay:              0.001

The specific Setup Parameters provided for the adapter are:

Parameter	Description
id (String)	ID allocated to the publisher.
max_rate (Double)	Maximum rate, in Hz, with which to publish data. Setting this lower than the image acquisition rate may result in frame skipping.
bpp (Integer)	Bytes per pixel.
interface (String)	Name of the network interface to send on (e.g. “eth0”). Empty selects the default interface.
address (String)	IP address of network interface on which to broadcast the UDP packets.
port (Integer)	Port used for the UDP socket.
topic_id (Integer)	Used by RTMS to identify the type of data in the packets.
sample_id (Integer)	Used to tag the images (samples) from this source, for identification purposes.
pkt_send_delay (Double)	Delay, in seconds, between sending each UDP packet. May be used if the publisher is ‘too fast’ for the RTMS subscriber to be able to receive all UDP packets.

Note: Using multicast, may saturate the organisation network infrastructure and shall normally only be used on isolated networks. For development purposes, the loopback interface on the development host, can be used to avoid that the packet leave the development machine.