Data Task

Overview

As described in the design document, Data Tasks are RTC Components that perform custom computations on data sets derived from Telemetry and/or Runtime Configuration data.

Due to the fact that input data, computation and output data are Data Task and instrument-specific, it is not possible for the RTC Tk to deliver a one-size-fits-all Data Task component that covers the entire spectrum of required calculation and that can be customised by only modifying configuration. Therefore, instrument RTC developers will have to implement the bulk of the Data Task code.

To make the implementation of the concrete Data Task application easier, the RTC Tk provides supporting libraries and a reference structure for Data Tasks.

Prerequisites

Depending on the actual Data Task flavour, some of the following dependencies may be required:

• From roadrunner:

  • NUMA++ - C++ libnuma interface.

  • ipcq - Interprocess communication over shared memory.

• CUDA toolkit - For GPU acceleration and GPU drivers, see GPU support for more information.

• openblas - Math library.

• Python - See Python support for information.

Customisation

Custom Data Tasks are implemented by instrument RTC developers, they have to write the bulk of the code themselves but they are encouraged to make their life easier by re-using the provided support libraries.

The tutorials Creating a Simple RTC Component and Creating a Custom Data Task show concrete examples on how to implement custom RTC Components and Data Tasks.

The reusable Data Task libraries can be found in reusableComponents/dataTask and the examples of implementation can be found in the _examples/exampleDataTask.

Telemetry-based Data Tasks

This section provides a high level overview of a simple, telemetry-based Data Task.

Structure

In the figure below the structural outline of a Data Task component is shown. The Runnable<RtcComponent> life-cycle is implemented in the BusinessLogic class, which in turn owns and controls a Computation object. The diagram also provides a non-exhaustive overview of the most important API methods.

The BusinessLogic class is the glue that holds the Data Task together. It provides life-cycle methods according to the states and state transitions of the RTC Component. By implementing these methods with calls to the Computation class it is possible to create custom Data Tasks.

The RTC Tk provided ComputationBase template class retrieves Telemetry Data samples from the shared memory queue using a configurable read/skip pattern. It makes use of the roadrunner::ipcq library and it hosts a worker thread that invokes user-defined callback methods in the derived Computation class. Additionally, the class also publishes pre-defined statistics using the ComponentMetrics service.

The derived Computation class defines the shared memory TopicType and it implements virtual methods for the concrete computation. Here users will spend the majority of their efforts implementing a specific computation algorithm. The main focus is on the CopyData callback for retrieving data from a shared memory queue and on the Compute and Publish callbacks that are triggered once enough samples have been processed.

Note

It is up to the user whether to publish the computation result directly in the computation class Publish callback method or to delegate result publication to a method that is implemented somewhere else, e.g. in the BusinessLogic class.

Behaviour

As part of the standard RTC Component functionality, the Data Task application receives commands from other external entities and invokes the methods of the BusinessLogic class accordingly.

The BusinessLogic class interfaces with the Computation class by invoking methods that control its configuration and state. The BusinessLogic also interfaces with the Runtime Configuration Repository and OLDB to allow loading of configuration data and publication of monitoring data.

Continuous Computations

Most of the Data Task perform continuous computations. This means, once the Data Task is Running it repeatedly performs read/compute/skip cycles until it is commanded to stop.

The sequence diagram above shows the basic interaction with the Computation class and its worker thread.

• In state Initialising the Computation object is created.

• In state Enabling the worker thread is spawned and user-callback OnThreadStart is invoked.

• In state Running the worker thread performs read/compute/skip cycles and invokes user-implemented computation class methods OnCycleStart, CopyData, Compute and Publish. The BusinessLogic can use inspection methods to retrieve the current status of the computation object.

• In state Disabling the worker thread is joined and terminated.

Note

Avoid doing heavy work in callback methods OnThreadStart, OnCycleStart and CopyData, these methods are supposed to be short and light-weight.

On-demand Computations

Some Data Tasks perform on-demand or one-shot computations. Here the Data Task is explicitly commanded, e.g with a Measure or Optimise command, to perform a single read/compute cycle.

The sequence diagram above shows the basic interaction sequence with the Computation class and its worker thread.

In the Measuring state, the method RunOnceSync is used to command the worker thread to perform a single read/compute cycle. Once the cycle is completed the Computation class will automatically enter its Idle state where it continuously resets the SHM read queue.

Configuration

The following configuration attributes are accepted by the ComputationBase class constructor:

cfg attribute	Type	Description
queue_name	RtcString	Name of the shared memory queue.
samples_to_read	RtcUInt32	Number of samples to read.
samples_to_skip	RtcUInt32	Number of sample to skip.
sample_timeout	RtcUInt32	Timeout for receiving a single sample [ms].
thread_policies	NUMA Policies	(optional) NUMA policies for the worker-thread.

Note

Samples_to_read and samples_to_skip can also be set dynamically via dedicated setters.

Note

The configuration schema (YAML file layout) of the actual Data Task is user-specific.

Errors

During component initialisation the main errors that can occur are caused by passing invalid configuration parameters.

Once the worker thread is spawned, asynchronous errors may occur that indicate:

• Problems connecting to the shared memory queue:

  • Telemetry Subscriber is not running or could not create the SHM segment.

• Problems reading data from shared memory queue:

  • Queue underflow causes read/skip timeout.

  • Queue overflow causes inconsistent queue state.

• Problems executing user function for computation:

  • User computation throws an exception.

  • User computation takes too long and causes a SHM queue overflow.

• Problems executing user function for result publication:

  • Result publication throws an exception.

  • Result publication takes too long and causes a SHM queue overflow.

• Problems executing other user functions:

  • User functions throw exceptions.

Other Data Task Flavours

There are also other flavours of Data Tasks that do not use telemetry data as an input. These flavours also do not use the toolkit-provided ComputationBase class.

Mostly, such Data Tasks are configuration-based, meaning they perform their computations on input data from the Runtime Configuration Repository either periodically, on-change or on-demand.

To facilitate implementation of such Data Tasks, the following interfaces are available:

• Alternative component live-cycles providing hooks for:

  • ActivityOptimising()

  • ActivityMeasuring()

• Runtime Repository Adapter

  • To read and write specific data points.

• Event Channel

  • To publish and subscribe to custom events.

See RTC Component for more information about using these interfaces.

Limitations and Known Issues

• Multiple calls to ComputationBase::Spawn can lead to an incorrect state or abort (RTCTK-1265).