Python support

Overview

Data Task components are envisioned to support Python to simplify the development of complicated algorithms that are being used to support the adaptive optics loop. Some of these algorithms are very large and complicated with very tight temporal requirements. Others may be complicated but have slightly looser temporal requirements. For these later calculations, Python can be very powerful and allow the computation to take advantage of the cleaner syntax of Python with many mature numerical libraries.

The method we have chosen to investigate is using pybind11 (see section Using Provided Python Bindings).

Source Code Location

The example source code can be found in the following sub-directory of the rtctk project:

_examples/exampleDataTask/optimiser

Modules

The provided example is composed by the following waf modules:

  • app - The data task application including default configuration.

  • pyLib - Python library containing calculation.

  • scripts - Helper scripts for deployment and control.

In addition, the example uses the following component framework libraries:

  • rtctkRtcComponent - For RtcComponent functionality

  • rtctkServicesPython - Framework-provided Python bindings for C++

Running the Example

The optimiser example can be run in isolation or as part of an SRTC system. The Python computation can be invoked after bringing the component to state Operational. By sending the Optimise command with a specific JSON payload, the Python interpreter is started and the computation is carried out.

The script rtctkExampleDataTaskOptimiser.sh is provided to bring the example data task and the supporting infrastructure online in a simple way.

rtctkExampleDataTaskOptimiser.sh

After installing the RTC Tk, run the example using the following sequence of commands:

# Deploy and start the example applications
rtctkExampleDataTaskOptimiser.sh deploy
rtctkExampleDataTaskOptimiser.sh start

# Use the client to step through the life-cycle of the respective component instance
rtctkExampleDataTaskOptimiser.sh send Init
rtctkExampleDataTaskOptimiser.sh send Enable
rtctkExampleDataTaskOptimiser.sh send Optimise '{"algorithm":"PythonInversion"}'
rtctkExampleDataTaskOptimiser.sh send Disable
rtctkExampleDataTaskOptimiser.sh send Reset

# Gracefully terminate the applications and clean-up
rtctkExampleDataTaskOptimiser.sh stop
rtctkExampleDataTaskOptimiser.sh undeploy

Using Provided Python Bindings

The RTC Toolkit library rtctkServicesPython provides functionality for passing vectors and matrices between C++ and Python code without internal data being copied. To use the framework-provided Python bindings, link against this library and include the headers shown below.

#include <rtctk/componentFramework/matrixBufferPy.hpp>
#include <rtctk/componentFramework/vectorPy.hpp>

Note

Currently, we only support no copy passing of data in Vectors and MatrixBuffers formats. (see Runtime Configuration Repository).

Before Python can be used in a Data Task application, it needs to be set up accordingly. This setup can either be done once in the constructor of the computation class or on demand before the respective Python computation starts.

py::gil_scoped_acquire gil;
auto py_compute_module = py::module::import("example_lib");
auto py_inversion = py_compute_module.attr("inversion");

In the first line, the Python interpreter’s GIL is locked. Then, the bindings for the MatrixBuffer class are loaded. In the last line a handle to the actual computation is retrieved.

Note

To allow for easy swapping of Python modules, the names of the modules and functions can also be retrieved from configuration. This may provide additional flexibility and may eliminate the need for recompilation.

To run the Python computation, the previously retrieved function handle is invoked with references to the data buffers of the input and output data as the provided arguments.

py_inversion(&m_IM, &m_CM);

See section Python Support in the Component Framework for reference and more guidelines.