Processing framework

Data Structures

struct  muse_processing_framecounter
struct  muse_processing

Typedefs

typedef struct
muse_processinginfo_s 
muse_processinginfo_t

Enumerations

enum  muse_frame_mode {
  MUSE_FRAME_MODE_ALL = 0, MUSE_FRAME_MODE_MASTER = 1, MUSE_FRAME_MODE_DATEOBS = 2, MUSE_FRAME_MODE_SUBSET = 3,
  MUSE_FRAME_MODE_SEQUENCE = 4
}
enum  muse_table_type { MUSE_TABLE_TYPE_CPL = 0, MUSE_TABLE_TYPE_PIXTABLE }
enum  muse_cube_type {
  MUSE_CUBE_TYPE_EURO3D = 0, MUSE_CUBE_TYPE_FITS, MUSE_CUBE_TYPE_EURO3D_X, MUSE_CUBE_TYPE_FITS_X,
  MUSE_CUBE_TYPE_LSF, MUSE_CUBE_TYPE_SDP
}

Functions

muse_processingmuse_processing_new (const char *aName, cpl_recipe *aRecipe)
 Create a new processing structure.
void muse_processing_delete (muse_processing *aProcessing)
 Free the muse_processing structure.
void muse_processing_append_used (muse_processing *aProcessing, cpl_frame *aFrame, cpl_frame_group aGroup, int aDuplicate)
 Add a frame to the set of used frames.
cpl_frame * muse_processing_new_frame (muse_processing *aProcessing, int aIFU, cpl_propertylist *aHeader, const char *aTag, cpl_frame_type aType)
 Create a new frame for a result file.
int muse_processing_save_image (muse_processing *aProcessing, int aIFU, muse_image *aImage, const char *aTag)
 Save a computed MUSE image to disk.
cpl_error_code muse_processing_save_cube (muse_processing *aProcessing, int aIFU, void *aCube, const char *aTag, muse_cube_type aType)
 Save a MUSE datacube to disk.
cpl_error_code muse_processing_save_table (muse_processing *aProcessing, int aIFU, void *aTable, cpl_propertylist *aHeader, const char *aTag, muse_table_type aType)
 Save a computed table to disk.
int muse_processing_save_cimage (muse_processing *aProcessing, int aIFU, cpl_image *aImage, cpl_propertylist *aHeader, const char *aTag)
 Save a computed FITS image to disk.
cpl_error_code muse_processing_save_header (muse_processing *aProcessing, int aIFU, cpl_propertylist *aHeader, const char *aTag)
 Save a FITS header to disk.
int muse_processing_save_mask (muse_processing *aProcessing, int aIFU, muse_mask *aMask, const char *aTag)
 Save a computed MUSE mask to disk.
cpl_boolean muse_processing_check_intags (muse_processing *aProcessing, const char *aTag, int aNChars)
 Check that a tag is part of the input tags of a processing structure.
cpl_error_code muse_processing_check_input (muse_processing *aProcessing, unsigned char aIFU)
 Check the input files for completeness.
cpl_table * muse_processing_sort_exposures (muse_processing *aProcessing)
 Sort input frames (containing lists of pixel table filenames) into different exposures.
muse_maskmuse_processing_mask_load (muse_processing *aProcessing, const char *aTag)
 Load a mask file and its FITS header.
static muse_processinginfo_tmuse_processinginfo_get (const cpl_recipe *aRecipe)
 Get processinginfo for a certain recipe.
void muse_processinginfo_register (cpl_recipe *plugin, cpl_recipeconfig *recipeconfig, muse_processing_prepare_header_func *prepare_header, muse_processing_get_frame_level_func *get_frame_level, muse_processing_get_frame_mode_func *get_frame_mode)
 Register extended functionalities for MUSE recipes.
void muse_processinginfo_delete (cpl_recipe *aRecipe)
 Clear all information from the processing info and from the recipe config.
cpl_error_code muse_processing_prepare_header (const cpl_recipe *aRecipe, const char *aFrametag, cpl_propertylist *aHeader)
 Prepare and check a FITS header for a certain frame tag.
cpl_frame_level muse_processing_get_frame_level (const cpl_recipe *aRecipe, const char *aFrametag)
 Get the level for a product frame with a certain tag.
int muse_processing_get_frame_mode (const cpl_recipe *aRecipe, const char *aFrametag)
 Get the mode for a product frame with a certain tag.
cpl_recipeconfig * muse_processing_get_recipeconfig (cpl_recipe *aRecipe)
 Get the recipe (frame) configuration.
cpl_error_code muse_processing_prepare_property (cpl_propertylist *aHeader, const char *aName, cpl_type aType, const char *aDescription)
 Prepare and check the specified property.
void muse_processing_recipeinfo (cpl_plugin *aPlugin)
 Output main pipeline configuration, inputs, and parameters.

Variables

static muse_processinginfo_tmuse_processinginfo = NULL

Detailed Description

The following functions are used to make the data processing in MUSE easier and uniform.


Typedef Documentation

typedef struct muse_processinginfo_s muse_processinginfo_t

Structure to hold the extended processing information.


Enumeration Type Documentation

Cube format types supported by the MUSE pipeline.

Enumerator:
MUSE_CUBE_TYPE_EURO3D 

Euro3D format

MUSE_CUBE_TYPE_FITS 

FITS NAXIS=3 format with 2 extensions

MUSE_CUBE_TYPE_EURO3D_X 

Euro3D format with reconstructed image(s) * in extra FITS extensions

MUSE_CUBE_TYPE_FITS_X 

FITS NAXIS=3 format with 2 extensions plus * reconstructed image(s) in extra extensions

MUSE_CUBE_TYPE_LSF 

MUSE LSF cube

MUSE_CUBE_TYPE_SDP 

MUSE FITS cube with ESO SDP headers

Definition at line 73 of file muse_processing.h.

Frames modes used in the MUSE pipeline.

Enumerator:
MUSE_FRAME_MODE_ALL 

any frame that is not further specified

MUSE_FRAME_MODE_MASTER 

the one combined master frame of this tag

MUSE_FRAME_MODE_DATEOBS 

one of the output frames of this type, * sorted based on observing dates

MUSE_FRAME_MODE_SUBSET 

build header using a subset of the frames * but keep an indexed filename

MUSE_FRAME_MODE_SEQUENCE 

sequence of master files, all based on * all inputs

Definition at line 55 of file muse_processing.h.

Table format types supported by the MUSE pipeline.

Enumerator:
MUSE_TABLE_TYPE_CPL 

Normal FITS tables handled by CPL

MUSE_TABLE_TYPE_PIXTABLE 

MUSE pixel table

Definition at line 67 of file muse_processing.h.


Function Documentation

void muse_processing_append_used ( muse_processing aProcessing,
cpl_frame *  aFrame,
cpl_frame_group  aGroup,
int  aDuplicate 
)

Add a frame to the set of used frames.

Parameters:
aProcessing the processing structure
aFrame the new used frame
aGroup the frame group to assign to the frame
aDuplicate if not zero, duplicate the frame before appending it into the used frameset

Insert aFrame into the usedframes frameset of aProcessing if it is not already present in that frameset (present being checked using the filename and tag information of existing frames against aFrame). If aDuplicate is zero, the frame is destroyed here, assuming that it was duplicated from some other frame before being passed into this function. In this case, any property (e.g. filename) queried from the frame can no longer be used afterwards!

Definition at line 193 of file muse_processing.c.

References usedframes.

Referenced by muse_basicproc_load_reduced(), muse_postproc_cube_load_output_wcs(), muse_processing_lsf_params_load(), muse_processing_mask_load(), muse_processing_sort_exposures(), muse_propertylist_load(), muse_sky_continuum_load(), muse_sky_lines_load(), muse_table_load(), and muse_table_load_filter().

cpl_error_code muse_processing_check_input ( muse_processing aProcessing,
unsigned char  aIFU 
)

Check the input files for completeness.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number
Returns:
CPL_ERROR_NONE on success or a cpl_error_code on any error.

Check the input frames for completeness. We check the min and max of all raw input files and the calibration files associated to the raw inputs. On error, a message will be printed and the relevant CPL error code will be returned.

Exceptions:
return CPL_ERROR_NULL_INPUT aProcessing is NULL
return CPL_ERROR_ILLEGAL_INPUT recipe config not found in aProcessing
output error message, return CPL_ERROR_DATA_NOT_FOUND a least one input requirement is not fulfilled

Definition at line 822 of file muse_processing.c.

References inframes, intags, muse_frameset_find(), muse_processing_get_recipeconfig(), and recipe.

Referenced by muse_basicproc_load().

cpl_boolean muse_processing_check_intags ( muse_processing aProcessing,
const char *  aTag,
int  aNChars 
)

Check that a tag is part of the input tags of a processing structure.

Parameters:
aProcessing the processing structure
aTag the tag to check for
aNChars the number of characters to check
Returns:
CPL_TRUE when the tag was found, CPL_FALSE otherwise and on error.

Check the intags array of aProcessing against aTag, using strncmp() and up to aNChars length.

Exceptions:
return CPL_ERROR_NULL_INPUT aProcessing and/or aTag are NULL

Definition at line 785 of file muse_processing.c.

References intags.

Referenced by muse_processing_sort_exposures().

void muse_processing_delete ( muse_processing aProcessing  ) 

Free the muse_processing structure.

Parameters:
aProcessing the structure to be freed

Definition at line 157 of file muse_processing.c.

References counter, inframes, intags, outframes, parameters, muse_processing_framecounter::tag, and usedframes.

cpl_frame_level muse_processing_get_frame_level ( const cpl_recipe *  aRecipe,
const char *  aFrametag 
)

Get the level for a product frame with a certain tag.

Parameters:
aRecipe the CPL recipe
aFrametag the tag of the output frame
Returns:
its frame level, CPL_FRAME_LEVEL_NONE if the tag was not registered.

Definition at line 188 of file muse_processinginfo.c.

References muse_processinginfo_get().

Referenced by muse_processing_new_frame().

int muse_processing_get_frame_mode ( const cpl_recipe *  aRecipe,
const char *  aFrametag 
)

Get the mode for a product frame with a certain tag.

Parameters:
aRecipe the CPL recipe
aFrametag the tag of the output frame
Returns:
its frame level, MUSE_FRAME_MODE_ALL if the tag was not registered.

Definition at line 203 of file muse_processinginfo.c.

References MUSE_FRAME_MODE_ALL, and muse_processinginfo_get().

Referenced by muse_processing_new_frame().

cpl_recipeconfig* muse_processing_get_recipeconfig ( cpl_recipe *  aRecipe  ) 

Get the recipe (frame) configuration.

Parameters:
aRecipe Pointer to the recipe
Returns:
the recipe config, or NULL if not found.

Definition at line 217 of file muse_processinginfo.c.

References muse_processinginfo_get().

Referenced by muse_processing_check_input().

muse_mask* muse_processing_mask_load ( muse_processing aProcessing,
const char *  aTag 
)

Load a mask file and its FITS header.

Parameters:
aProcessing the processing structure, includes the input frames list
aTag the tag
Returns:
a new muse_mask * or NULL on error
Remarks:
The new mask has to be deallocated using muse_mask_delete().
Only FITS keywords from the primary FITS header will be loaded.
Exceptions:
return NULL, propagate CPL error code muse_frameset_find() fails to find a frame of the given tag
return NULL, propagate CPL error code muse_mask_load() returns NULL

Definition at line 1083 of file muse_processing.c.

References inframes, muse_mask::mask, muse_frameset_find(), muse_mask_load(), and muse_processing_append_used().

muse_processing* muse_processing_new ( const char *  aName,
cpl_recipe *  aRecipe 
)

Create a new processing structure.

Parameters:
aName name of the recipe
aRecipe the CPL recipe
Returns:
the processing structure

Definition at line 134 of file muse_processing.c.

References counter, inframes, intags, muse_cplparameterlist_duplicate(), name, outframes, parameters, recipe, muse_processing_framecounter::tag, and usedframes.

cpl_frame* muse_processing_new_frame ( muse_processing aProcessing,
int  aIFU,
cpl_propertylist *  aHeader,
const char *  aTag,
cpl_frame_type  aType 
)

Create a new frame for a result file.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number (use negative value to suppress -nn postfix)
aHeader the custom properties to be included in the header
aTag tag to apply for output frames, used as default prefix
aType the type of data to be stored
Returns:
the new frame

Create a new frame for a result file. Set the correct type, group, level, tag. The filename consists of the prefix, possibly an underscore and a running number (for non-unique files), a minus and the IFU number, and the suffix ".fits".

As prefix, the frame tag is used on default. One may create a different prefix by providing the "MUSE PRIVATE FILE PREFIX" property in the header.

All "MUSE PRIVATE " properties are removed from the header.

Exceptions:
set CPL_ERROR_NULL_INPUT, return NULL aProcessing is NULL

Definition at line 405 of file muse_processing.c.

References MUSE_FRAME_MODE_DATEOBS, MUSE_FRAME_MODE_MASTER, MUSE_FRAME_MODE_SEQUENCE, MUSE_FRAME_MODE_SUBSET, muse_pfits_get_dateobs(), muse_processing_get_frame_level(), muse_processing_get_frame_mode(), muse_processing_prepare_header(), and recipe.

Referenced by muse_processing_save_cimage(), muse_processing_save_cube(), muse_processing_save_header(), muse_processing_save_image(), muse_processing_save_mask(), muse_processing_save_table(), muse_sky_lines_save(), and muse_sky_save_continuum().

cpl_error_code muse_processing_prepare_header ( const cpl_recipe *  aRecipe,
const char *  aFrametag,
cpl_propertylist *  aHeader 
)

Prepare and check a FITS header for a certain frame tag.

Parameters:
aRecipe the CPL recipe
aFrametag the tag of the output frame
aHeader the FITS header of the output frame
Returns:
CPL_ERROR_NONE if the header was OK

Definition at line 172 of file muse_processinginfo.c.

References muse_processinginfo_get().

Referenced by muse_processing_new_frame().

cpl_error_code muse_processing_prepare_property ( cpl_propertylist *  aHeader,
const char *  aName,
cpl_type  aType,
const char *  aDescription 
)

Prepare and check the specified property.

Parameters:
aHeader the header to be prepared
aName the property name
aType the property type
aDescription the description (comment) of this property
Returns:
CPL error code

Check the specified property for its existence and correct type and append the comment to it.

Exceptions:
return CPL_ERROR_DATA_NOT_FOUND the property was not found
return CPL_ERROR_TYPE_MISMATCH the property has different type
return CPL_ERROR_NULL_INPUT the header or the name was NULL

Definition at line 240 of file muse_processinginfo.c.

void muse_processing_recipeinfo ( cpl_plugin *  aPlugin  ) 

Output main pipeline configuration, inputs, and parameters.

Parameters:
aPlugin the CPL plugin of the recipe in question

Output pipeline name (PACKAGE_NAME) and version (PACKAGE_VERSION) and if aPlugin is non-NULL and the message or log levels are set to debug, give all input frames (filenames and tags), non-default parameters, and relevant environment variables.

This function is meant to be called at the beginning of a MUSE recipe.

Definition at line 338 of file muse_processinginfo.c.

int muse_processing_save_cimage ( muse_processing aProcessing,
int  aIFU,
cpl_image *  aImage,
cpl_propertylist *  aHeader,
const char *  aTag 
)

Save a computed FITS image to disk.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number
aImage the image to be saved
aHeader the custom properties to be included in the header
aTag tag to apply for output frames, used as prefix
Returns:
the CPL error code

Save an image to disk. The filename consists of the prefix, a "-" sign, the IFU number of the CCD, and the suffix ".fits". If the image saving was successful, add the output frame to the list of output frames in the processing structure.

Exceptions:
CPL_ERROR_NULL_INPUT an argument was NULL

Definition at line 671 of file muse_processing.c.

References muse_processing_new_frame(), and outframes.

cpl_error_code muse_processing_save_cube ( muse_processing aProcessing,
int  aIFU,
void *  aCube,
const char *  aTag,
muse_cube_type  aType 
)

Save a MUSE datacube to disk.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number
aCube the cube to be saved
aTag tag to apply for output frames, used as default prefix
aType type of cube to save (MUSE_CUBE_TYPE_FITS, MUSE_CUBE_TYPE_EURO3D, MUSE_CUBE_TYPE_SDP, or MUSE_CUBE_TYPE_LSF)
Returns:
CPL_ERROR_NONE or the relevant cpl_error_code on error

Save a datacube to disk. The filename consists on the prefix, a "-" sign, the IFU number of the CCD and the suffix ".fits". If saving went without error, add the output frame to the list of output frames in the processing structure.

Exceptions:
CPL_ERROR_NULL_INPUT an argument was NULL
CPL_ERROR_ILLEGAL_INPUT an unknown aType was given

Definition at line 529 of file muse_processing.c.

References MUSE_CUBE_TYPE_EURO3D, MUSE_CUBE_TYPE_FITS, MUSE_CUBE_TYPE_LSF, MUSE_CUBE_TYPE_SDP, muse_datacube_save(), muse_euro3dcube_save(), muse_lsf_cube_save(), muse_processing_new_frame(), and outframes.

Referenced by muse_postproc_cube_resample_and_collapse().

cpl_error_code muse_processing_save_header ( muse_processing aProcessing,
int  aIFU,
cpl_propertylist *  aHeader,
const char *  aTag 
)

Save a FITS header to disk.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number
aHeader the header to be saved
aTag tag to apply for output frames, used as prefix
Returns:
the CPL error code

Save a bare header to disk. The filename consists of the prefix, a "-" sign, the IFU number of the CCD, and the suffix ".fits". If the header saving was successful, add the output frame to the list of output frames in the processing structure.

Exceptions:
CPL_ERROR_NULL_INPUT an argument was NULL

Definition at line 711 of file muse_processing.c.

References muse_processing_new_frame(), and outframes.

int muse_processing_save_image ( muse_processing aProcessing,
int  aIFU,
muse_image aImage,
const char *  aTag 
)

Save a computed MUSE image to disk.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number
aImage the image to be saved
aTag tag to apply for output frames, used as prefix
Returns:
the CPL error code

Save an image to disk. The filename consists on the prefix, a "-" sign, the IFU number of the CCD and the suffix ".fits". If the image saving was successful, add the output frame to the list of output frames in the processing structure.

Exceptions:
CPL_ERROR_NULL_INPUT an argument was NULL

Definition at line 488 of file muse_processing.c.

References muse_image::header, muse_image_save(), muse_processing_new_frame(), and outframes.

Referenced by muse_postproc_cube_resample_and_collapse().

int muse_processing_save_mask ( muse_processing aProcessing,
int  aIFU,
muse_mask aMask,
const char *  aTag 
)

Save a computed MUSE mask to disk.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number
aMask the mask to be saved
aTag tag to apply for output frames, used as prefix
Returns:
the CPL error code

Save an mask to disk. The filename consists on the prefix, a "-" sign, the IFU number of the CCD and the suffix ".fits". If the mask saving was successful, add the output frame to the list of output frames in the processing structure.

Exceptions:
CPL_ERROR_NULL_INPUT an argument was NULL

Definition at line 751 of file muse_processing.c.

References muse_mask::header, muse_mask_save(), muse_processing_new_frame(), and outframes.

cpl_error_code muse_processing_save_table ( muse_processing aProcessing,
int  aIFU,
void *  aTable,
cpl_propertylist *  aHeader,
const char *  aTag,
muse_table_type  aType 
)

Save a computed table to disk.

Parameters:
aProcessing the processing structure
aIFU the IFU/channel number
aTable the table to be saved
aHeader the custom properties to be included in the header (for aType == MUSE_TABLE_TYPE_CPL)
aTag tag to apply for output frames, used as prefix
aType type of table to save (MUSE_TABLE_TYPE_CPL or MUSE_TABLE_TYPE_PIXTABLE)
Returns:
CPL_ERROR_NONE or the relevant cpl_error_code on error

Save a table to disk. The filename consists on the prefix, a "-" sign, the IFU number of the CCD and the suffix ".fits". If the image saving was successful, add the output frame to the list of output frames in the processing structure.

Exceptions:
CPL_ERROR_NULL_INPUT a mandatory argument was NULL
CPL_ERROR_ILLEGAL_INPUT an unknown aType was given

Definition at line 607 of file muse_processing.c.

References muse_pixtable_save(), muse_processing_new_frame(), MUSE_TABLE_TYPE_CPL, MUSE_TABLE_TYPE_PIXTABLE, and outframes.

cpl_table* muse_processing_sort_exposures ( muse_processing aProcessing  ) 

Sort input frames (containing lists of pixel table filenames) into different exposures.

Parameters:
aProcessing the processing structure, especially containing input/output frames and the tag
Returns:
the exposure table on success or NULL on error
Remarks:
This function assumes that all relevant FITS keywords (DATE-OBS and EXTNAME with the CHANnn number) are contained in the primary header.

This function uses the DATE-OBS header keyword to check which of the input objects belong to the same exposure. As they all have been split up from the same exposure in the beginning, DATE-OBS should be identical for related files.

If one IFU of the same exposure is given multiple times in the input frameset, only the first one is used and a warning is printed for all others.

Note:
This function also modifies the usedframes component of the aProcessing structure, and adds all newly found exposures to that frameset. However, it is possible to run this function more than once, since no duplicate frames are added to that frameset.
Exceptions:
set CPL_ERROR_NULL_INPUT, return NULL invalid processing pointer
set CPL_ERROR_DATA_NOT_FOUND, return NULL no exposures found

Definition at line 936 of file muse_processing.c.

References inframes, muse_pfits_get_dateobs(), muse_processing_append_used(), muse_processing_check_intags(), and muse_utils_get_ifu().

void muse_processinginfo_delete ( cpl_recipe *  aRecipe  ) 

Clear all information from the processing info and from the recipe config.

Parameters:
aRecipe the CPL recipe

To be called before closing the program.

Definition at line 143 of file muse_processinginfo.c.

References muse_processinginfo, and muse_processinginfo_get().

void muse_processinginfo_register ( cpl_recipe *  plugin,
cpl_recipeconfig *  recipeconfig,
muse_processing_prepare_header_func *  prepare_header,
muse_processing_get_frame_level_func *  get_frame_level,
muse_processing_get_frame_mode_func *  get_frame_mode 
)

Register extended functionalities for MUSE recipes.

Parameters:
plugin the CPL recipe
recipeconfig the recipe configuration
prepare_header function to declare the properties of a frame
get_frame_level function that returns the processing level of
get_frame_mode function that returns the output mode of a frame

This functions registers all extensions that are used to use our framework.

Definition at line 108 of file muse_processinginfo.c.

References muse_processinginfo.


Variable Documentation

NULL-terminated array of all registered processing information

Definition at line 72 of file muse_processinginfo.c.

Referenced by muse_processinginfo_delete(), muse_processinginfo_get(), and muse_processinginfo_register().


Generated on 26 Jan 2017 for MUSE Pipeline Reference Manual by  doxygen 1.6.1