Metadata-Version: 2.4
Name: pyhdrl
Version: 0.1.0
Summary: Python language bindings for the ESO High Level Data Reduction C libraries
Home-page: http://www.eso.org/sci/software/pycpl
Author: European Southern Observatory, Australian Astronomical Optics Macquarie
Author-email: cpl-help@eso.org
License: GPL-3.0-or-later
Project-URL: Homepage, http://www.eso.org/sci/software/pycpl
Project-URL: Documentation, http://www.eso.org/sci/software/pycpl
Project-URL: Source, https://ftp.eso.org/pub/dfs/pipelines/libraries/pycpl
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: C
Classifier: Programming Language :: C++
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Software Development
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Unix
Classifier: Development Status :: 4 - Beta
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pycpl>=1.0.3
Provides-Extra: doc
Requires-Dist: sphinx; extra == "doc"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Dynamic: license-file

# PyHDRL &mdash; Python 3 bindings for the ESO High Level Data Reduction Library

PyHDRL provides a Python 3 API (language bindings) for the ESO High Level Data Reduction Library (HDRL) using pybind11.
It allows for using the ESO High Level Data Reduction Library in Python scripts, or directly in an interactive Python session,
and thus allows for using HDRL algorithms in recipes implemented in Python as part of an instrument pipeline package.

PyHDRL complements the PyCPL and PyEsoRex packages and is part of the ESO Python infrastructure packages for instrument pipelines

## The PyHDRL Module

The PyHDRL module is organized into three sub-modules, loosely following the PyCPL module layout. The three PyHDRL submodules are:

- **hdrl.core** which provides the core data types required by the bindings of the implemented HDRL algorithms,
- **hdrl.func** providing the Python API for the implemented HDRL algorithms,
- **hdrl.debug** a helper module for testing type conversions between PyCPL and PyHDRL, not intended for use by developers

The PyHDRL sub-module **hdrl.func** provides a Python API for the following HDRL functionalities:
- Effective airmass,
- Bad-pixel detection,
- Barycentric correction,
- Bias,
- Object catalog generation,
- Dark,
- Efficiency (WIP),
- Flat,
- Fringing,
- Limiting magnitude,
- Overscan,
- Fixed pattern noise,
- Differential Atmospheric Refraction,
- Image and Cube Resampling,
- Response (WIP),
- Strehl

## User Documentation

The documentation for the general user, including download links, installation instructions, tutorial, and the API reference are available on the [PyHDRL web site (TBD)](https://www.eso.org/sci/software/pycpl/). If you are interested in installing and using PyHDRL for production use it is recommended that you continue reading there.

## Installation

The following instructions are directed towards developers and experienced users. Please note that they are not a step by step tutorial!

### Software prerequisites

- C++ compiler with C++17 support (g++ >= 9.5 or clang++ >= 5)
- cmake >= 3.12
- python >= 3.9
- Python development headers 
    * package python3-devel (On Fedora, Red Hat, CentOS, etc.)
    * package python3-dev (On Ubuntu, Debian, etc.)
- pybind11 >= 2.8.0
- cpl >= 7.2.2
- hdrl >= 1.16
- pycpl >= 1.0.3

#### Installing pybind11

You can choose either the packaged pybind from your linux vendor if you are not using any virtual environment, e.g. for RPM based
distribution (Fedora, RedHat, CentOS, etc.):
```shell
sudo dnf install python3-pybind11
```
**or** you can install pybind11 with `pip`.
```shell
python3 -m pip install pybind11
```
In general the preferred method of installing pybind11 is to use using `pip`.

#### Installing CPL

In order to install PyHDRL an installation of CPL, including the development files are needed. A compatible, public version can be obtained
from the ESO repositories as RPM or MacPorts package. To setup the ESO repositories follow the instructions for
[RPM packages](https://www.eso.org/sci/software/pipelines/installation/rpm.html) and
[MacPorts packages](https://www.eso.org/sci/software/pipelines/installation/macports.html) respectively. CPL can also be installed from the public source packages which can be obtained from the [CPL download page](https://www.eso.org/sci/software/cpl/download.html).

Development versions of CPL are also available from the [ESO GitLab repositories](https://gitlab.eso.org) (**ESO internal access only**). Building CPL from a checked out working copy requires additional (build-time) dependencies, like the GNU autotools (autoconf, automake and libtool), and the libltdl development package to be installed.

The typical installation process is the following, when installing:

- from the ESO Fedora repositories:
  ```shell
  sudo dnf install cpl-devel
  ```
- from the ESO MacPorts repositories:
  ```shell
  sudo port install cpl
  ```
- from a public release tar archive:
  ```shell
  tar -zxvf cpl-7.3.2.tar.gz
  cd cpl-7.3.2
  ./configure --prefix=/usr/local --with-cfitsio=/usr/local --with-wcslib=/usr/local --with-fftw=/usr/local
  make
  sudo make install
  ``` 
- from an ESO GitLab working copy:
  ```shell
  git clone https://gitlab.eso.org/pipelines/cext.git
  cd cext
  ./autogen.sh
  ./configure --prefix=/usr/local
  make
  sudo make install
  cd ..
  git clone https://gitlab.eso.org/pipelines/cpl.git
  cd cpl
  ./autogen.sh
  ./configure --prefix=/usr/local --with-cfitsio=/usr/local --with-wcslib=/usr/local --with-fftw=/usr/local --with-system-cext --with-cext=/usr/local
  make
  sudo make install
  ```

#### Installing HDRL

In order to install PyHDRL an installation of HDRL, including the development files are needed. PyHDRL also requires that HDRL is available as
an installed shared object library. Building and installing HDRL as a shared object library was added in version 1.16 of HDRL and requires
the **non-default** option `--enable-standalone` when configuring the HDRL source tree prior to building.

Currently no source, RPM, or Macports packages are available for HDRL 1.16. This requires HDRL to be build from an ESO Gitlab working copy:

```shell
git clone https://gitlab.eso.org/pipelines/hdrl.git
cd hdrl
./autogen.sh
./configure --prefix=/usr/local --enable-standalone --with-cpl=/usr/local --with-gsl=/usr/local
make
sudo make install
```

#### Installing PyCPL

PyHDRL requires that a PyCPL installation is available in the same Python environment. Thus, make sure to choose the right installation
method, depending on where PyHDRL will be installed in a subsequent step.

For production use, public PyCPL source distribution (sdist) packages are available from the 
[ESO ftp server](https://ftp.eso.org/pub/dfs/pipelines/libraries/pycpl/). Installing the downloaded PyCPL source distribution can then
be done like:

- User-space installation, no venv/conda:
  ```shell
  python3 -m pip install --upgrade --user pycpl-1.0.3.tar.gz
  ```
- Installation using a Python 3 venv/conda environment:
  ```shell
  python3 -m pip install -upgrade pycpl-1.0.3.tar.gz
  ```

For developers of PyHDRL it is **strongly recommended** to use a Python virtual environment during development and therefore as target of the PyCPL, and the following PyHDRL installation! 

Developers may also prefer to use the [PyCPL GitLab repository](https://gitlab.eso.org/pipelines/pycpl) (**ESO internal access only!**)
rather than a released version version of PyCPL. PyCPL can then be installed from the cloned git repository like:

- User-space pip install, no venv/conda:
  ```shell
  cd pycpl
  python3 -m pip3 install --user -U .
  ```
- Installation using a Python 3 venv/conda environment:
  ```shell
  cd pycpl
  python3 -m pip install -U .
  ```
pip will automatically install the required Python dependencies (astropy, numpy, pybind11, setuptools).

### Installing PyHDRL

Public PyHDRL source distribution (sdist) packages will be available from the
[ESO ftp server](https://ftp.eso.org/pub/dfs/pipelines/libraries/pycpl/). Installing the downloaded PyHDRL source distribution can then be done like:

* User-space installation, no venv/conda:
  ```shell
  python3 -m pip install --user -U pyhdrl-0.1.0.tar.gz
  ```
* Installation using a Python 3 venv/conda environment:
  ```shell
  python3 -m pip install -U pyhdrl-0.1.0.tar.gz
  ```

Developers and contributors rather should clone the [PyHDRL GitLab repository](https://gitlab.eso.org/pipelines/pyhdrl)
(**ESO internal access only!**) to obtain the PyHDRL source code.

Then, for installing PyHDRL from within the source tree (from an unpacked source distribution, or a working copy):

* User-space pip install, no venv/conda:
  ```shell
  cd pyhdrl
  python3 -m pip3 install --user -U .
  ```
* Installation inside a Python 3 venv/conda environment:
  ```shell
  cd pyhdrl
  python3 -m pip install -U .
  ```
pip will automatically install the required Python dependencies if they are not already present in the target environment.

Environment variables may be required if your CPL or HDRL installation is not in its expected location. CPL and HDRL installations will be found automatically, if it is installed in the system using RPM or MacPorts packages. For manual installations it is safe to assume that the installation directory of CPL and HDRL must be set explicitly. Non-default locations can be set with the environment variables `CPL_ROOT` **or** `CPLDIR` for CPL and `HDRL_ROOT` **or** `HDRLDIR` for HDRL. If both variables are set for CPL or HDRL, `CPL_ROOT` and `HDRL_ROOT`
take precedence, respectively.

When environment variables like `HDRLDIR` need to be used the installation commands become

- User-space pip install, no venv/conda:
  ```shell
  cd pyhdrl
  HDRLDIR=/home/username/ESO python3 -m pip3 install --user -U .
  ```
- Installation inside a Python 3 venv/conda environment:
  ```shell
  cd pyhdrl
  HDRLDIR=/home/username/ESO python3 -m pip install -U .
  ```
This also applies to the other environment variables which are recognized by the PyCPL build process:

- `PYHDRL_BUILD_DEBUG`
- `PYHDRL_BUILD_SANITIZE`
- `PYHDRL_BUILD_VERBOSE`

These are boolean variables which may be set to 0 or 1 to deactivate or activate the respective configuration option. The default for
all these variables is 0, i.e. inactive. The variables can be used to build PyHDRL in debug mode, with support for address sanitizer, and
with verbose output.

### Documentation

The PyHDRL source tree also contains the source for the PyHDRL Reference Manual.

#### Prerequisites

Building this manual requires the Sphinx Python library and its dependencies. These can be installed using `pip` by specifying the `[doc]` extra rquirements when installing PyHDRL, e.g.
```shell
cd pyhdrl
python3 -m pip install -U '.[doc]'
```
#### Building the documentation

First change to the docs subdirectory of the PyHDRL sources, e.g. `cd pyhdrl/doc`. The documentation can then be built using `make`, with different make targets corresponding to different output formats. For example:

- HTML output: `make html`
- PDF output (requires LaTeX): `make latexpdf`

To list all of the available targets/formats just run `make` without a target in the `docs` directory.

### Testing

The PyHDRL source tree includes a comprehensive set of unit tests.

#### Prerequisites

Running the PyHDRL tests requires the pytest Python library and its dependencies. These can be installed using `pip`
by specifying the `[test]` extra requirements when installing PyHDRL, e.g.
```shell
cd pyhdrl
python3 -m pip install -U .[test]
```

#### Unit tests

To run the unit tests simply run `pytest` in the top level directory of the PyHDRL source tree, e.g.
```shell
cd pyhdrl
python3 -m pytest -r fEsx --log-file=pyhdrl_unit_tests.log
```

There are many options to configure the output of pytest and select which tests are run. See the pytest documentation for details.

## Using PyHDRL

Once installed, PyHDRL is imported as the module `hdrl`.
```python
import hdrl
```

## Debugging PyHDRL

TBD

## Support

TBD

## Roadmap

TBD

## Contributing

TBD

## Acknoledgements

TBD

## License

GPL-3.0-or-later

## Project status

TBD

<!-- ***

# Editing this README

When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!).  Thank you to [makeareadme.com](https://www.makeareadme.com/) for this template.

## Suggestions for a good README
Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.

## Name
Choose a self-explaining name for your project.

## Description
Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.

## Badges
On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.

## Visuals
Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.

## Installation
Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.

## Usage
Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.

## Support
Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.

## Roadmap
If you have ideas for releases in the future, it is a good idea to list them in the README.

## Contributing
State if you are open to contributions and what your requirements are for accepting them.

For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.

You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.

## Authors and acknowledgment
Show your appreciation to those who have contributed to the project.

## License
For open source projects, say how it is licensed.

## Project status
If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers. -->
