ELT Linux Installation Guide

This guide describes how to install the ELT Development Environment (a Linux platform). The guide includes the installation of standard (typically third-party) packages that are supported in the DevEnv. The guide does not include the installation of ELT Control System infrastructure software (e.g. the ELT COmmon Software, ECOS), links to which are provided at the end of this guide.

The ELT Linux DevEnv is based on Fedora (public Linux distribution from the Fedora community). Fedora is forked from the RedHat Linux and is currently the upstream source version for RedHat commercial distribution. Fedora is now shipping for 64 bit platforms; currently there is no 32 bit ISO image. This is primarily due to the fact that most computers in production are 64 bit. Therefore Fedora can be installed on most relatively-modern bare-metal computers with x86_64 architecture (Intel, AMD) like laptops, desktops or servers. Alternatively Fedora can also be installed on virtual machines (VM) under Intel-based hypervisors like commercial VMware or public ones like VirtualBox.

The installation of DevEnv is based on PUPPET, an open-source configuration management tool. The RPM puppet-elt contains the puppet scripts to complete and verify the DevEnv installation, consisting of:

Installation of supporting RPMs

Configuration of services

Creation of user accounts for software development

The ESO/ELT Fedora repository

The current release of the ELT Linux DevEnv is based on the original Fedora image with a minimal installation of software packages or RPMs (RPM Package Manager). The real DevEnv installation consists in the download and installation of software packages relevant to the development of the ELT project. Most of these packages (RPMs) were originally retrieved from public Fedora and EPEL (Extra Packages for Enterprise Linux) repositories. In addition to these RPMs, the DevEnv also installs other public-domain tools and ESO packages not available at public repositories. The process should guarantee the same result in DevEnv installations done at ESO or by external contractors.

Traceability: For testing and support purposes it is necessary to provide a mechanism that can retrieve and install any of the previous releases of the DevEnv. In reality it means to have access to all RPMs referred by previous installations of the DevEnv. For this reason, the DevEnv cannot rely on public repositories like Fedora and EPEL; they do not support traceability as old RPMs might be removed with new ones.

The DevEnv installation process has to rely on it:

<https://ftp.eso.org/pub/elt/repos/fedora/releases/38/>

<https://ftp.eso.org/pub/elt/repos/fedora/updates/38/>

<https://ftp.eso.org/pub/elt/repos/fedora/eso/38/>

At our ESO/ELT repositories we combine the packages downloaded from Fedora release with Fedora updates, and with those packages developed at ESO. With new releases of the DevEnv new RPMs are accumulated to our repository; old RPMs are never removed. This repository is also the official channel to provide minor and patch releases to the DevEnv (A major release, e.g. a new OS, will require a new repository).

Prerequisites

Hardware requirements

The minimum requirements for the installation of the ELT Linux DevEnv are:

4x CPUs x86_64 (Intel or AMD)

8GB RAM

40 GB disk

1x NIC

HW compatible with Fedora

Depending on your own requirements, e.g. disk-space, number of users, heavy usage of GUI components etc. or in your current HW configuration these requirements should be exceeded.

Configuring BIOS for low latency environments (RT)

If you plan to use the host for real-time applications (RT) you may need to set your BIOS options for low latency. Please notice that factory BIOS defaults are optimized to provide a good balance between performance and power efficiency for general-purpose environments. However there are environments where you many need to optimize your hardware for maximum throughput or lowest latency to provide optimal responsiveness where real-time responses are needed. The available BIOS options may vary, depending upon server model, processor/memory architecture, and BIOS revision. You have to consult your Hardware Owner’s Manual for more details.

For Dell PowerEdge 12th Generation Severs please follow recommendations for low latency as indicated in Dell document:

<https://www.eso.org/~eltmgr/configuring-low-latency-environments-on-dell-poweredge-12g-servers.pdf>

Step 1: Fedora Installation

Fedora image and documentation

The DVD ISO used for the ELT Linux DevEnv is the Fedora-Server-dvd-x86_64-38-1.2.iso and can be downloaded from our ESO HTTPS area at:

<https://ftp.eso.org/pub/elt/repos/Fedora-Server-dvd-x86_64-38-1.6.iso> with sha256sum: 66b52d7cb39386644cd740930b0bef0a5a2f2be569328fef6b1f9b3679fdc54d

Fedora installation documents. There are many web pages that describe the server minimal Fedora installation step by step, snapshots included. Here just a few of them:

<https://able.bio/KY64/minimal-installation-fedora-linux–73410e6d>

<https://docs.fedoraproject.org/en-US/fedora/f36/install-guide/install/Preparing_for_Installation/>

Network configuration:

<https://fedoraproject.org/wiki/Networking/CLI>

You can download the DVD ISO image with any browser and the URL above.

Fedora installation

1. Installation Settings

OS installation details are not part of this document and some screenshot may refer to previous Fedora base versions. Please use any of the many documents already available in the web (see any of the list in the OS documentation section above). The ELT DevEnv requires the following setting (in bold text where input is required):

LOCALIZATION

DATE & TIME: Set date and local time

KEYBOARD: English (US)

LANGUAGE SUPPORT: English (United States)

SECURITY

SECURITY POLICY: No profile selected

SOFTWARE

INSTALLATION SOURCE: Local media

SOFTWARE SELECTION: Fedora Custom Operating System

SYSTEM

INSTALLATION DESTINATION: Select the disk for the OS installation. Choose also the storage configuration Automatic as described below:

If your selected disk for the OS is already partitioned (e.g. from a previous OS installation) you might need to delete the existing partition table and reclaim the disk space for the new OS. Just follow this sequence:

NETWORK & HOSTNAME: Select your NIC, activate it, and select a hostname (e.g. elthost):

By default, the NIC is configured with DHCP. If DHCP is what you need (e.g. your elthost is a VM on your Desktop/Laptop) then you are done. If you need to give the VM a static IP address you can go in the configuration window:

USER SETTINGS

ROOT PASSWORD: Set root password and Allow root SSH login with password.

Do not create any other user account at this time. The ELT DevEnv installation will create later accounts eltdev and eltmgr automatically.

2. Commence Installation

When above selections are completed

3. Finish Installation and Reboot

Note: You might need to stop the system and dismount the DVD before rebooting. In same cases, the DVD remains mounted and next reboot will bring you back to the Fedora installation.

After rebooting the system will presents the following text prompt:

Fedora 38 (Thirty Eight)
Kernel 6.6.8-100.fc38.x86_64 on an x86_64
elthost login:

Login as root with the password selected during the OS installation:

elthost login: root
Password: <your_password>
Last login: <date & time> on tty1

IMPORTANT: DO NOT execute any “dnf update” in this Minimal Server Installation as you might get newer RPMs than those required in our DevEnv distribution. The installation of DevEnv is unable to downgrade them. If this happens you have two alternatives: remove the newer RPMs by hand (it is risky and not always possible), or start the installation from the beginning.

4. Increase Filesystem

The default partition table only allocates 15GB for the “/” partition, leaving the rest of the disk free. You can increase the “/” partition to the full disk size with the following sequence:

[root@elthost ~]# df Filesystem 1K-blocks Used Avail Use% Mounted /dev/mapper/fedora_fedora-root 15718400 1342472 14375928 9% /
[root@elthost ~]# lvextend /dev/mapper/fedora_fedora-root -l100%FREE
[root@elthost ~]# xfs_growfs /dev/mapper/fedora_fedora-root
[root@elthost ~]# df Filesystem 1K-blocks Used Avail Use% Mounted /dev/mapper/fedora_fedora-root 25151488 1400300 23743100 6% /

5. Access the internet

To complete the ELT Linux installation is necessary for the machine to have access to internet. If the network configuration during the OS installation was skipped or incomplete you can still configure hostname and IP address with commands:

[root@elthost ~]# hostnamectl set-hostname <hostname>
[root@elthost ~]# nmcli device # to see available NICs
[root@elthost ~]# nmcli connection modify <device> ipv4.addresses <IP>/<mask>
[root@elthost ~]# nmcli connection modify <device> ipv4.gateway <IP>
[root@elthost ~]# nmcli connection modify <device> ipv4.dns <IP>
[root@elthost ~]# nmcli connection modify <device> ipv4.dns-search "<comma-separtated list of qualified names>"
[root@elthost ~]# nmcli connection modify <device> ipv4.method manual
[root@elthost ~]# nmcli device reapply <device>

Example of the configuration of host eltint63 at ESO:

[root@elthost ~]# hostnamectl set-hostname eltint63
[root@elthost ~]# nmcli connection modify ens192 ipv4.addresses 134.171.3.174/24
[root@elthost ~]# nmcli connection modify ens192 ipv4.gateway 134.171.3.254
[root@elthost ~]# nmcli connection modify ens192 ipv4.dns 134.171.7.11
[root@elthost ~]# nmcli connection modify ens192 ipv4.dns-search "hq.eso.org, eso.org"
[root@elthost ~]# nmcli connection modify ens192 ipv4.method manual
[root@elthost ~]# nmcli device reapply ens192

If you are installing the ELT DevEnv as a VM on a local hypervisor as VirtualBox or VMware Worsktation, please notice that the network card of your VM can be created basically with two options:

NAT: The guest VM shares the IP of the host. With the internet access of the host but the guest is isolated from the world. The guest is typically configured with DHCP. This is the default.

Bridged: The guest VM has direct access to the network card of the host. With your own internet access and the guest can be accessed from the world. You can chose DHCP or a static IP.

Step 2: Package Installation

Installation of puppet packages

With the minimal installation of Fedora completed and with the system connected to internet it is possible now to download the RPMs required to install PUPPET and the installation scripts corresponding to the latest release of the DevEnv. To install PUPPET packages and scripts execute:

[root@elthost ~]# sh <(curl https://ftp.eso.org/pub/elt/repos/fedora/eso/GetDevEnv.sh)

Installing the ELT DevEnv with puppet

The RPM puppet-elt delivers two puppet scripts and their supportive files:

/root/elt/puppet-force-align

–help|-h prints this text

–verbose|-v Puppet script executed with –verbose option. Log is also sent to terminal

–debug|-d Puppet script executed with –debug option.

/root/elt/puppet-check

If you need a particular variant of ELT DevEnv there are some environment variables defining them. The name of the variant has to be exported before proceeding with the effective installation. If nothing is indicated, the puppet installation assumes these default values:

ELT_ROLE=ELTDEV

ELT_DM=no

ELT_STABLE_STREAM=yes

ELT_BETA_STREAM=no

To proceed with the installation as root, execute the puppet-force-align to execute the installation and puppet-check to verify it:

[root@elthost ~]# cd /root/elt
[root@elthost ~]# ./puppet-force-align

The script may takes 40 minutes or more to complete; it depends largely in the internet speed and the connection to the ESO/ELT repository. The screen-shot above shows a complete installation with no errors. A “progress indicator” in the form of “dots” every 5 seconds indicates the script is still progressing. The procedure will always generate a log-file that can be used in case of problems. The log-file contains detailed information about each step performed and is located in the /tmp directory. The log-file is named elt-puppet-YYYYMMDD.log where YYYYMMDD is the string describing the year, month and day of execution. If puppet-force-align is executed again in the same day the results will be appended to the existing log-file. The values of ELT development variables are saved by the installation script in the shell profile configuration file: /etc/profile.d/eltdev.sh. They will be loaded in the environment with the next login in the system as a reminder of the current installation. You can still change the configuration, with certain limitations, by re-exporting a new value and re- executing the puppet script /root/elt/puppe-force-align. See the different options below.

It defines the scope of the installation. The currently known variants for the ELT_ROLE in the DevEnv are:

ELTDEV: Software development workstation. Includes tools and packages necessary for a standard developer. This is the default choice if nothing is indicated.

MINIMAL: A minimal installation, for use in specific hosts (e.g. Jenkins or in operations). It is subset of ELTDEV.

[root@elthost ~]# export ELT_ROLE=MINIMAL
[root@elthost ~]# /root/elt/puppet-force-align

Note: You can upgrade a MINIMAL installation to ELTDEV, but not viceversa.

The ELT_DM environment variable

DM stands for “Display Manager” and it sets or doesn’t set the execution of a daemon for a display manager in the console with these values:

NO: no Display Manager will be launched after the installation. This is the default value.

YES: Display Manager will be launched after the installation.

If you want the installation to finish with the launch of a display manager, execute:

[root@elthost ~]# export ELT_DM=yes
[root@elthost ~]# /root/elt/puppet-force-align

Note: you can change for “yes” to “no” and viceversa.

The ELT_STABLE_STREAM and ELT_BETA_STREAM environment variables

These two variables control enabling stable and beta streams. ELT DevEnv provides three types of streams:

release - packages that are part of the release never change. Their version is frozen. Packages from the releases stream allow the reproduction of an ELT DevEnv version. Usually, packages in release repositories are manually tested (in addition to automated tests).
stable - packages in the stable stream should pass automated tests on Jenkins. Versions of packages in the stable repository may change frequently. The goal of the stable stream is to provide updates to the software quickly and without compromising builds and tests of dependent projects. The stable stream is the best candidate for ELT Control Software development.
beta - packages are subject to promotion into the stable stream. Those haven’t passed automated tests yet, and they may break the build and tests of dependent projects.

For example if you want to pin packges of ELT DevEnv to a specific release, you should disable beta and stable streams:

[root@elthost ~]# export ELT_STABLE_STREAM=no
[root@elthost ~]# /root/elt/puppet-force-align

Note: you can change for “yes” to “no” and viceversa.

Reboot the system after a successful DevEnv installaltion

Very likely, the puppet-force-align will install a new kernel that will only becomes active in the next reboot of the system. So to complete the installation, please always reboot the machine:

[root@elthost ~]# reboot

Verifying the ELT DevEnv installation

The installation script puppet-force-align finishes with the invocation of the verfication script puppet-check, however you can execute puppet-check any time later to verify the system remains complaint with the ELT DevEnv:

[root@elthost ~]# cd /root/elt
[root@elthost ~]# ./puppet-check

This is the screen-shot of the execution of puppet-check when everything is correct:

Step 3: Installing ELT Common Software (ECOS, including CII)

At this point the ELT DevEnv is installed and verified, but there is no ELT Control System infrastructure software installed on the machine.

To install the infrastructure software, please follow the steps (and links) below:

Installing ECOS: <https://gitlab.eso.org/ecos/ecos/#meta-package>

Configuring CII Services: <https://gitlab.eso.org/ecos/eltsw-docs/-/wikis/KnowledgeBase/CII#how-to-post-install-cii-services>

Other ELT DevEnv Operations

ELT DevEnv releases

The ELT Linux DevEnv is defined by the environment variable $ELT_RELEASE. It follows standard numeration major.minor.patch-iteration, where an increase in:

The major number indicates a considerable change largely affecting the complete DevEnv, e.g. with a new OS. (2: CentOS-7, 3: CentOS-8, 4: Fedora-34, 5: Fedora-38).

The minor number indicates a change affecting important components of the DevEnv, like toolkits or their releases.

The patch number will be used to fix a severe bug of a minor release.

The iteration number is meaningless. Only for integration purposes. Typically DevEnv will be released with the following frequency:

Major releases will be released with a frequency of at least 6 months. A major release will be announced with at least two months in advance.

Minor releases are expected to be released with a frequency of months

Patch releases will be made available to fix a severe bug in a Minor release.

To know which release of the DevEnv is installed on any system execute:

[root@elthost ~]# echo $ELT_RELEASE

The release is also indicated in the login welcome message at /etc/motd.

Updating the ELT DevEnv

To check if there are newer releases available, execute as root:

[root@elthost ~]# dnf makecache
[root@elthost ~]# dnf --showduplicates list puppet-elt

The first command is needed to dowload the latest metadata from repositories. The option –showduplicates will list all available releases in the repository; and in green font the current release. You can install an upgrade to any newer release by explicitely indicating that release:

[root@elthost ~]# dnf -y update puppet-elt-5.0.1

or to the latest release:

[root@elthost ~]# dnf -y update puppet-elt

After the installation of a newer or latest puppet-elt RPM, complete the upgrade with the execution of the remaining puppet installation and verification:

[root@elthost ~]# ./puppet-force-align
[root@elthost ~]# ./puppet-check

With updates you might get new kernels, therefore it is always recommended to reboot the host after the upgrade. At least, users of the system will have to logout/login to reload the new enviroment.

Installing an older release

Instead proceed from scratch with the installation of Fedora as described in section 4.4. Choose one of the DevEnv releases available from the ESO anonymous FTP server:

<https://ftp.eso.org/pub/elt/repos/fedora/eso/38>

Please notice that elt-latest is soft link pointing to the latest release of the DevEnv. Install the puppet packages as described in section 4.5 but changing string “latest” with the numeration of the release of your choice, e.g. to install release 5.1.1, execute:

[root@elthost ~]# sh <(curl https://ftp.eso.org/pub/elt/repos/fedora/eso/GetDevEnv.sh) elt-5.1.1

Complete the installation and verification with sections 4.6 and 4.7.