Operating system installation

This section describes operating system installation, as well as necessary and additional packages installation. The ECSS-10 system version 3.14 is running under Ubuntu Server 18.04.x LTS 64bit.

Preliminary requirements

• Installation bootable media with operating system distribution;
• Prepared server with updated BIOS, ILO (if available), connected network for Internet access;
• In BIOS, USB Flash or CD/DVD is set as first priority for downloading from the installation media;
• Sufficient volume of disk space and memory in accordance with the project. 

Якорь
inst_os inst_os

Operating system installation

To install the OS, do the following:
After downloading from the installation media, select "Install Ubuntu Server".
Select system language and keyboard layout.

Configuring network interfaces 

Configure network interface for the Internet connection:

Image RemovedImage Added

Creating disk partitions

Select "Custom storage layout":

Image RemovedImage Added

Next, create additional sections in LVM group in accordance with Table 1.

Image RemovedImage Added

Table 1 — Option of placing information in the file system on physical media for servers 

...

Example of creating partitions for 200Gb disk:

Image RemovedImage Added

Якорь
names names

Configuring user and server names

The "hostname" parameter must be configured in the system servers.

...

Image RemovedImage Added

OpenSSH server installation

At the end of the OS installation, you will be prompted to install additional software for remote connection — you need to install OpenSSH server.

Image RemovedImage Added

Якорь
swap swap

Disabling swap

Swap file In Ubuntu 18.04 is located in root directory — /swap. img.

...

sudo swapoff -a

sudo rm /swap.img

Якорь
TZ TZ

Setting time zone

When installing Ubuntu-18.04, setting a time zone is not prompted.

sudo timedatectl set-timezone Asia/Novosibirsk

Checking operating system installation

Basically system checking is about correctness of creating disk partitions and SSH access availability.

...

• <user> — user name specified during installation;
• <IP_ecss> — IP address of the host specified during installation.

Якорь
etc_hosts etc_hosts

Configuring /etc/hosts

The domain name of the ecss1 host must correspond to the address 127.0.1.1. You also need to register the ecss2 host address. To do this, you need to register the IP addresses of ecss hosts in the /etc/hosts file.

...

127.0.0.1 localhost
127.0.1.1 ecss2
192.168.1.21 ecss1

Configuring network interfaces 

Якорь
network network

Network settings must be performed using Netplan.

Example:

Configure a server with 4 network interfaces with channel aggregation (802.3ad) and necessary VLANs. There is a gateway for Internet access — 192.168.1.203

...

To apply new network settings, run the netplan apply command. No network or system restart is required. 

OS updating and necessary software installation 

Якорь
inst_soft inst_soft

System update 

Якорь
repo repo

Adding ELTEX repository:

sudo sh -c "echo 'deb [arch=amd64] http://archive.eltex.org/ssw/bionic/3.14 stable main extras external' > /etc/apt/sources.list.d/eltex-ecss10-stable.list"

...

sudo apt update
sudo apt upgrade

Якорь
inst_soft inst_soft

Necessary software installation

List of mandatory service software:

...

sudo dpkg --get-selections

ECSS packages installation 

Якорь
inst_ecss inst_ecss

Preliminary requirements

• Installed and updated operating system (Ubuntu-18.04);
• Absence of user named ssw in the system;
• Disk space partitioning in accordance with recommendations;
• Configured Operating system installation v2 network;
• Installed set of necessary packages;
• Access to ELTEX repository.

For ECSS-10 system installation, you must install packages in order they are described in the documentation below.

Installation of required packages  

ecss-mysql installation 

Якорь
ecss-mysql ecss-mysql

The first step is to install ecss-mysql package.

...

If the system is deployed in cluster, then package installation and database replication configuration must be performed according to the instructions in MySQL master-master replication deployment scheme using keepalive appendix.

When installing package, MySQL server is installed with the necessary settings, and necessary databases are created. During installation, the following data will be requested:

...

ecss-node installation 

Якорь
ecss-node ecss-node

Installation of mandatory ecss-node package includes installation and initial configuration of the main subsystems.

...

During the package installation ssw user is created, on whose behalf all ecss services are launched*. The necessary directories are being created, DNS, SSL ceritificates, and NTP service are being configured. During the installation, questions necessary for the formation of configuration files will be asked.

Configuring certificates

Relevant only if a self-signed certificate was generated,only then ecss10root.crt will be installed in the system (when copying, it also tries to download ecss10root.crt, or if this file was placed during manual installation). If there are already certificates, then no actions will be taken. At the end, the validity of the certificate is also checked.

...

• url to http_terminal (https://ecss1:9999 )
• Login (admin)
• Password (password)
• Node with certificates (core1@ecss1)

DNS 

Якорь
DNS DNS

During ecss-node package installation internal DNS addresses are being configured. Depending on the current system configuration the following message may be displayed during installation:

...

sasha@ecss1:~$ systemctl status dnsmasq.service 
● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
   Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2021-09-24 20:52:03 +07; 2 weeks 3 days ago
 Main PID: 10914 (dnsmasq)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/dnsmasq.service
           └─10914 /usr/sbin/dnsmasq -x /run/dnsmasq/dnsmasq.pid -u dnsmasq -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,19036,8,2,49aac11d7b6f6446702e54a1607371607a1a41

Sep 24 20:52:03 ecss1 systemd[1]: Starting dnsmasq - A lightweight DHCP and caching DNS server...
Sep 24 20:52:03 ecss1 dnsmasq[10890]: dnsmasq: syntax check OK.
Sep 24 20:52:03 ecss1 systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.

ecss-media-server installation 

Якорь
ecss-media-server ecss-media-server

ecss-media-server package is a mandatory component for processing VoIP traffic. The media server is designed for processing speech and video information over RTP, organizing conferences, recording conversations, playing media files and various combinations of these modes.

...

During installation, a number of questions will be asked in order to create necessary configuration files. If the system is non-redundant, you can refuse MSR settings. A default configuration will be created. If the system is redundant, it is enough to configure only the bind-address at the initial stage, the rest of the settings can be done later. See "Media server configuration".

ecss-restfs installation 

Якорь
ecss-restfs ecss-restfs

RestFS is a component that provides HTTP API for working with files. To install, follow these steps:

...

Setting up RestFS is given in the section RestFS RestFS configuration.

ecss-media-resources installation 

Якорь
ecss-media-resources ecss-media-resources

The package includes a set of system audio files designed for playing answering machine phrases and use in IVR scenarios, as well as a set of tools for working with custom audio files.

...

sudo apt install ecss-media-resources

ecss-web-conf installation 

Якорь
ecss-web-conf ecss-web-conf

Web configurator makes the system management more illustrative and comfortable. Web configurator installation is not mandatory, but recommended.

...

sudo apt install ecss-web-conf

Additional packages optional installation 

Якорь
ecss-other ecss-other

The repository also contains additional packages that can be installed optionally based on the project.

...

Checking interfaces availability by dns names 

Якорь
dns dns

You can check dnsmasq operation by simple ping:

...

All interfaces should be accessible.

Time synchronization on servers 

Якорь
NTP NTP

...

date command displays current system time without parameters.

NTP installation and configuring 

NTP configuration is configured during ecss-node package installation.

...

It is necessary to enter external servers separated by space (by default ntp.ubuntu.com):

Image RemovedImage Added

It is necessary to allow (Yes) or forbid (No) activation of the tos orphan mode (a mode for cluster in which servers independently regulate synchronization). If the system is installed in cluster, then the ECSS servers should have the same time, even if external NTP servers are unavailable. Therefore, it is necessary to select "Yes".

Image RemovedImage Added

The accuracy of cluster time by Strаtum. By default — 7:

Image RemovedImage Added

It is proposed to enter the addresses of neighboring cluster servers to synchronize them with each other. In this example ecss1 is configured, therefore ecss2 address is entered. When configuring ecss2, ecss1 address is entered correspondingly. If there are several servers, you need to list them separated by space.

Image RemovedImage Added

Next, it is proposed to configure subnets  addresses from which other devices are allowed to synchronize with this server:

Image RemovedImage Added

Networks that can have access to this server are specified so that other nodes, as well as other devices could synchronize time with this server. Format of networks specifying : <net_address|net_mask>. If there are several nets, you need to list them separated by space.

Image RemovedImage Added

After installation, settings are saved in /etc/ecss/ecss-ntp.conf file. Example of the resulting file for ecss1 server:

...

As seen, the server stratum value has become equal to 2.

Configuring Token 

Якорь
token token

Token is a USB license protection key. Its availability is necessary for the correct operation of the licensing system and SSW in general. Earlier ECSC servers came with eToken keys for the license purchase, recently new installations are equipped with Rutoken USB keys.

Software installation and Token connection

All the libraries necessary for RuToken operation are installed from ELTEX repository together with ecss-node package.

...

If the key was already connected to the server earlier and it was reconnected, it is recommended to restart the server.

Checking Token operation

To check token operation, you can use pkcs11-tool application. It is possible to check the following:

...

Restarting Token via SSH in case it freezes

To restart USB token, perform the following set of actions:

1. Install usb-reset utility:

   Без форматирования
   sudo snap install usb-reset sudo snap connect usb-reset:hardware-observe core:hardware-observe sudo snap connect usb-reset:raw-usb core:raw-usb Slot 0 (0x0): Aktiv Rutoke

   
   

2. Check that USB token has indeed frozen. Example:

   Без форматирования
   pkcs11-tool --module /usr/lib/ecss/ecss-ds/lib/lpm_storage-3.14.8.70203.423017/priv/x64/librtpkcs11ecp.so -L

   The output should either show nothing at all, or show all slots as empty.

   
   

3. Get the idVendor, idProduct of the USB token. Command for Rutoken:

   Без форматирования
   sudo lsusb -v | grep -C 10 "Rutoken ECP"

   Find the parameters idVendor, idProduct in the specified output:

   Без форматирования

   lsusb -v | grep -C 10 "Rutoken ECP" FIXME: alloc bigger buffer for device capability descriptors bDescriptorType 1 bcdUSB 2.00 bDeviceClass 0 (Defined at Interface level) bDeviceSubClass 0 bDeviceProtocol 0 bMaxPacketSize0 16 idVendor 0x0a89 idProduct 0x0030 bcdDevice 1.00 iManufacturer 1 Aktiv iProduct 2 Rutoken ECP iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 93 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0x80

   
   

4. Restart the USB device:

   Без форматирования
   sudo usb-reset <idVendor>:<idProduct> Пример: sudo usb-reset 0a89:0030

   
   

5. Check that the slot(s) appeared:

   Без форматирования
   pkcs11-tool --module /usr/lib/ecss/ecss-ds/lib/lpm_storage-<VERSION>/priv/x64/librtpkcs11ecp.so -L Available slots: Slot 0 (0x0): Aktiv Rutoken ECP 00 00 ...

   
   

Token operation problem on DEPO servers

If tokens disconnection from DEPO servers is periodically recorded, then syslog should be checked for EHCI driver errors. If errors are present, then it is necessary to go to Server BIOS and enable XHCI mode (BIOS path: Advanced/USB Configuration: XHCI Pre-Boot Driver — Enabled, XHCI — enabled).

Configuring listen interface for epmd service 

Якорь
epmd epmd

Example of listen interface configuring for epmd service in accordance with the network configuration given in Configuring network interfaces section.

For the ecss1 server, the following sequence of actions must be performed:

...

System start and activation 

Якорь
activate activate

To start and activate the operating system, perform the following set of actions:

...

Connect to the distributed CoCon Management Console:

ssh admin@localhost -p8023

...

At this stage, the system is considered fully installed and ready for configuration.

Cluster system installation features

Installing ECSS-10 on a cluster 

Якорь
inst_cluster inst_cluster

Host preparation

When installing ECSS-10 system in a cluster, it is necessary to perform the following on both servers in accordance with the project:

• Operating system installation;
• Network Setup;
• Necessary software installation;

• ECSS packages installation;

• NTP configuring;
• EPMD configuring;
• Token verification. 

Setting cluster name

It is necessary to specify the same cluster name on both servers for system operation. To do this, open mycelium1.config file in text editor:

...

Configuring RestFS for a cluster 

Якорь
restfs_config restfs_config

To work in a cluster, you need to configure RestFS operation based on a GlusterFS server.

Installing and configuring snmpd

Install Net-SNMP agent:

sudo aptitude install snmpd

...

sudo netstat -tulpan | grep snmpd
udp        0      0 127.0.0.1:3161          0.0.0.0:*                           7723/snmpd     

man keepalived

# Set the default VRRP version to use
vrrp_version <2 or 3>        # default version 2

global_defs {
    vrrp_version 3          # VRRP protocol version (2 or 3)
    script_user nobody      # system user with limited rights, from which accessibility check scripts will be launched
    enable_script_security  # do not run scripts as root if part of the path to them is writable for normal users
}

include /etc/keepalived/sip.conf
include /etc/keepalived/mysql.conf
include /etc/keepalived/ipnet.conf

vrrp_script check_sip {
    script "/usr/bin/ecss_pa_sip_port 65535"
    interval 2
    timeout 2
}

# Address configuration for the first virtual address of the SIP adapter
vrrp_instance SIP1 {
    state MASTER                  # Initial state at a start
    interface <network_interface> # Name of the network interface on which the VRRP protocol will run
    virtual_router_id <ID>        # The unique identifier of the router (0..255)
    priority 100                  # Priority (0..255) the higher the more
    advert_int 1                  # Notification sending interval (sec)
    preempt_delay 60              # Master wait interval at daemon start (sec) at BACKUP initial state

    unicast_src_ip <src_real IP>  #  Own real IP address
    unicast_peer {
        <real_remote IP>          # Neighbour real IP адрес address
    }

    virtual_ipaddress {
        # Virtual IP address and a mask
        # dev - network interface on which virtual address will operate
        # label - virtual interface label (for ease of identification)
        <virtual_sip_IP>/<netmask> dev <>  label <label>
    }

    track_script {
        check_sip
    }
}

# Address configuration for the second virtual address of the SIP adapter
vrrp_instance SIP2 {
    state MASTER                  # Initial state at a start
    interface <network_interface> # Name of the network interface on which the VRRP protocol will run
    virtual_router_id <ID>        # The unique identifier of the router (0..255)
    priority 50                   # Priority (0..255) the higher the more
    advert_int 1                  # Notification sending interval (sec)
    preempt_delay 60              # Master wait interval at daemon start (sec) at BACKUP initial state

    unicast_src_ip <src_real IP>  #  Own real IP address
    unicast_peer {
        <real_remote IP>          # Neighbour real IP адрес address
    }

    virtual_ipaddress {
        # Virtual IP address and a mask
        # dev - network interface on which virtual address will operate
        # label - virtual interface label (for ease of identification)
        <virtual_sip_IP>/<netmask> dev <>  label <label>
    }

    track_script {
        check_sip
    }
}

vrrp_script check_sip {
    script "/usr/bin/ecss_pa_sip_port 65535"
    interval 2
    timeout 2
}

# Address configuration for the first virtual address of the SIP adapter
vrrp_instance SIP1 {
    state BACKUP                  # Initial state at a start
    interface <network_interface> # Name of the network interface on which the VRRP protocol will run
    virtual_router_id <ID>        # The unique identifier of the router (0..255)
    priority 50                   # Priority (0..255) the higher the more
    advert_int 1                  # Notification sending interval (sec)
    preempt_delay 60              # Master wait interval at daemon start (sec) at BACKUP initial state 

    unicast_src_ip <src_real IP>  # Own real IP address
    unicast_peer {
        <real_remote IP>          # Neighbour real IP адрес address
    }

    virtual_ipaddress {
        # Virtual IP address and a mask
        # dev - network interface on which virtual address will operate
        # label - virtual interface label (for ease of identification)
        <virtual_sip_IP>/<netmask> dev <>  label <label>
    }

# Address configuration for the second virtual address of the SIP adapter
vrrp_instance SIP2 {
    state MASTER                  # Initial state at a start
    interface <network_interface> # Name of the network interface on which the VRRP protocol will run
    virtual_router_id <ID>        # The unique identifier of the router (0..255)
    priority 100                  # Priority (0..255) the higher the more
    advert_int 1                  # Notification sending interval (sec)
    preempt_delay 60              # Master wait interval at daemon start (sec) at BACKUP initial state

    unicast_src_ip <src_real IP>  #  Own real IP address
    unicast_peer {
        <real_remote IP>          # Neighbour real IP адрес address
    }

    virtual_ipaddress {
        # Virtual IP address and a mask
        # dev - network interface on which virtual address will operate
        # label - virtual interface label (for ease of identification)
        <virtual_sip_IP>/<netmask> dev <>  label <label>
    }

    track_script {
        check_sip
    }
}

# First mysql node configuration:

vrrp_script check_mysql {
    script "/usr/bin/mysql --defaults-file=/etc/mysql/debian.cnf -e 'SELECT 1;'"
    user root
    interval 2
    fall 1
    timeout 2
}

vrrp_instance MySQL {
    state MASTER                     # Initial state at a start
    interface <network_interface>    # Initial state at a start
    virtual_router_id <ID>           # Unique router id (0..255)
    priority 100                     # Priority (0..255) the higher the more
    advert_int 1                     # Notification sending interval (sec)
    preempt_delay 60                 # Master wait interval at daemon start (sec) at BACKUP initial state

    unicast_src_ip  <src_real IP>    # Own real IP address
    unicast_peer {
         <real_remote IP>            # Neighbour real IP адрес address
    }

    virtual_ipaddress {
        # Virtual IP address and a mask
        # dev - network interface on which virtual address will operate
        # label - virtual interface label (for ease of identification)
        <virtual_sip_IP>/<netmask> dev <>  label <label>
   }

    track_script {
        check_mysql
    }
}

#  Second mysql node configuration:

vrrp_script check_mysql {
    script "/usr/bin/mysql --defaults-file=/etc/mysql/debian.cnf -e 'SELECT 1;'"
    user root
    interval 2
    fall 1
    timeout 2
}

vrrp_instance MySQL {
    state BACKUP                     # Initial state at a start
    interface <network_interface>    # Name of the network interface, on which VRRP will operate
    virtual_router_id <ID>           # Unique router id (0..255)
    priority 50                      # Priority (0..255) the higher the more
    advert_int 1                     # Notification sending interval (sec)
    preempt_delay 60                 # Master wait interval at daemon start (sec) at BACKUP initial state

    unicast_src_ip  <src_real IP>    # Own real IP address
    unicast_peer {
         <real_remote IP>            # Neighbour real IP адрес address
    }

    virtual_ipaddress {
        # Virtual IP address and a mask
        # dev - dev - network interface on which virtual address will operate
        # label - virtual interface label (for ease of identification)
        <virtual_sip_IP>/<netmask> dev <>  label <label>
   }

    track_script {
        check_mysql
    }
}

# First node configuration:

vrrp_script check_ipnet {
    script "/usr/bin/ecss_ipnet_port 65531"
    interval 1
    fall 1
    rise 1
}

vrrp_instance IPNET {
    state MASTER                   # Initial state at start
    interface <network_interface>  # Name of the network interface, on which VRRP will operate
    virtual_router_id <ID>         # Unique router id (0..255)
    priority 100                   # Priority (0..255) the higher the more
    advert_int 0.05                # Notification sending interval (sec)
    preempt_delay 60               # Master wait interval at daemon start (sec) at BACKUP initial state

    unicast_src_ip  <src_real IP>  # Own real IP address
    unicast_peer {
         <real_remote IP>          # Neighbour real IP адрес address
    }

    virtual_ipaddress {
        # Virtual IP address and a mask
        # dev - network interface on which virtual address will operate
        # label - virtual interface label (for ease of identification)
        <virtual_sip_IP>/<netmask> dev <>  label <label>
    }

    track_script {
        check_ipnet
    }
}

# Second node configuration:

vrrp_script check_ipnet {
    script "/usr/bin/ecss_ipnet_port 65531"
    interval 1
    fall 1
    rise 1
}

vrrp_instance IPNET {
    state BACKUP
    interface <network_interface>
    virtual_router_id <ID>
    priority 50
    advert_int 0.05
    preempt_delay 60

    unicast_src_ip  <src_real IP>
    unicast_peer {
         <real_remote IP>
    }

    virtual_ipaddress {
        <virtual_sip_IP>/<netmask> dev <>  label <label>
    }

    track_script {
        check_ipnet
    }
}

System start

After everything is configured, proceed to system start and activation. The sequence of actions:

...

After successful subsystems start on ecss1 and license activation all ecss2 services can be started.

Checking the installation and entry of the system into the cluster 

Якорь
nodes-info nodes-info

To check the status of the cluster nodes, you need to log in to the command console (CoCon) on any of the servers:

...

This completes the installation stage. After checking it can be proceeded to configuration.

Decommissioning of single server

If for some reason you need to decommission first server of ecss1 cluster out of service, then do the following on second ecss2 server:

...

127.0.0.1      localhost
127.0.1.1      ecss2
192.168.1.2    ecss1

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Checking the correctness of installation procedures

After completing all the installation procedures, you should check the correctness and completeness of the performed actions. To do this, use the checklist given in the section ECSS-10 installation checklist.