Migration from 1.4.0 to 1.5.0

General information

This document describes the process of migrating the system from version 1.4.0 to version 1.5.0.
Migration includes updating the PostgreSQL and ClickHouse database schemas, as well as transferring data.

System requirements

Recommended resources for successful migration (based on a system with 3,000 employees, 2,200 passes, and 300 cameras):

• Free disk space: no less than 25 GB;
• RAM: no less than 16 GB;
• Services: PostgreSQL and ClickHouse must be running on the same host;
• Internet connection: stable (for downloading the migration archive);
• Installed software: clickhouse-client.

Migration scenarios

Preliminary steps

Creating a temporary directory

Create a temporary folder for backup and migration files in the user's home directory:

mkdir ~/backup
cd ~/backup
mkdir db_migration-1.4.0-1.5.0

Database backup

Before dumping the database, it is necessary to stop the services.

sudo systemctl stop evi-core evi-scud evi-live evi-archive evi-analyzer

echo "SELECT pg_terminate_backend(pid) FROM pg_stat_activity where usename='core'" | sudo -u postgres psql -A -t

Perform a dump of the current PostgreSQL database:

sudo -u postgres pg_dump core > evi-backup.dump

Then proceed to the Migration on the current server or Migration on a new server step, depending on requirements.

Migration on the current server

Uploading and unpacking the migration archive

While in the ~/backup folder, run the following commands:

wget https://archive.eltex-co.ru/evi-raw/evi-1.5.0/db_migration-1.4.0-1.5.0.tar
tar -xf db_migration-1.4.0-1.5.0.tar -C db_migration-1.4.0-1.5.0
cd db_migration-1.4.0-1.5.0

Removing the current EVI Platform

sudo apt purge evi-core-web evi-core evi-scud evi-live evi-archive evi-analyzer nginx

Preparation for installation

Perform the necessary preliminary steps as described in the Preparation for installation section.

Installing the ClickHouse database

Since the PostgreSQL database is already installed, the next step is to install ClickHouse: see the Installing the database section, ClickHouse database subsection.

Running the migration script

sudo bash migrate.sh <host> <user> <password> <psql_db_name_from> <psql_db_name_to> <psql_port> <ch_def_user_password> <ch_db_name>

• host — IP address of the host with PostgreSQL and ClickHouse;
• user — username for the database (will be created if necessary);
• password — user password;
• psql_db_name_from — name of the PostgreSQL database from which the migration is performing;
• psql_db_name_to — name of the PostgreSQL database to which the migration will be performed;
• psql_port — PostgreSQL port;
• ch_def_user_password — password for the ClickHouse user;
• ch_db_name — name of the ClickHouse database to which the migration will be performed.

sudo bash migrate.sh localhost core core core core 5432 core core

After successfully executing the migration script, the "Restoring DB done <3" message will appear in the console.

Then proceed to the Verification of configuration files section.

Migration on a new server

Preparing for installation

Perform the necessary preliminary steps as described in the Preparation for installation section.

Installing the databases

Install and configure PostgreSQL and ClickHouse as described in the Installing the database section.

Creating a backup folder with a backup copy of data

Back up all data as described in the Preliminary steps section.

Transferring prepared files

Copy the backup folder along with all its contents from the source server to the new one.

For data transfer, it is recommended to use the scp utility, which provides secure copying via SSH.

scp -r ~/backup <username>@<remote-server>:~

• user — username of the new server;
• remote-server — address of the new server.

After the transfer, switch to the backup folder in the new server:

cd ~/backup

Creating a user in PostgreSQL

sudo -u postgres psql -c "CREATE USER core; ALTER USER core WITH PASSWORD 'core'; ALTER USER core WITH SUPERUSER;"

Restoring a database from a dump

Download database of the current version:

sudo -u postgres createdb core
sudo -u postgres psql -d core -f evi-backup.dump

Downloading and unpacking the migration archive

wget https://archive.eltex-co.ru/evi-raw/evi-1.5.0/db_migration-1.4.0-1.5.0.tar
tar -xf db_migration-1.4.0-1.5.0.tar -C db_migration-1.4.0-1.5.0
cd db_migration-1.4.0-1.5.0

Running the migration script

sudo bash migrate.sh <host> <user> <password> <psql_db_name_from> <psql_db_name_to> <psql_port> <ch_def_user_password> <ch_db_name>

• host — IP address of the host with PostgreSQL and ClickHouse;
• user — username for the database (will be created if necessary);
• password — user password;
• psql_db_name_from — name of the PostgreSQL database from which the migration is performing;
• psql_db_name_to — name of the PostgreSQL database to which the migration will be performed;
• psql_port — PostgreSQL port;
• ch_def_user_password — password for the ClickHouse user;
• ch_db_name — name of the ClickHouse database to which the migration will be performed.

sudo bash migrate.sh localhost core core core core 5432 core core

After successfully executing the migration script, the "Restoring DB done <3" message will appear in the console.

Then proceed to installing and configuring EVI services, as described in the Ubuntu 22.04.5 LTS section.

Verification of configuration files 

Version 1.5.0 includes changes to the structure and settings of the configuration files, which can be viewed at Platform configuration section.

Verifying the migration result

After completing the migration, perform the following steps:

1. Ensure that all services run without errors;
2. Verify data availability (users, passes, events);
3. Check the service log files for database connection errors.