Searching the knowledge base
Inhaltsverzeichnis
< Alle Themen

Natrix Management platform – Installation

Aktualisiert

The Natrix Management platform will entirely run on Linux (Ubuntu) system. Beside Debian this is a widely used OS especially in cloud environments. Most of the applications are running as Docker containers which makes it very easy to set them up. Let’s start with preparing everything for a ‘dockerized’ environment.

Installation of Docker

Installation of the Docker Engine

As always when new applications shall be installed on a Linux system, it is highly recommended to update the package list first. Let’s assume we operate under the ‘root’ account.

apt update

Next, we install some prerequisite packages so that apt can consume packages over HTTPS:

apt install apt-transport-https ca-certificates curl software-properties-common

Next, we install some prerequisite packages so that apt can consume packages over HTTPS:

apt install apt-transport-https ca-certificates curl software-properties-common

Then we add the GPG key for the official Docker repository to your system:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add –

Let’s now add the Docker repository to our APT sources:

add-apt-repository “deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable”

Another apt update updates the package database with the Docker packages from the newly added repo

apt update

Making sure that we are installing from the Docker repo and not the standard Ubuntu repo:

apt-cache policy docker-ce
docker-ce:
  Installed: (none)
  Candidate: 5:20.10.8~3-0~ubuntu-focal
  Version table:
     5:20.10.8~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.7~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.6~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.5~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.4~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.3~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.2~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.1~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:20.10.0~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:19.03.15~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:19.03.14~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:19.03.13~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:19.03.12~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:19.03.11~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:19.03.10~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages
     5:19.03.9~3-0~ubuntu-focal 500
        500 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages

If we see this or a similar output that confirms that we intend to install Docker from the Docker repository we can finally install the Docker engine

apt install docker-ce

Docker should now be installed, the daemon started and the process to start at boot enabled. Check if it is running:

systemctl status docker
. docker.service – Docker Application Container Engine
   Loaded: loaded (/lib/ ystem/system/docker.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-09-30 15:24:57 CEST; 36s ago
     Docs: https://docs.docker.com
 Main PID: 10761 (dockerd)
    Tasks: 44
   Cgroup: /system.slice/docker.service
           +-10761 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
           +-10973 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 9000 -container-ip 172.17.0.2 -container-port 9000
           +-10980 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 9000 -container-ip 172.17.0.2 -container-port 9000
           +-10994 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 8000 -container-ip 172.17.0.2 -container-port 8000
           +-11001 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 8000 -container-ip 172.17.0.2 -container-port 8000

Let’s make sure that Docker is automatically started after every new start of our system:

systemctl enable docker
Synchronizing state of docker.service with SysV service script with /lib/ ystem/ ystem-sysv-install.
Executing: /lib/ ystem/ ystem-sysv-install enable docker

Finally, we can check if our Docker Engine is functional by launching the ‘Hello World’ container

docker run hello-world
Unable to find image ‘hello-world:latest’ locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete
Digest: sha256:393b81f0ea5a98a7335d7ad44be96fe76ca8eb2eaa76950eb8c989ebf2b78ec0
Status: Downloaded newer image for hello-world:latest
 
Hello from Docker!
This message shows that your installation appears to be working correctly.
 
To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the “hello-world” image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.
 
To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash
 
Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/
 
For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Installation of the Docker-Compose

In the case of applications that depend on several services, the organization of all containers for joint starting, communicating and shutting down can quickly become unwieldy. Docker Compose is a tool that allows you to run multi-container application environments based on definitions set in a YAML file. It uses service definitions to build fully customizable, multi-container environments that can share networks and data volumes.

The following command installs the version 2.0.0 of docker-compose from GitHub:

curl -L “https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)” -o /usr/local/bin/docker-compose

Next, set the correct permissions for the docker-compose command to be executable:

chmod +x /usr/local/bin/docker-compose

After the installation we can check the version and accuracy of the installation:

docker-compose –version
docker-compose version 1.29.2, build 5becea4c

Installation of Portainer

We will install each of our functional applications which run as Docker containers with a dedicated volume.

The data generated and used by containers are not persisted after we restart or remove containers. So, we can use Docker volumes and bind mounts to manage data in Docker containers to solve this issue. We can use it to persist data in a container or share data between containers.

For Portainer we create a separate volume called docker_data_portainer:

docker volume create portainer_data

The Portainer docker image will be installed with pointing to this volume. Furthermore, we enable restart of the container whenever the Docker engines restarts.

docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

The first launch of the Portainer container, which runs on port 9000, requires us to change the password of the user Admin.

We connect to our local Docker Engine

and see our first container up and running:

Installation of Eclipse Kapua™

The next container which we are going to install is Eclipse Kapua™.

We clone the Eclipse Kapua™ Docker container into the opt directory:

git clone https://github.com/eclipse/kapua.git kapua
Cloning into ‘kapua’...
remote: Enumerating objects: 236072, done.
Remote: Counting objects: 100% (13155/13155), done.
Remote: Compressing objects: 100% (4393/4393), done.
Remote: Total 236072 (delta 5674), reused 12462 (delta 5242), pack-reused 222917
Receiving objects: 100% (236072/236072), 39.41 MiB | 6.95 MiB/s, done.
Resolving deltas: 100% (107008/107008), done.

For deploying the Eclipse Kapua™ container(s), a YAML script is being provided. This YAML script is actually called from docker-compose, when the docker-deploy script initiates docker-compose to build and deploy the various different containers of Eclipse Kapua™.

There are in total 7 containers of which Eclipse Kapua™ is composed of.

  • compose_events-broker_1            -> Eclipse Kapua™ Service Event Broker
  • compose_es_1                                 -> ElasticSearch Time Series database for Telemetry data
  • compose_db_1                                 -> Eclipse Kapua™ internal PostgreSQL database
  • compose_broker_1                         -> Eclipse Kapua™ MQTT broker
  • compose_kapua-api_1                    -> Eclipse Kapua™ Restful API server
  • compose_kapua-console_1           -> Eclipse Kapua™ Console application
  • compose_job-engine_1 ^             -> Eclipse Kapua™ Job Engine

There are some more containers configured in the YAML script which unfortunately cause an issue when downloading. These are

  • consumer-telemetry                       
  • consumer-lifecycle                                                         

They can’t be pulled from the Eclipse Kapua Docker repository as they seem to be not present there. (Eclipse IoT(8), 2021)

ERROR: pull access denied for kapua/kapua-consumer-lifecycle, repository does not exist or may require ‘docker login’: denied: requested access to the resource is denied

As a consequence, we commented these containers out in the provided YAML file, which can be found under /opt/kapua/kapua/deployment/docker/compose/docker-compose.yml

version: ‘3.1’

services:
  db:
    image: kapua/kapua-sql:${IMAGE_VERSION}
    ports:
      - 8181:8181
      - 3306:3306
  es:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - cluster.name=kapua-datastore
      - discovery.type=single-node
      - transport.host=_site_
      - transport.ping_schedule=-1
      - transport.tcp.connect_timeout=30s
  events-broker:
    image: kapua/kapua-events-broker:${IMAGE_VERSION}
    ports:
      - 5672:5672
  broker:
    image: kapua/kapua-broker:${IMAGE_VERSION}
    expose:
      - 1893
    ports:
      - 1883:1883
      - 8883:8883
      - 5682:5672
      - 61614:61614
    depends_on:
      - db
      - events-broker
    environment:
      - KAPUA_DISABLE_SSL
      - KAPUA_DISABLE_DATASTORE
      - KAPUA_CRT
      - KAPUA_CA
      - KAPUA_KEY
      - KAPUA_KEY_PASSWORD
      - KAPUA_KEYSTORE
      - KAPUA_KEYSTORE_PASSWORD
      - LOGBACK_LOG_LEVEL
  #consumer-telemetry:
  #  image: kapua/kapua-consumer-telemetry:${IMAGE_VERSION}
  #  ports:
  #    - 8090:8080
  #    - 8001:8001
  #  depends_on:
  #    - db
  #    - es
  #    - events-broker
  #    - broker
  #  environment:
  #    - BROKER_URL
  #    - LOGBACK_LOG_LEVEL
  #consumer-lifecycle:
  #  image: kapua/kapua-consumer-lifecycle:${IMAGE_VERSION}
  #  ports:
  #    - 8091:8080
  #    - 8002:8001
  #  depends_on:
  #    - db
  #    - events-broker
  #    - broker
  #  environment:
  #    - BROKER_URL
  #    - LOGBACK_LOG_LEVEL
  kapua-console:
    image: kapua/kapua-console:${IMAGE_VERSION}
    ports:
      - 8080:8080
      - 8443:8443
    depends_on:
      - broker
      - db
      - es
      - events-broker
    environment:
      - KAPUA_DISABLE_SSL
      - KAPUA_DISABLE_DATASTORE
      - KAPUA_CA
      - KAPUA_CRT
      - KAPUA_KEY
      - KAPUA_KEY_PASSWORD
      - KAPUA_KEYSTORE
      - KAPUA_KEYSTORE_PASSWORD
      - LOGBACK_LOG_LEVEL
      - KAPUA_CONSOLE_URL
      - KAPUA_OPENID_JWT_ISSUER
      - KAPUA_OPENID_CLIENT_ID
      - KAPUA_OPENID_CLIENT_SECRET
      - KAPUA_OPENID_JWT_AUDIENCE
      - KAPUA_OPENID_AUTH_ENDPOINT
      - KAPUA_OPENID_TOKEN_ENDPOINT
      - KAPUA_OPENID_LOGOUT_ENDPOINT
      - KEYCLOAK_URL
      - KEYCLOAK_CLIENT_ID
      - KEYCLOAK_REALM
  kapua-api:
    image: kapua/kapua-api:${IMAGE_VERSION}
    ports:
      - 8081:8080
      - 8444:8443
    depends_on:
      - broker
      - db
      - es
      - events-broker
    environment:
      - KAPUA_DISABLE_SSL
      - KAPUA_DISABLE_DATASTORE
      - KAPUA_CA
      - KAPUA_CRT
      - KAPUA_KEY
      - KAPUA_KEY_PASSWORD
      - KAPUA_KEYSTORE
      - KAPUA_KEYSTORE_PASSWORD
      - LOGBACK_LOG_LEVEL
 job-engine:
    image: kapua/kapua-job-engine:${IMAGE_VERSION}
    expose:
      - 8080
    depends_on:
      - broker
      - db
      - events-broker
    environment:
      - KAPUA_DISABLE_SSL
      - KAPUA_DISABLE_DATASTORE
      - KAPUA_CA
      - KAPUA_CRT
      - KAPUA_KEY
      - KAPUA_KEY_PASSWORD
      - KAPUA_KEYSTORE
      - KAPUA_KEYSTORE_PASSWORD
      - LOGBACK_LOG_LEVEL

Please check for the most recent version available to download: https://github.com/eclipse/kapua

By default, the latest version of images will be used, but we recommend specifying the version we want to use in the ‘IMAGE_VERSION’ environment variable.

export IMAGE_VERSION=1.5.2

When we run the /tmp/kapua/kapua/deployment/docker/unix/docker-deploy.sh script, we get now the following output (presuming all containers have been pulled beforehand already). 

/tmp/kapua/kapua/deployment/docker/unix# ./docker-deploy.sh
Deploying Eclipse Kapua...
Creating compose_events-broker_1 ... done
Creating compose_es_1            ... done
Creating compose_db_1            ... done
Creating compose_broker_1        ... done
Creating compose_kapua-api_1     ... done
Creating compose_kapua-console_1 ... done
Creating compose_job-engine_1 ... done
Deploying Eclipse Kapua... DONE!

In Portainer we see that all 7 Eclipse Kapua™ containers have been successfully started (beside Portainer and Grafana):

The last step is to configure the containers in a way that they – like Portainer – restart every time the Docker engine starts.

docker update –restart unless-stopped compose_kapua-console_1
compose_kapua-console_1

docker update –restart unless-stopped compose_kapua-api_1
compose_kapua-api_1

docker update –restart unless-stopped compose_broker_1
compose_broker_1

docker update –restart unless-stopped compose_events-broker_1
compose_events-broker_1

docker update –restart unless-stopped compose_db_1
compose_db_1

docker update –restart unless-stopped compose_es_1
compose_es_1

docker update –restart unless-stopped compose_job-engine_1
compose_job-engine_1

Now it’s the best time to login into the Eclipse Kapua™ console the first time.  It runs on port 8080.tWe are being welcomed by a login screen, and provide the default credentials which are:

User: kapua-sys

Password: kapua-password

Finally, here we are:

In the individual Maker projects, we will learn how to customize Eclipse Kapua™ in more detail.

Installation of Grafana

Next container which we need to install is the Grafana data visualization application.

docker run -d -p 3000:3000 –-restart always –name=grafana8 -v grafana-storage:/var/lib/grafana grafana/grafana:8.1.2
Unable to find image ‘grafana/grafana:8.1.2’ locally
8.1.2: Pulling from grafana/grafana
540db60ca938: Pull complete
475d6aa6cde2: Pull complete
86c565d1875f: Pull complete
bacbab00d598: Pull complete
eba2484373d9: Pull complete
4f4fb700ef54: Pull complete
de780c7f2383: Pull complete
40175e15d294: Pull complete
Digest: sha256:811ee7d685fe45e5625928716d189c518f2b96edaa86122a04cc6faf1e988180
Status: Downloaded newer image for grafana/grafana:8.1.2
52b32412bcbcbf7d454844cbdc031d5f3aa073fda1f96f29232e160dd37f5a59

The Grafana container is automatically started (port 3000) , and we can launch the login page.

The default login credentials after a first install are:

User: admin

Password: admin

After a first successful login, you will be prompted to enter a new password.

Once we did this, the Grafana landing pages appears.

Installation of Telegraf

The installation of the Telegraf is quite simple. We create a directory e.g. under /usr/share which we call telegraf. In this directory we do place two files. One file is the ‘telegraf.conf’ file  and the other is the docker-compose file docker-compose.yml which we need to create first.

Telegraf configuration file

We can download a sample telegraf.conf file from the influxdata / telegraf GitHub presence. (Telegraf(1), 2021)

Project specific configuration settings are being discussed in the respective Maker projects.

docker-compose file for Telegraf

The docker-compose file for Telegraf looks like the following:

version: '3'
services:
  telegraf:
    image: telegraf
    container_name: telegraf
    restart: always
    environment:
        HOST_PROC: /rootfs/proc
        HOST_SYS: /rootfs/sys
        HOST_ETC: /rootfs/etc
    volumes:
        - ./telegraf.conf:/etc/telegraf/telegraf.conf:ro
        - /var/run/docker.sock:/var/run/docker.sock:ro
        - /sys:/rootfs/sys:ro
        - /proc:/rootfs/proc:ro
        - /etc:/rootfs/etc:ro

Creating the Telegraf container

If we now look into our /usr/share/telegraf folder, we need to see the just two created files:

/usr/share/telegraf# ls -l
total 340
-rw-r--r-- 1 root root    430 Okt 29 16:38 docker-compose.yml
-rw-r--r-- 1 root root 343408 Okt 29 16:40 telegraf.conf

Let’s create the container:

docker-compose -f docker-compose.yml up -d
Creating telegraf ... done

All configuration settings need to be done using the telegraf.conf file in this directory!