The SmartHome Controller CCU from HomeMatic©

Aktualisiert8. November 2024

When it comes to SmartHome we are irretrievable confronted with the question about the commercial actors and sensors we should use. Beside their functionality and of course price they all vary in two important aspects: the used radio frequency and the protocol to communicate with their respective control. Fortunately, the Natrix gateway is open enough to work with many different radio standards such as e.g. ZWave and Zigbee. openHAB as the Natrix-Gateway SmartHome control center is supporting drivers from many vendors which provide modules for those radio standards.

While the Natrix-Project appreciates this openness and flexibility, we also want to reduce complexity in the management of the Natrix-SmartHome solution. Therefore, we focus on one specific vendor for the IoT gadgets we use, as for example thermostats or thermometers. However, we will add the one or the other gadget from another vendor to our framework just to prove evidence on the openness and flexibility of the Natrix-Gateway concept.

The preferred vendor the Natrix-Project is collaborating with is HomeMatic©. (Homematic(1), 2021). The portfolio of this company has proven excellent stability, usability and flexibility to a fair price. They come with an own Gateway control, called CCU – currently in version 3 (CCU3).

The good thing is that there is an open source project out there, called RaspberryMatic, so we don’t rely on this proprietary Gateway control from the vendor Homematic©. (RaspberryMatic(1), 2021)

RaspberryMatic is a feature-rich operating system alternative based on the open OCCU environment in order to be able to operate the core of a HomeMatic© control center (CCU) on our own hardware (e.g. Raspberry, Tinkerboard, etc.). So, compatible with the Natrix-Gateway.

The RaspberryMatic operating system was developed by users for users, is independent, free of charge for „normal“ HomeMatic© users and 100% compatible with a standard CCU. In addition to suitable hardware, only a suitable radio module (RPI-RF-MOD, HM-MOD-RPI-PCB) is required for operation, which is plugged directly into the GPIO interface of the Natrix-Gateway.

The HM-MOD-RPI-PCB radio module

While we are able to run the CCU3 – thanks to the RaspberryMatic open source project – we still need to find a way how to connect the Natrix Gateway to all the sensors and actors which are offered by the HomeMatic© program. This can be done by using a little piece of hardware – a radio module -which is called HM-MOD-RPI-PCB.

The HM-MOD-RPI-PCB HomeMatic radio module for Raspberry Pi is a circuit board that can be plugged onto the GPIO interface of the Raspberry Pi and thus serve as an interface to HomeMatic© devices.

The actual radio module is called HM-MOD-RPI-UART. It has five single-pole sockets with a grid dimension of 2mm. The additional board for connection to the GPIO is called TRX1. This module has two six-pole sockets. The additional board essentially contains some capacitors and allows a direct connection to the GPIO of our Natrix-Gateway.

This radio module comes along with an SDK – the Homematic Open-Central-Control-Unit Software Development Kit, short OCCU. Basically, this SDK consists out of binary files for several hardware architectures (as for Raspberry PI for example), the CCU3 Web-Interface and some other CCU3 open source files and toolchains.

The radio module establishes a connection to HomeMatic© devices on an 868MHz frequency band. The module is coupled to the Natrix-Gateway via the serial interface that can evaluate this serial interface and then process the information further.

Both the radio module and the OCCU SDK are provided by the company ELV. (ELV(1), 2021)

Installation of the HM-MOD-RPI-PCB

The Natrix-Gateway must be disconnected from any power supply when working on the GPIO pins. The module gets plugged onto the first 6 pins of the GPIO so that it protrudes into the board of the Natrix-Gateway.

Installing RaspberryMagic

RaspberryMagic – as already mentioned – emulates the CCU software which runs on the original HomeMatic© Controller on the Natrix-Gateway (or on any other compatible Raspberry PI). RaspberryMatic relies on the OCCU.

Basic virtual installation of RaspberryMatic

RaspberryMatic comes with a bunch of virtualized platforms it provides support for. We go with Docker as this is the most common one, suitable for our Natrix-Gateway and it is a technology which we use it on the Natrix-Management platform as well.

Installing Docker on the Natrix-Gateway

We prefer installing Docker manually although there is a script available which manages many manual installation steps automatically. In the first step, the repositories are updated, and the packages required for Docker are installed. (Docker(0), 2021)

apt-get update
apt-get install -y -qq --no-install-recommends apt-transport-https ca-certificates curl

Now the key for Docker can be added to the key ring and checked. If this key is missing, the origin of the Docker package cannot be trusted, and installation is therefore not possible.

curl -fsSL "https://download.docker.com/linux/raspbian/gpg" | sudo apt-key add -
apt-key finger 9DC858229FC7DD38854AE2D88D81803C0EBFCD88

The reference to the Docker repository is not known under a freshly installed Raspbian. Therefore, it has to be included in the list of sources. The following command creates a new “docker.list” file in the “/etc/apt/sources.list.d/” directory with a reference to the Docker repository. Instead of “stable” you can also specify “test” in order to install the current test version instead of the stable version.

tee /etc/apt/sources.list.d/docker.list \
    <<< "deb [arch=armhf] https://download.docker.com/linux/raspbian buster stable"

The preparations for installing Docker are now complete so that the actual installation can be carried out. To do this, the repositories are updated again, and Docker is then installed itself. The update can be carried out in the future using the same command. The preparatory steps are then no longer required.

apt-get update
apt-get install -y --no-install-recommends docker-ce

The usermod command with -aG will append (-a) the specified user (pi) to the group listed after the -G flag. In the above case, we are adding the pi user to the docker group. This enables the user pi to run docker commands without sudo. You can obviously substitute any other username if you are not using the default pi user.

groupadd docker

usermod -aG docker pi

In some circumstances it has been discovered that after a reboot or after some hours of operation, the bridge interface ‘docker0’ has no inet address anymore. Means, it loses its IPV4 static address. A workaround for the Debian running on our Natrix Gateway is the need to hardcode the bridge IPv4 address. We need to create a file with the extension ‘.network’ in the folder /etc/system/network and provide the following content to it:

nano /etc/systemd/network/98-docker0.network

[Match]

Name=docker0

[Network]

Address=172.17.0.1/16

Let’s ensure that the ‘system-networkd.service’ is started automatically after rebooting the Natrix Gateway.

‘systemd-networkd’ is a system daemon that manages network configurations. It detects and configures network devices as they appear; it can also create virtual network devices.

systemctl start systemd-networkd.service

After a reboot, the docker0 interface should always have the static IP address 172.17.0.1 assigned to its internal ‘bridge’ network.

ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:99:96:8e:f7  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

We can check this also within Docker itself:

docker network inspect bridge
[
    {
        "Name": "bridge",
        "Id": "a74703508f21d5885916f1d0748f466a1c34c48b73dbc664cd349e8eddad5957",
        "Created": "2021-11-10T09:00:25.451119895+01:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

We can test the accurate functionality of the docker daemon which started right after the installation by running a test container provided by Docker:

docker run -it hello-world

Unable to find image 'hello-world:latest' locally

latest: Pulling from library/hello-world

9b157615502d: Pull complete

Digest: sha256:37a0b92b08d4919615c3ee023f7ddb068d12b8387475d64c622ac30f45c29c51

Status: Downloaded newer image for hello-world:latest

Hello from Docker!

This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:

 1. The Docker client contacted the Docker daemon.

 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.

    (arm32v7)

 3. The Docker daemon created a new container from that image which runs the

    executable that produces the output you are currently reading.

 4. The Docker daemon streamed that output to the Docker client, which sent it

    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:

 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:

 https://hub.docker.com/

For more examples and ideas, visit:

 https://docs.docker.com/get-started/

Installing docker-compose on the Natrix-Gateway

The installation of docker-Compose is usually done using pip3. In order to prepare this, we need to install the libraries libffi-dev and libssl-dev as well as the packages python3-dev, python3 and python-pip3.

apt-get install libffi-dev libssl-dev
apt install python3-dev
apt-get install -y python3 python3-pip

With that preparation done, we finally can install docker-compose:

pip3 install docker-compose

Install Portainer

We use the lightweight container management system Portainer to manage all our images, volumes, networks and finally containers on our Natrix Gateway:.

docker pull portainer/portainer-ce:linux-arm
linux-arm: Pulling from portainer/portainer-ce
7721cab3d696: Pull complete
0645e7e2a110: Pull complete
bfbb644d4a9f: Pull complete
Digest: sha256:468d2e52c3c1d1d7d25850c7aa1ae7c40b6dcea30822c6e4d5ff38cbbe9d4d7c
Status: Downloaded newer image for portainer/portainer-ce:linux-arm
docker.io/portainer/portainer-ce:linux-arm


root@NG-dca63298e6a5:~# docker run --restart always -d -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:linux-arm -name MyPortainer
27ba476d7fc22821d24483694eb566403ce20af96500220f2ca63bcee43acd93

To finally use Portainer from the outside world (on port 9000) we need to adapt our firewall settings on the Natrix Gateway

iptables -A INPUT -i wlan0 -p tcp -m tcp --dport 9000 -j ACCEPT
iptables -A INPUT -i eth0 -p tcp -m tcp --dport 9000 -j ACCEPT
iptables-save > /etc/sysconfig/iptables

Now we can launch Portainer the first time: ‘http://<Your Natrix Gateway IP address>:9000 and provide a password for the default user ‘admin’

After login, we see an overview about all installed containers.

Install pivccu-modules-dkms kernel modules

piVCCU is a project to install the original Homematic CCU3 firmware inside our docker environment. This is for example used for the both types of radio control modules RPI-RF-MOD and HM-MOD-RPI-PCB.

wget -q -O - https://www.pivccu.de/piVCCU/public.key | sudo apt-key add -
sudo sh -c 'echo "deb https://www.pivccu.de/piVCCU stable main" >/etc/apt/sources.list.d/pivccu.list'
sudo apt update
sudo apt install build-essential bison flex libssl-dev
sudo apt install pivccu-modules-dkms

It can be the case that the correct kernel header files are not installed on the system. A message such as ‘Your kernel headers for kernel 5.10.17-v8+ cannot be found’ may appear then.

Update kernel on the Natrix-Gateway

The tool rpi-source is helpful if you use rpi-update kernels or want to build an in-kernel module. rpi-source installs the kernel source used to build rpi-update kernels and the kernel on the Natrix-Gateway OS image. This makes it possible to build loadable kernel modules.

First we have to install some dependencies to run rpi-source:

apt install git bc bison flex libssl-dev

Then we can install the tool itself

wget https://raw.githubusercontent.com/RPi-Distro/rpi-source/master/rpi-source -O /usr/local/bin/rpi-source && sudo chmod +x /usr/local/bin/rpi-source && /usr/local/bin/rpi-source -q --tag-update

Now we can run ‘rpi-source’ to update the kernel.

rpi-source

Below there is an alternative solution instead of using ‘rpi-source’:

sudo apt update

sudo apt install --reinstall raspberrypi-bootloader raspberrypi-kernel

Either way we need to reboot the system and can see afterwards that the piVCCU service has started without any issues.

service pivccu-dkms start
service pivccu-dkms status
● pivccu-dkms.service - piVCCU DKMS Modules
   Loaded: loaded (/lib/systemd/system/pivccu-dkms.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Thu 2021-11-04 11:53:52 CET; 10min ago
  Process: 371 ExecStart=/var/lib/piVCCU/dkms/ensure_modules.sh (code=exited, status=0/SUCCESS)
 Main PID: 371 (code=exited, status=0/SUCCESS)

Nov 04 11:52:25 NG-dca63298e6a5 ensure_modules.sh[371]: rtc-rx8130.ko:
Nov 04 11:52:25 NG-dca63298e6a5 ensure_modules.sh[371]: Running module version sanity check.
Nov 04 11:52:26 NG-dca63298e6a5 ensure_modules.sh[371]:  - Original module
Nov 04 11:52:26 NG-dca63298e6a5 ensure_modules.sh[371]:    - No original module exists within this kernel
Nov 04 11:52:26 NG-dca63298e6a5 ensure_modules.sh[371]:  - Installation
Nov 04 11:52:26 NG-dca63298e6a5 ensure_modules.sh[371]:    - Installing to /lib/modules/5.10.17-v7l+/kernel/drivers/pivccu/
Nov 04 11:52:29 NG-dca63298e6a5 ensure_modules.sh[371]: depmod....
Nov 04 11:52:29 NG-dca63298e6a5 ensure_modules.sh[371]: DKMS: install completed.
Nov 04 11:53:52 NG-dca63298e6a5 systemd[1]: pivccu-dkms.service: Succeeded.
Nov 04 11:53:52 NG-dca63298e6a5 systemd[1]: Started piVCCU DKMS Modules.

Installing the HM-MOD-RPI-PCB support modules

Because we want to use the radio module HM-MOD-RPI-PCB which is connected to the Natrix-Gateway’s GPIO, we need to install the necessary support modules for it.

apt install pivccu-modules-raspberrypi

It is necessary to disable both Bluetooth support and the serial console to ensure a proper communication of the radio module via the GPIO pins with those modules.

Disabling serial console on the Natrix-Gateway

To disable the serial console on the Natrix-Gateway, the entries ‘console=serial0,xxx’ and ‘console=ttyAMA0,xxx’ must be deleted in the file ‘/boot/cmdline.txt’.

nano /boot/cmdline.txt
console=tty1 root=PARTUUID=d3cc5c02-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles

Disabling Bluetooth on the Natrix-Gateway

To disable the Bluetooth, we require the following parameter in the ‘/boot/config.txt’ file: ‘dtoverlay=pi3-disable-bt’. We also need to add the parameter ‘dtoverlay=pivccu-raspberrypi’ and set ‘enable_uart=1’

nano /boot/config.txt
.
.
.
[all]
#dtoverlay=vc4-fkms-v3d
enable_uart=1
dtoverlay=pi3-disable-bt
dtoverlay=pivccu-raspberrypi
gpu_mem=16

Furthermore, the hciuart service needs to be disabled.

Caution: This is in contradiction to some Maker projects which leads to the point that the Natrix-Gateway Smarthome image can’t be used with any Maker project. Please refer to the respective images which are being provided for each Maker project.

systemctl disable hciuart.service

Installing the ‘eq3_char_loop’ module

Next step is to install the ‘eq3_char_loop’ kernel module to the automatic module load section. ‘eq-3’ is the loopback driver for HomeMatic.

sh -c 'echo eq3_char_loop >/etc/modules-load.d/eq3_char_loop.conf'

With modprobe, modules can be loaded and unloaded while the system is running. modprobe automatically resolves dependencies, i.e. if the module to be loaded requires other modules, these are automatically loaded or removed in the correct order. We use modprobe to load ‘eq3_char_loop’ module

modprobe eq3_char_loop

This should come back without any error.

We are now ready to finally deploy RaspberryMatic in Docker. A good practice to do this is via docker-compose and using the below docker-compose file which we place into ‘/opt/RaspberryMatic’

version: '3.1'
services:
  raspberrymatic:
    image: ghcr.io/jens-maus/raspberrymatic:latest
    container_name: ccu
    privileged: true
    volumes:
      - ccu_data:/usr/local    
    ports:
      - "6080:80"
      - "2001:2001"
      - "2010:2010"
      - "9292:9292"
      - "6181:8181"
    restart: always
    stop_grace_period: 30s
volumes:
    ccu_data:

Please take care on that the HomeMatic UI, which runs internally on port 80, can be reached on port 6080 from outside of the docker environment.

We are ready to compose the ‘docker-compose.yaml’ file:

docker-compose up -d
Pulling raspberrymatic (ghcr.io/jens-maus/raspberrymatic:latest)...
latest: Pulling from jens-maus/raspberrymatic
0a3dafed1498: Already exists
Digest: sha256:8204500c684e7cce76562748b2ccda9eba5e45402e691555a4e81fb9b928a6f3
Status: Downloaded newer image for ghcr.io/jens-maus/raspberrymatic:latest
Starting ccu ... done

What we finally need to do in order to reach the HomeMatic UI on port 6080 for both interfaces eth0 and wlan0 is to open the respective ports for both interfaces wlan0 and eth0:

iptables -A INPUT -i wlan0 -p tcp -m tcp --dport 6080 -j ACCEPT

iptables -A INPUT -i eth0 -p tcp -m tcp --dport 6080 -j ACCEPT

iptables-save > /etc/sysconfig/iptables

Checking for an accurate start of RaspberryMatic

Once we created and started the RaspberryMatic container, we can check in Portainer if the ‘ccu’ container is successfully running.

If you go the logs, we shall find the following log entries which indicate a proper start of the container, including detecting the radio module:

Starting watchdog...
Identifying onboard hardware: oci, OK
Initializing RTC Clock: no hardware found
Running sysctl: OK
Checking for Factory Reset: not required
Checking for Backup Restore: not required
Initializing System: OK
Starting logging: OK
Populating /dev using udev: done
Init onboard LEDs: init, OK
Starting irqbalance: OK
Starting network: eth0: link up, fixed, firewall, inet up, 172.18.0.2, OK
Identifying Homematic RF-Hardware: HmRF: none, HmIP: none, OK
Updating Homematic RF-Hardware: no GPIO/USB connected RF-hardware found
Starting hs485dLoader: disabled
Starting xinetd: OK
Starting eq3configd: OK
Starting lighttpd: OK
Starting ser2net: disabled
Starting ssdpd: OK
Starting sshd: OK
Starting NUT services: disabled
Initializing Third-Party Addons: OK
Starting LGWFirmwareUpdate: ...OK
Setting LAN Gateway keys: OK
Starting hs485d: disabled
Starting multimacd: not required
Starting rfd: no BidCos-RF hardware found
Starting HMIPServer: .....OK
Starting ReGaHss: .OK
Starting CloudMatic: OK

Starting NeoServer: disabled
Starting Third-Party Addons: OK
Starting crond: OK
Setup onboard LEDs: booted, OK
Setup onboard LEDs: shutdown, OK
Stopping crond: OK
Stopping Third-Party Addons: OK
Stopping NeoServer: OK
Stopping ReGaHss: ...............Starting watchdog...
Identifying onboard hardware: oci, OK
Initializing RTC Clock: no hardware found
Running sysctl: OK
Checking for Factory Reset: not required
Checking for Backup Restore: not required
Initializing System: OK
Starting logging: OK
Populating /dev using udev: done
Init onboard LEDs: init, OK
Starting irqbalance: OK
Starting network: eth0: link up, fixed, firewall, inet down, 172.18.0.2, OK
Identifying Homematic RF-Hardware: ....HmRF: HM-MOD-RPI-PCB/GPIO, HmIP: HM-MOD-RPI-PCB/GPIO, OK
Updating Homematic RF-Hardware: HM-MOD-RPI-PCB: 2.8.6, OK
Starting hs485dLoader: disabled
Starting xinetd: OK
Starting eq3configd: OK
Starting lighttpd: OK
Starting ser2net: disabled
Starting ssdpd: OK
Starting sshd: OK
Starting NUT services: disabled
Initializing Third-Party Addons: OK
Starting LGWFirmwareUpdate: ...OK
Setting LAN Gateway keys: OK
Starting hs485d: disabled
Starting multimacd: .OK
Starting rfd: .OK
Starting HMIPServer: ..........OK
Starting ReGaHss: .OK
Starting CloudMatic: OK
Starting NeoServer: disabled
Starting Third-Party Addons: OK
Starting crond: OK
Setup onboard LEDs: booted, OK

iptables -A INPUT -i wlan0 -p tcp -m tcp --dport 6080 -j ACCEPT
iptables -A INPUT -i eth0 -p tcp -m tcp --dport 6080 -j ACCEPT
iptables-save > /etc/sysconfig/iptable

First steps on RaspMatic

We can launch the RaspMatic UI with ‘http://<Your Natrix Gateway IP address>:9000 and need to provide a password first. After this, we select the ‘custom’ settings wizard, grant full access to all APIs and open the firewall (we protect this container with the Natrix Gateway firewall, no need to have this twice).

We finally can launch the Homepage of RaspMatic

Adding the XML-API Addon

HomeMatic© provides an API which opens the CCU to be accessible from the outside world. We require this API (which is based on XML-RPC) for connecting our openHAB instance with the CCU software.

From here it can be downloaded. To install the Addon, go to [Settings]->[Control Panel]->[Additional Software] and install the package from there.

We are now ready to bind our openHAB instance with RaspberryMatic to embrace the full flexibility and huge range of smart devices we can work with.

Connecting openHAB3 with RaspMatic

What we finally want to achieve is to manage all RaspMatic controlled things (actors and sensors) via openHAB. Therefore, it is mandatory to link both applications which each other. This is done via openHAB leveraging the ‘Homematic-Binding’. In order to do this, we select [Things] and click on the [+] sign on the bottom right-hand side. A new window appears from where we can select the binding we want to add to our openHAB framework.

We select the ‘Homematic-Binding’ entry in the list, click on it and start the installation of the binding. After the installation it appears in the list of all installed bindings, from where we can start the configuration phase.

The configuration is pretty much straight forward as we only need to apply the IP address of the RaspberryMatic instance (running on docker). All other parameters can be left default. The RaspberryMatic container exposes the XML-API via port 2010 and binds to all existing interfaces, so to localhost as well. We use the localhost address to bind openHAB with RaspberryMatic via the XML-API. This because it makes us indepenednet from IP address changes on the Natrix Gateway itself.

The Natrix-Gateway

Installing the OS

Eclipse Kura™

Preparing for openHab