Network Management with OpenNMS
OpenNMS allows you to visualize and monitor all components in local and remote networks. It combines comprehensive error, performance, traffic monitoring and alarm generation. OpenNMS is open source and available free of charge under the General Public License (GPL). Open-NMS and its administration interface are based on Java.
Administrators use a web interface and the configuration files for administration. Among other things, the solution supports Simple Network Management Protocol (SNMP) for data retrieval and can scan entire networks for new devices for monitoring. The advantage of OpenNMS compared to Nagios is the better and more detailed integration of SNMP, which, however, must be configured beforehand. OpenNMS can retrieve significantly more SNMP traps and integrate them into monitoring quit easily. Services and SNMP queries can also be integrated a little easier in OpenNMS. OpenNMS is also able to monitor sensors on servers – such as the temperature of the processor.
OpenNMS monitors the connected servers with various functions. A discovery process is used to determine which servers and network devices OpenNMS should monitor. OpenNMS then tries to recognize these services on the servers. With the help of the discovery function, OpenNMS automatically recognizes new servers in the network and can also integrate them into the monitoring. In addition to this activity, the monitoring of system performance is an important area that can be defined for individual servers or server groups. The tool can also monitor and evaluate events and log files from servers. OpenNMS prepares the results of these various tasks in the web interface in graphical and tabular form. In addition, OpenNMS sends configured messages under certain circumstances, for example by e-mail.
In the context of the Natrix Project we only work with Natrix Gateway which are most likely not in the same network as the OpenNMS Server. Therefore, we work with an application called OpenNMS Minions which we have to deploy on the Natrix Gateway.
The purpose of the minions is to make it easier to expand the surveillance area. If we previously wanted to monitor a network at a different location, which was protected from direct access by firewalls, for example, you had to make do with NAT or the construction of tunnels, which could lead to a larger set of firewall rules. When using the Minions, only a connection between Minion and OpenNMS installation has to be set up on a manageable number of ports (usually the port of the OpenNMS website and port 61616). The minions take over the polling and the data collection on site and transmit the data to the OpenNMS server, which also takes over the central administration of the minions.
Deploying OpenNMS Server
There a couple of installation steps needed to get openNMS up and running on our Natrix Management platform.
Deploying postgresql for persistence
OpenNMS manages persistence via a PostgreSQL. So the first step is to install a PostgreSQL server and client instance on the Natrix Management platform.
apt -y install postgresqlNow we have to create an opennms database user and password.
sudo -u postgres createuser -P opennmsWith that user we are now able to create an empty database where openNMS can store all its data into.
sudo -u postgres createdb -O opennms opennmsThe final step to provide a running postgreSQL instance to openNMS is to set a password for the user postgres.
sudo -u postgres psql -c "ALTER USER postgres WITH PASSWORD 'CHOOSE-YOUR-PASSWORD;"Upgrading the Kernel of the Natrix Management Platform
It has been discovered that it is good practice to update the Kernel of the Natrix Management Platform to the latest version for running the openNMS smoothly.
All currently available updates must be installed, this ensures that later no or at least fewer package conflicts arise. Old packages should also be deleted:
apt-get update && apt-get upgrade -y
apt-get autoremove
Now we clean everything up and start the kernel upgrade process:
apt clean && apt update
apt upgrade -y
apt dist-upgrade -y
apt autoremove
After the Kernel update and a reboot we can check the current Kernel on our Natrix Management Platform.
cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL=https://bugs.debian.org/
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
cat: 'PRETTY_NAME=Debian GNU/Linux 10 (buster)': No such file or directory
cat: 'NAME=Debian GNU/Linux': No such file or directory
cat: 'VERSION_ID=10': No such file or directory
cat: 'VERSION=10 (buster)': No such file or directory
cat: 'VERSION_CODENAME=buster': No such file or directory
cat: 'ID=debian': No such file or directory
cat: 'HOME_URL=https://www.debian.org/': No such file or directory
cat: 'SUPPORT_URL=https://www.debian.org/support': No such file or directory
cat: 'BUG_REPORT_URL=https://bugs.debian.org/': No such file or directoryStarting with the openNMS Server installation:
With GnuPG installed, which allows us to encrypt and sign our data and communications and which features a versatile key management system, along with access modules for all kinds of public key directories, we can add OpenNMS repository GPG key. (GNUPG(1), 2021)
apt -y install gnupg ca-certificates
apt-key adv --fetch-keys https://debian.opennms.org/OPENNMS-GPG-KEYNow let us add the openNMS repository.
apt -y install software-properties-common
add-apt-repository -s 'deb https://debian.opennms.org stable main'
apt updateThis enables us now to install openNMS (Horizon) and all its dependencies.
apt -y install opennmsIt is good practice to disable the OpenNMS Horizon repository after installation to prevent unwanted upgrades when upgrading other packages on the server. Disable automatic upgrades makes sense because after an upgrade many manual configurations needs to be done again.
apt-mark hold libopennms-java
> libopennmsdeps-java \
> opennms-common \
> opennms-db
libopennms-java set on hold.
libopennmsdeps-java set on hold.
opennms-common set on hold.
opennms-db set on hold.
Customizing OpenNMS Server
An important thing to acknowledge is that there is huge customization work to be done in the openNMS instance to get it working properly with the Minion instance running on our Natrix Gateway.
Checking for the correct JVM for running the openNMS core instance
openNMS requires to know how to access the postgreSQL database. This is done in the ‘/etc/ opennms/opennms-datasources.xml’ file
sudo /usr/share/opennms/bin/runjava -s
runjava: Looking for an appropriate JVM...
runjava: Checking for an appropriate JVM in JAVA_HOME...
runjava: Skipping... JAVA_HOME not set.
runjava: Checking JVM in the PATH: "/etc/alternatives/java"...
runjava: Did not find an appropriate JVM in the PATH: "/etc/alternatives/java"
runjava: Searching for a good JVM...
runjava: Found a good JVM in "/usr/lib/jvm/java-11-openjdk-amd64/bin/java".
runjava: Value of "/usr/lib/jvm/java-11-openjdk-amd64/bin/java" stored in configuration file.
Setting up persistence on the openNMS core instance
openNMS requires to know how to access the postgreSQL database. This is done in the ‘/etc/ opennms/opennms-datasources.xml’ file
nano /etc/opennms/opennms-datasources.xml
<jdbc-data-source name="opennms"
database-name="opennms"
class-name="org.postgresql.Driver"
url="jdbc:postgresql://localhost:5432/opennms"
user-name="< OPENNMS-USERNAME >"
password="< OPENNMS-PASSWORD >" />
<jdbc-data-source name="opennms-admin"
database-name="template1"
class-name="org.postgresql.Driver"
url="jdbc:postgresql://localhost:5432/template1"
user-name="postgres"
password="< POSTGRES-PASSWORD >" />
With the JDBC connection properties configured and the postgreSQL credentials provided, we can start initializing the database and detect system libraries persisted in /opt/opennms/etc/libraries.properties
sudo /usr/share/opennms/bin/install -disSetting up the Broker connection to the openNMS core instance
Distributing openNMS components such as Minions require a messaging infrastructure. openNMS supports ActiveMQ or Apache Kafka infrastructures, and it comes with an embedded ActiveMQ system which by default an external network cannot reach. To get the Minion instance on our Natrix Gateway communicating with this embedded ActiveMQ Broker, a single line in the ‘/etc/opennms/opennms-activemq.xml’ file needs to be edited. It is about uncommenting the transportConnector parameter:
nano /etc/opennms/opennms-activemq.xml
.
.
.
<!--
The transport connectors expose ActiveMQ over a given protocol to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
-->
<transportConnectors>
<transportConnector name="vm" uri="vm://localhost"/>
<!-- Uncomment this line to allow external TCP connections -->
<!--
WARNING: Access to port 61616 should be firewalled to prevent unauthorized injection
of data into OpenNMS when this port is open.
-->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?useJmx=false&maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
<!-- Uncomment this line to allow localhost TCP connections (for testing purposes) -->
<!--transportConnector name="openwire" uri="tcp://127.0.0.1:61616?useJmx=false&maximumConnections=1000&wireformat.maxFrameSize=104857600"/-->
</transportConnectors>
Finally, we can enable the openNMS core instance on system boot and to start the instance.
systemctl enable opennms
systemctl opennms start
First start of the openNMS core instance
When openNMS has started successfully, we can login the first time and provide a password for the default user ‘admin’
