I’m part for a local Freifunk community and because of that I’m partially responsible for quite a lot of infrastructure.

Recently we moved a new service into the responsibility of our admin group. The migration was mainly done an prepared by me.

The service was Freitrix, which is a Matrix homeserver, if you don’t know Matrix yet, check it out matrix.org.

While I created the new setup from scratch, I learned that Synapse,the software we use as a homeserver supports Prometheus.

We already had a munin setup running which monitored some of our servers. But if you know munin, by default it doesn’t look very fancy and of course it doesn’t have a direct integration with synapse ;).

So a few days after the migration I started with prometheus, it was pretty easy. As Prometheus doesn’t come as a deb package or any other format, just a simple tar.gz the installation is a bit complicated.

Based on a Debian server I put everything in /opt. As user for the service I created a new system user called “prometheus” useradd -m -s /bin/bash prometheus and installed a systemd service to keep it running.

systemctl edit --force --full prometheus.service

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network-online.target

[Service]
User=prometheus
Restart=on-failure
ExecStart=/opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.tsdb.path=/opt/prometheus/data --web.listen-address=127.0.0.1:9090

[Install]
WantedBy=multi-user.target

Make sure to adjust the permissions of the prometheus binary.

After that I was basically done, I configured the self monitoring endpoints and checked if it worked you can simply follow the quick start guide. After that I installed Grafana and configured it to work with prometheus.

It’ relatively easy to configure the endpoints that prometheus should query.

From here I only needed to set up synapse to allow the connection to prometheus and install the node exporter on the servers I wanted to monitor.

One note at this point, I used nginx in front of the node exporter, but of course this doesn’t restrict the access to the endpoints. So if you want to restrict the access to this endpoint, use password authentication from the Apache tools.

I’m quite happy with the end result, for Grafana you can find already pre-configured dashboards and add them to your instance. It gives us a nice overview about what is happening on the servers.