Category Archives: FOSS

Bits and pieces about Linux and free software..

Flashing OpenWRT onto Ubiquiti UniFi AP AC

Recently OpenWRT released version 20.02 and it’s great!

The Wiki instructions for flashing Ubiquiti AP AC devices (including Pro, Lite and LR versions) has a number of options. One uses tftp but requires a specific version of the stock firmware, others use mtd flashing tools (either from OpenWRT or stock) flashing tools, and another via serial.

I chose option 2, using mtd-tools from OpenWRT, as I think it’s the most simple. However, it did require a few extra tweaks for 20.02 due to library errors.

Ubiquiti AP AC Lite
Continue reading

How to create VLAN trunks and access ports for VMs on Linux bridges using NetworkManager (and have them talk)

TL;DR instead of creating a VLAN on a physical interface (like bond0.123), then turning that into a bridge for a VM, create a tagged interface on the main bridge (e.g. br0.123) and then add that into a new bridge for a VM.

Create a new bridge for the VLAN.

nmcli con add ifname br-123 type bridge con-name br-123
nmcli con modify br-123 ipv4.method disabled ipv6.method ignore
nmcli con up br-123

Create the VLAN on existing bridge (assuming br0 already exists) and attach to the new bridge (br-123).

nmcli con add type vlan con-name br0.123 ifname br0.123 dev br0 id 123
nmcli con modify br0.123 master br-123 slave-type bridge
nmcli con up br0.123

This will allow two VMs on the same host to talk to each other over the VLAN, where one is using a tagged interface on br0 (as a trunk) and the other is using br-123 (as an access port or native for VLAN 123).

+------+  +-------+                  +------------------+
| eth0 |--|       |                  |       VM1        |
+------+  |       |   +----------+   |   +----------+   |
          | bond0 |---|   br0    |---|---| eth0.123 |   |
+------+  |       |   | (bridge) |   |   |  (VLAN)  |   |
| eth1 |--|       |   +----------+   |   +----------+   |
+------+  +-------+        |         +------------------+
                           |
                           |                       +------------------+
                           |                       |       VM2        |
                      +---------+   +----------+   |   +----------+   |
                      | br0.123 |---|  br-123  |---|---|   eth0   |   |
                      | (VLAN)  |   | (bridge) |   |   | (native) |   |
                      +---------+   +----------+   |   +----------+   |
                                                   +------------------+

Some background

A common method for connecting VMs to a real network is to attach them to a bridge. In this case, a bridge is created on the KVM host and a physical interface is attached to it. Network interfaces of the VMs are then attached to that bridge and they are directly on the same network as the host (we’ll call this a flat network).

+------+  +-------+                  +----------------+
| eth0 |--|       |                  |       VM       |
+------+  |       |   +----------+   |   +--------+   |
          | bond0 |---|   br0    |---|---|  eth0  |   |
+------+  |       |   | (bridge) |   |   | (flat) |   |
| eth1 |--|       |   +----------+   |   +--------+   |
+------+  +-------+                  +----------------+

That’s great, but now they’re also connected to everything else on the network and maybe you didn’t want that. One solution to this is to use VLANs to put VMs on isolated networks.

The bridge mentioned above could also be used as a trunk, which can carry tagged VLAN traffic that the VM creates (assuming the physical switch is configured to accept those VLANs). In this instance, the VM would create a VLAN on its network interface and tagged traffic will flow out of the bridge onto the physical network, letting that VM talk to other VMs or devices on that same VLAN.

Continue reading

Auto-update Pi-hole with systemd timer

I have two Pi-hole servers at home running Fedora with DNS over TLS, both of which auto update on different days (to avoid having both down if something goes wrong).

First, create a script to get the latest updates if any are available, rebooting the Pi-hole if they were successful.

cat << \EOF | sudo tee /usr/local/sbin/update-pihole.sh
#!/bin/bash
PIHOLE_UPDATE="$(pihole -up --check-only)"
if ! grep -q 'Everything is up to date' <<< "${PIHOLE_UPDATE}" ; then
  pihole -up
  if [[ $? -eq 0 ]] ; then
    echo "$(date "+%h %d %T") update: success" >> /var/log/pihole.log
    reboot
  fi
else
    echo "$(date "+%h %d %T") update: nothing to do" >> /var/log/pihole.log
fi
EOF

Make the script executable.

sudo chmod a+x /usr/local/sbin/update-pihole.sh

Next, let’s create a systemd service for the update, which is required to be able to create a timer.

cat << EOF | sudo tee /etc/systemd/system/update-pihole.service 
[Unit]
Description=Update pihole
After=network-online.target
 
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/update-pihole.sh
EOF

Now we can create the systemd timer. Here I am updating weekly on Mondays at midnight (my other Pi-hole updates weekly on Thursdays), feel free to adjust as you see fit. If you have two, perhaps update on alternate days, like I do.

cat << EOF | sudo tee /etc/systemd/system/update-pihole.timer 
[Unit]
Description=Timer for updating pihole
Wants=network-online.target
 
[Timer]
OnBootSec=
OnCalendar=Mon *-*-* 00:00:00
Persistent=true

[Install]
WantedBy=timers.target
EOF

Now that we have the timer, we can tell systemd about it and enable the timer.

sudo systemctl daemon-reload
sudo systemctl enable --now update-pihole.timer

That it’s it! Your systems will now check for updates and be rebooted if necessary. It might be good to pair this with some monitoring to ensure that your Pi-holes are working as expected and be alerted if otherwise…

Rootless podman containers under system accounts, managed and enabled at boot with systemd

While you can run containers as root on the host, or run rootless containers as your regular user (either as uid 0 or any another), sometimes it’s nice to create specific users to run one or more containers. This provides neat separation and can also improve security posture.

We also want those containers to act as regular system services; managed with systemd to auto-restart and be enabled on boot.

This assumes you’ve just installed Fedora (or RHEL/CentOS 8+) server and have a local user with sudo privileges. First, let’s also install some SELinux tools.

sudo dnf install -y /usr/sbin/semanage

Setting up the system user

Let’s create our system user, placing their home dir under /var/lib. For the purposes of this example I’m using a service account of busybox but this can be anything unique on the box. Note, if you prefer to have a real shell, then swap /bin/false with /bin/bash or other.

export SERVICE="busybox"

sudo useradd -r -m -d "/var/lib/${SERVICE}" -s /bin/false "${SERVICE}"

In order for our user to run containers automatically on boot, we need to enable systemd linger support. This will ensure that a user manager is run for the user at boot and kept around after logouts.

sudo loginctl enable-linger "${SERVICE}"
Continue reading

Podman volumes and SELinux

In my previous post on volumes with podman, I touched on SELinux but I think it’s worthy of its own post to work through some details.

If your host has SELinux enabled, then container processes are confined to the system_u:system_r:container_t:s0 domain. Volumes you pass to podman will need to have appropriate labels, otherwise the container won’t be able access the volume, no-matter what the filesystem permissions are.

When running rootless containers (as your non-root user), files in your homedir will probably have the context of unconfined_u:object_r:data_home_t:s0, however the context that is required for container volumes is system_u:object_r:container_file_t:s0.

Fortunately, container volumes which podman creates at runtime will have the appropriate context set automatically. However, for host-dir volumes podman will not change the context by default, you will need to tell it to.

Let’s spin up a busybox container without setting the SELinux context and note that the container is not able to access the host-dir volume.

Continue reading

Volumes and rootless Podman

Containers generally run from an image and have no access to the host file system. This is fine for stand-alone ephemeral containers, but others require persistent data. While we can pass in environment variables to configure the app inside the container, sometimes a place to read and write more complex files is needed. When you update or replace such a container (perhaps with a new release) you want the new container to have access to the same data as the previous one.

The tricky thing with rootless containers is that you’re not root on the host and, as per my previous post, containers can run as any user id. If the container runs as root (uid 0) then that is fine as it actually maps to your non-root user on the host (e.g. 1000) and management of the data is therefore easy. However, containers running as other users (e.g. 123) will map to a uid on the host based on the subuid offset range (e.g. 100122) which will not match your non-root user and therefore data management is harder.

Providing persistent storage to a container is done by setting up a bind mounts using the --volume (or -v) option with podman. Note that volumes are not restricted to just one per container, simply repeat the -v option for as many volumes as required!

The volume option uses a colon separated format specifying the source on the host and then the destination directory where the volume is mounted to inside the container (e.g. src:/dest).

The source is generally one of two types; either a container volume (like a container image) or a regular host directory.

Continue reading

User IDs and (rootless) containers with Podman

When a Linux container image is created, like any system it can have specific users, or maybe it only has root (uid 0). Containers have a specific entry point which runs the program the image was created for and this might run as any user in the image, it’s up to whoever created it.

You can use podman (a daemonless container engine) to easily see what uid an image will use, by getting the container to run the id command instead of the default entry point.

For example, here we can see that busybox wants to run as root (uid 0).

# podman run --rm --entrypoint '' docker.io/busybox id
uid=0(root) gid=0(root) groups=0(root)

However, grafana wants to run as the grafana user with uid 472 .

# podman run --rm --entrypoint '' docker.io/grafana/grafana id
uid=472(grafana) gid=0(root) groups=0(root)

OK, so inside the containers we are running as different users, but as we’re running as root those same uids are also used on the host system.

Running containers as root

Let’s run a grafana container as root and see that the actual grafana-server process on the host is running under uid 472.

# podman run -d docker.io/grafana/grafana
ee275572c8dcb0922722b7d6c3e5ff0bda8c9af3682018c4fc53675d8e189e59
# ps -o user $(pidof grafana-server)
USER
472
Continue reading

How to create bridges on bonds (with and without VLANs) using NetworkManager

Some production systems you face might make use of bonded network connections that you need to bridge in order to get VMs onto them. That bond may or may not have a native VLAN (in which case you bridge the bond), or it might have VLANs on top (in which case you want to bridge the VLANs), or perhaps you need to do both.

Let’s walk through an example where we have a bond that has a native VLAN, that also has the tagged VLAN 123 on top (and maybe a second VLAN 456), all of which need to be separately bridged. This means we will have the bond (bond0) with a matching bridge (br-bond0), plus a VLAN on the bond (bond0.123) with its matching bridge (br-vlan123). It should look something like this.

+------+   +---------+                           +---------------+
| eth0 |---|         |          +------------+   |  Network one  |
+------+   |         |----------|  br-bond0  |---| (native VLAN) |
           |  bond0  |          +------------+   +---------------+
+------+   |         |                                            
| eth1 |---|         |                                            
+------+   +---------+                           +---------------+
            | |   +---------+   +------------+   |  Network two  |
            | +---| vlan123 |---| br-vlan123 |---| (tagged VLAN) |
            |     +---------+   +------------+   +---------------+
            |                                                     
            |     +---------+   +------------+   +---------------+
            +-----| vlan456 |---| br-vlan456 |---| Network three |
                  +---------+   +------------+   | (tagged VLAN) |
                                                 +---------------+

To make it more complicated, let’s say that the native VLAN on the bond needs a static IP and to operate at an MTU of 1500 while the other uses DHCP and needs MTU of 9000.

OK, so how do we do that?

Continue reading

How to create Linux bridges and Open vSwitch bridges with NetworkManager

My virtual infrastructure Ansible role supports connecting VMs to both Linux and Open vSwitch bridges, but they must already exist on the KVM host.

Here is how to convert an existing Ethernet device into a bridge. Be careful if doing this on a remote machine with only one connection! Make sure you have some other way to log in (e.g. console), or maybe add additional interfaces instead.

Export interfaces and existing connections

First, export the the device you want to convert so we can easily reference it later (e.g. eth1).

export NET_DEV="eth1"

Now list the current NetworkManager connections for your device exported above, so we know what to disable later.

sudo nmcli con |egrep -w "${NET_DEV}"

This might be something like System eth1 or Wired connection 1, let’s export it too for later reference.

export NM_NAME="Wired connection 1"

Create a Linux bridge

Here is an example of creating a persistent Linux bridge with NetworkManager. It will take a device such as eth1 (substitute as appropriate) and convert it into a bridge. Note that we will be specifically giving it the device name of br0 as that’s the standard convention and what things like libvirt will look for.

Continue reading

Accessing USB serial devices in Fedora Silverblue

One of the things I do a lot on my Fedora machines is talk to devices via USB serial. While a device is correctly detected at /dev/ttyUSB0 and owned by the dialout group, adding myself to that group doesn’t work as it can’t be found. This is because under Silverblue, there are two different group files (/usr/lib/group and /etc/group) with different content.

There are some easy ways to solve this, for example we can create the matching dialout group or write a udev rule. Let’s take a look!

On the host with groups

If you try to add yourself to the dialout group it will fail.

sudo gpasswd -a ${USER} dialout
gpasswd: group 'dialout' does not exist in /etc/group

Trying to re-create the group will also fail as it’s already in use.

sudo groupadd dialout -r -g 18
groupadd: GID '18' already exists

So instead, we can simply grab the entry from the OS group file and add it to /etc/group ourselves.

grep ^dialout: /usr/lib/group |sudo tee -a /etc/group

Now we are able to add ourselves to the dialout group!

Continue reading