Use swap on NVMe to run more dev KVM guests, for when you run out of RAM

Leave a Reply

I often spin up a bunch of VMs for different reasons when doing dev work and unfortunately, as awesome as my little mini-itx Ryzen 9 dev box is, it only has 32GB RAM. Kernel Samepage Merging (KSM) definitely helps, however when I have half a dozens or so VMs running and chewing up RAM, the Kernel’s Out Of Memory (OOM) killer will start executing them, like this.

[171242.719512] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/machine.slice/machine-qemu\x2d435\x2dtest\x2dvm\x2dcentos\x2d7\x2d00.scope,task=qemu-system-x86,pid=2785515,uid=107
[171242.719536] Out of memory: Killed process 2785515 (qemu-system-x86) total-vm:22450012kB, anon-rss:5177368kB, file-rss:0kB, shmem-rss:0kB
[171242.887700] oom_reaper: reaped process 2785515 (qemu-system-x86), now anon-rss:0kB, file-rss:68kB, shmem-rss:0kB

If I had more slots available (which I don’t) I could add more RAM, but that’s actually pretty expensive, plus I really like the little form factor. So, given it’s just dev work, a relatively cheap alternative is to buy an NVMe drive and add a swap file to it (or dedicate the whole drive). This is what I’ve done on my little dev box (actually I bought it with an NVMe drive so adding the swapfile came for free).

Continue reading →

Using pipefail with shell module in Ansible

Leave a Reply

If you’re using the shell module with Ansible and piping the output to another command, it might be a good idea to set pipefail. This way, if the first command fails, the whole task will fail.

For example, let’s say we’re running this silly task to look for /tmp directory and then trim the string “tmp” from the result.

ansible all -i "localhost," -m shell -a \
'ls -ld /tmp | tr -d tmp'

This will return something like this, with a successful return code.

localhost | CHANGED | rc=0 >>
drwxrwxrw. 26 roo roo 640 Se 28 19:08 /

Continue reading →

Running a non-root container on Fedora with podman and systemd (Home Assistant example)

2 Replies

Similar to my post about running Home Assistant on Fedora in Docker, this is about using podman instead and integrating the container as a service with systemd. One of the major advantages to me is the removal of Docker daemon and integration with the rest of the system including management of dependencies like regular services.

This assumes you’ve just installed Fedora server and have a local user with sudo privileges. Let’s also install some SELinux tools.

sudo dnf install -y /usr/sbin/semanage

Create non-root user

Let’s create a specific user to run the Home Assistant service.

Continue reading →

Monitoring OpenWrt with collectd, InfluxDB and Grafana

4 Replies

In my previous blog post I showed how to set up InfluxDB and Grafana (and Prometheus). This is how I configured my OpenWrt devices to provide monitoring and graphing of my network.

OpenWrt includes support for collectd (and even graphing inside Luci web interface) so we can leverage this and send our data across the network to the monitoring host.

OpenWrt stats in Grafana
Continue reading →

Setting up a monitoring host with Prometheus, InfluxDB and Grafana

Leave a Reply

Prometheus and InfluxDB are powerful time series database monitoring solutions, both of which are natively supported with graphing tool, Grafana.

Setting up these simple but powerful open source tools gives you a great base for monitoring and visualising your systems. We can use agents like node-exporter to publish metrics on remote hosts which Prometheus will scrape, and other tools like collectd which can send metrics to InfluxDB’s collectd listener (as per my post about OpenWRT).

Prometheus’ node exporter metrics in Grafana

I’m using CentOS 7 on a virtual machine, but this should be similar to other systems.

Continue reading →

Install newer git from software collections and enable globally

Leave a Reply

Work on Linux almost always means git for me, but the version provided by CentOS and RHEL is too old. Software collections is a convenient way to get a newer version and enable it for everyone by default.

First, enable software collections (different for RHEL and CentOS).

# CentOS
sudo yum install centos-release-scl
# RHEL
sudo yum-config-manager --enable rhel-server-rhscl-7-rpms

Install the newer version of git you want (e.g. git 2.18).

sudo yum install rh-git218

Enable it for everyone for any new sessions.

cat << EOF | sudo tee /etc/profile.d/git-scl.sh
source scl_source enable rh-git218
EOF

Test with a new session.

git --version
$SHELL
git --version

Securing Linux with Ansible

Leave a Reply

The Ansible Hardening role from the OpenStack project is a great way to secure Linux boxes in a reliable, repeatable and customisable manner.

It was created by former colleague of mine Major Hayden and while it was spun out of OpenStack, it can be applied generally to a number of the major Linux distros (including Fedora, RHEL, CentOS, Debian, SUSE).

The role is based on the Secure Technical Implementation Guide (STIG) out of the Unites States for RHEL, which provides recommendations on how best to secure a host and the services it runs (category one for highly sensitive systems, two for medium and three for low). This is similar to the Information Security Manual (ISM) we have in Australia, although the STIG is more explicit.

Continue reading →

Pi-hole with DNS over TLS on Fedora

Leave a Reply

Quick and dirty guide to using Pi-hole with Stubby to provide both advertisement blocking and DNS over TLS. I’m using Fedora 30 ARM server edition on a Raspberry Pi 3.

Download Fedora Server ARM edition and write it to an SD card for the Raspberry Pi 3.

sudo fedora-arm-image-installer --resizefs --image=Fedora-Server-armhfp-30-1.2-sda.raw.xz --target=rpi3 --media=/dev/mmcblk0

Make sure your Raspberry Pi can already resolve DNS queries from some other source, such as your router or internet provider.

Continue reading →

Running Home Assistant on Fedora with Docker

Leave a Reply

Home Assistant is a really great, open source home automation platform written in Python which supports hundreds of components. They have a containerised version called Hass.io which can run on a bunch of hardware and has a built-in marketplace to make the running of addons (like Let’s Encrypt) easy.

I’ve been running Home Assistant on a Raspberry Pi for a couple of years, but I want something that’s more poweful and where I have more control. Here’s how you can use the official Home Assistant containers on Fedora (note that this does not include their Hass.io marketplace).

First, install Fedora Server edition, which comes with the handy web UI for managing the system called Cockpit.

Once you’re up and running, install Docker and the Cockpit plugin.

sudo dnf install -y docker cockpit-docker

Now we can start and enable the Docker daemon and restart cockpit to load the Docker plugin.

Continue reading →

Manage and tweak Fedora with Ansible (and apply Korora settings by default)

7 Replies

Korora Project is a Linux distro I created over 13 years ago, which (since 2010) takes Fedora and applies dozens of tweaks in an effort to make it more usable “out of the box” for every day users.

Even with one or two others helping, it has been a lot of work so I’ve taken a break from the project for the last year to focus on other things. There has been no release of Korora since and so lately I’ve been running stock Fedora 29 Workstation (GNOME) on my laptop.

I enjoy the Korora defaults though and given that my family also runs Korora, I wanted a way to be able to move them to stock Fedora while keeping the same packages as well as the look and feel.

So, I created a Korora Ansible Role (it’s also on Ansible Galaxy) to apply the same Korora tweaks for stock Fedora Workstation (GNOME) plus an example playbook which uses it.

I tried to make it flexible by using variables so that users can change default package lists and settings for each machine, as required.

Running it on your local machine is pretty trivial, there’s a shell script with a sample inventory for localhost.

Continue reading →