Archive for the 'FOSS' Category

Live migrating Btrfs from RAID 5/6 to RAID 10

Recently it was discovered that the RAID 5/6 implementation in Btrfs is broken, due to the fact that can miscalculate parity (which is rather important in RAID 5 and RAID 6).

So what to do with an existing setup that’s running native Btfs RAID 5/6?

Well, fortunately, this issue doesn’t affect non-parity based RAID levels such as 1 and 0 (and combinations thereof) and it also doesn’t affect a Btrfs filesystem that’s sitting on top of a standard Linux Software RAID (md) device.

So if down-time isn’t a problem, we could re-create the RAID 5/6 array using md and put Btrfs back on top and restore our data… or, thanks to Btrfs itself, we can live migrate it to RAID 10!

Continue reading ‘Live migrating Btrfs from RAID 5/6 to RAID 10’

Command line password management with pass

Why use a password manager in the first place? Well, they make it easy to have strong, unique passwords for each of your accounts on every system you use (and that’s a good thing).

For years I’ve stored my passwords in Firefox, because it’s convenient, and I never bothered with all those other fancy password managers. The problem is, that it locked me into Firefox and I found myself still needing to remember passwords for servers and things.

So a few months ago I decided to give command line tool Pass a try. It’s essentially a shell script wrapper for GnuPG and stores your passwords (with any notes) in individually encrypted files.

I love it.

Continue reading ‘Command line password management with pass’

Setting up OpenStack Ansible All-in-one behind a proxy

Setting up OpenStack Ansible (OSA) All-in-one (AIO) behind a proxy requires a couple of settings, but it should work fine (we’ll also configure the wider system). There are two types of git repos that we should configure for (unless you’re an OpenStack developer), those that use http (or https) and those that use the git protocol.

Firstly, this assumes an Ubuntu 14.04 server install (with at least 60GB of free space on / partition).

Continue reading ‘Setting up OpenStack Ansible All-in-one behind a proxy’

Booting Fedora 24 cloud image with KVM

Fedora 24 is on the way, here’s how you can play with the cloud image on your local machine.

Download the image:
wget https://alt.fedoraproject.org/pub/alt/stage/24_RC-1.2/CloudImages/x86_64/images/Fedora-Cloud-Base-24-1.2.x86_64.qcow2

Make a new local backing image (so that we don’t write to our downloaded image) called my-disk.qcow2:
qemu-img create -f qcow2 -b Fedora-Cloud-Base-24-1.2.x86_64.qcow2 my-disk.qcow2

The cloud image uses cloud-init to configure itself on boot which sets things like hostname, usernames, passwords and ssh keys, etc. You can also run specific commands at two stages of the boot process (see bootcmd and runcmd below) and output messages (see final_message below) which is useful for scripted testing.

Continue reading ‘Booting Fedora 24 cloud image with KVM’

How to find out which process is listening on a port

Say that you notice UDP port 323 is open (perhaps via netstat -lun) and you’ve no idea what that is!

With lsof it’s easy to find out which process is guilty:


[15:27 chris ~]$ sudo lsof -i :323
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
chronyd 1044 chrony 1u IPv4 19197 0t0 UDP localhost:323
chronyd 1044 chrony 2u IPv6 19198 0t0 UDP localhost:323

In this case, it’s chrony, the modern time keeping daemon.

As Jonh pointed out in the comments, you can also use netstat with the -p flag.

For example, show all processes listening (-l) on both TCP (-t) and UDP (-u) by port number (-n) showing the process (-p), while I grep for port 323 to show what’s running:

[19:08 chris ~]$ sudo netstat -lutnp |grep 323
udp 0 0 127.0.0.1:323 0.0.0.0:* 1030/chronyd
udp6 0 0 ::1:323 :::* 1030/chronyd

Signal Return Orientated Programming attacks

When a process is interrupted, the kernel suspends it and stores its state in a sigframe which is placed on the stack. The kernel then calls the appropriate signal handler code and after a sigreturn system call, reads the sigframe off the stack, restores state and resumes the process. However, by crafting a fake sigframe, we can trick the kernel into executing something else.

My friend Rashmica, an intern at OzLabs, has written an interesting blog post about this for some work she’s doing with the POWER architecture in Linux.

TRIM on LVM on LUKS on SSD, revisited

A few years ago I wrote about enabling trim on an SSD that was running with LVM on top of LUKS. Since then things have changed slightly, a few times.

With Fedora 24 you no longer need to edit the /etc/crypttab file and rebuild your initramfs. Now systemd supports a kernel boot argument rd.luks.options=discard which is the only thing you should need to do to enable trim on your LUKS device.

Edit /etc/default/grub and add the rd.luks.options=discard argument to the end of GRUB_CMDLINE_LINUX, e.g.:
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.luks.uuid=luks-de023401-ccec-4455-832bf-e5ac477743dc rd.luks.uuid=luks-a6d344739a-ad221-4345-6608-e45f16a8645e rhgb quiet rd.luks.options=discard"
GRUB_DISABLE_RECOVERY="true"

Next, rebuild your grub config file:
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

If you’re using LVM, the setting is the same as the previous post. Edit the /etc/lvm/lvm.conf file and enabled the issue_discards option:
issue_discards = 1

If using LVM you will need to rebuild your initramfs so that the updated lvm.conf is in there.
sudo dracut -f

Reboot and try fstrim:
sudo fstrim -v /

Now also thanks to systemd, you can just enable the fstrim timer (cron) to do this automatically:
sudo systemctl enable fstrim.timer

Running scripts before and after suspend with systemd

I’ve had this question a few times, so it’s probably a good candidate for my blog.

If you want to do something before you suspend, like unload a module or run some script, it’s quite easy with systemd. Similarly, you can easily do something when the system resumes (like reload the module).

The details are in the systemd-suspend man page:
man systemd-suspend.service

Simply put an executable script of any name under /usr/lib/systemd/system-sleep/ that checks whether the first argument is pre (for before the system suspends) or post (after the system wakes from suspend).

If it is pre, then do the thing you want to before suspend, if it’s post then do the thing you want to do after resume. Simple!

Here’s a useless example:
#!/bin/sh
if [ "${1}" == "pre" ]; then
  # Do the thing you want before suspend here, e.g.:
  echo "we are suspending at $(date)..." > /tmp/systemd_suspend_test
elif [ "${1}" == "post" ]; then
  # Do the thing you want after resume here, e.g.:
  echo "...and we are back from $(date)" >> /tmp/systemd_suspend_test
fi

Automatic power saving on a Linux laptop with PowerTOP and systemd

If you have a laptop and want to get more battery life, you may already know about a handy tool from Intel called PowerTOP.

PowerTOP not only monitors your system for interrupts but has a tunable section where you can enable various power saving tweaks. Toggling one such tweak in the PowerTOP interface will show you the specific Linux system command it ran in order to enable or disable it.

PowerTOP Tweaks

Furthermore, it takes an argument ––auto-tune which lets you enable all of the power saving measures it has detected.

Continue reading ‘Automatic power saving on a Linux laptop with PowerTOP and systemd’

Providing git:// (protocol) access to repos using GitLab

I mirror a bunch of open source projects in a local GitLab instance which works well.

By default, GitLab only provides https and ssh access to repositories, which can be a pain for continuous integration (especially if you were to use self-signed certificates).

However, it’s relatively easy to configure your GitLab server to run a git daemon and provide read-only access to anyone on any repos that you choose.

Continue reading ‘Providing git:// (protocol) access to repos using GitLab’