fedora – Just another Linux geek

Accessing USB serial devices in Fedora Silverblue

One of the things I do a lot on my Fedora machines is talk to devices via USB serial. While a device is correctly detected at /dev/ttyUSB0 and owned by the dialout group, adding myself to that group doesn’t work as it can’t be found. This is because under Silverblue, there are two different group files (/usr/lib/group and /etc/group) with different content.

There are some easy ways to solve this, for example we can create the matching dialout group or write a udev rule. Let’s take a look!

On the host with groups

If you try to add yourself to the dialout group it will fail.

sudo gpasswd -a ${USER} dialout
gpasswd: group 'dialout' does not exist in /etc/group

Trying to re-create the group will also fail as it’s already in use.

sudo groupadd dialout -r -g 18
groupadd: GID '18' already exists

So instead, we can simply grab the entry from the OS group file and add it to /etc/group ourselves.

grep ^dialout: /usr/lib/group |sudo tee -a /etc/group

Now we are able to add ourselves to the dialout group!

Making dnf on Fedora Silverblue a little easier with bash aliases

Fedora Silverblue doesn’t come with dnf because it’s an immutable operating system and uses a special tool called rpm-ostree to layer packages on top instead.

Most terminal work is designed to be done in containers with toolbox, but I still do a bunch of work outside of a container. Searching for packages to install with rpm-ostree still requires dnf inside a container, as it does not have that function.

I add these two aliases to my ~/.bashrc file so that using dnf to search or install into the default container is possible from a regular terminal. This just makes Silverblue a little bit more like what I’m used to with regular Fedora.

cat >> ~/.bashrc << EOF
alias sudo="sudo "
alias dnf="bash -c '#skip_sudo'; toolbox -y create 2>/dev/null; toolbox run sudo dnf"
EOF

If the default container doesn’t exist, toolbox creates it. Note that the alias for sudo has a space at the end. This tells bash to also check the next command word for alias expansion, which is what makes sudo work with aliases. Thus, we can make sure that both dnf and sudo dnf will work. The first part of the dnf alias is used to skip the sudo command so the rest is run as the regular user, which makes them both work the same.

We need to source that file or run a new bash session to pick up the aliases.

Fedora Silverblue is an amazing immutable desktop

I recently switched my regular Fedora 31 workstation over to the 31 Silverblue release. I’ve played with Project Atomic before and have been meaning to try it out more seriously for a while, but never had the time. Silverblue provided the catalyst to do that.

What this brings to the table is quite amazing and seriously impressive. The base OS is immutable and everyone’s install is identical. This means quality can be improved as there are less combinations and it’s easier to test. Upgrades to the next major version of Fedora are fast and secure. Instead of updating thousands of RPMs in-place, the new image is downloaded and the system reboots into it. As the underlying images don’t change, it also offers full rollback support.

This is similar to how platforms like Chrome OS and Android work, but thanks to ostree it’s now available for Linux desktops! That is pretty neat.

It doesn’t come with a standard package manager like dnf. Instead, any packages or changes you need to perform on the base OS are done using rpm-ostree command, which actually layers them on top.

And while technically you can install anything using rpm-ostree, ideally this should be avoided as much as possible (some low level apps like shells and libvirt may require it, though). Flatpak apps and containers are the standard way to consume packages. As these are kept separate from the base OS, it also helps improve stability and reliability.

Pi-hole with DNS over TLS on Fedora

Quick and dirty guide to using Pi-hole with Stubby to provide both advertisement blocking and DNS over TLS. I’m using Fedora 30 ARM server edition on a Raspberry Pi 3.

Download Fedora Server ARM edition and write it to an SD card for the Raspberry Pi 3.

sudo fedora-arm-image-installer \
--resizefs \
--image=Fedora-Server-armhfp-32-1.6-sda.raw.xz \
--target=rpi3 \
--media=/dev/mmcblk0

Make sure your Raspberry Pi can already resolve DNS queries from some other source, such as your router or internet provider.

Running Home Assistant on Fedora with Docker

Home Assistant is a really great, open source home automation platform written in Python which supports hundreds of components. They have a containerised version called Hass.io which can run on a bunch of hardware and has a built-in marketplace to make the running of addons (like Let’s Encrypt) easy.

I’ve been running Home Assistant on a Raspberry Pi for a couple of years, but I want something that’s more poweful and where I have more control. Here’s how you can use the official Home Assistant containers on Fedora (note that this does not include their Hass.io marketplace).

First, install Fedora Server edition, which comes with the handy web UI for managing the system called Cockpit.

Once you’re up and running, install Docker and the Cockpit plugin.

sudo dnf install -y docker cockpit-docker

Now we can start and enable the Docker daemon and restart cockpit to load the Docker plugin.

Continue reading Running Home Assistant on Fedora with Docker

Fedora on ODROID-HC1 mini NAS (ARMv7)

Hardkernel is a Korean company that makes various embedded ARM based systems, which it calls ODROID.

One of their products is the ODROID-HC1, a mini NAS designed to take a single 2.5″ SATA drive (HC stands for “Home Cloud”) which comes with 2GB RAM and a Gigabit Ethernet port. There is also a 3.5″ model called the HC2. Both of these are based on the ODROID-XU4, which itself is based on the previous iteration ODROID-XU3. All of these are based on the Samsung Exynos5422 SOC and should work with the following steps.

The Exynos SOC needs proprietary first stage bootloaders which are embedded in the first 1.4MB or so at the beginning of the SD card in order to load U-Boot. As these binary blobs are not re-distributable, Fedora cannot support these devices out of the box, however all the other bits are available including the kernel, device tree and U-Boot. So, we just need to piece it all together and the result is a stock Fedora system!

To do this you’ll need the ODROID device, a power supply (5V/4A for HC1, 12V/2A for HC2), one of their UART adapters, an SD card (UHS-I) and probably a hard drive if you want to use it as a NAS (you may also want a battery for the RTC and a case).

ODROID-HC1 with UART, RTC battery, SD card and 2.5″ drive.

Note that the default Fedora 27 ARM image does not support the Realtek RTL8153 Ethernet adapter out of the box (it does after a kernel upgrade) so if you don’t have a USB Ethernet dongle handy we’ll download the kernel packages on our host, save them to the SD card and install them on first boot. The Fedora 28 image works out of the box, so if you’re installing 28 you can skip that step.

Continue reading Fedora on ODROID-HC1 mini NAS (ARMv7)

Fixing webcam flicker in Linux with udev

I recently got a new Dell XPS 13 (9360) laptop for work and it’s running Fedora pretty much perfectly.

However, when I load up Cheese (or some other webcam program) the video from the webcam flickers. Given that I live in Australia, I had to change the powerline frequency from 60Hz to 50Hz to fix it.

sudo dnf install v4l2-ctl v4l2-ctl --set-ctrl power_line_frequency=1

I wanted this to be permanent each time I turned my machine on, so I created a udev rule to handle that.

cat << EOF | sudo tee /etc/udev/rules.d/50-dell-webcam.rules SUBSYSTEM=="video4linux", \ SUBSYSTEMS=="usb", \ ATTRS{idVendor}=="0c45", \ ATTRS{idProduct}=="670c", \ PROGRAM="/usr/bin/v4l2-ctl --set-ctrl \ power_line_frequency=1 --device /dev/%k", \ SYMLINK+="dell-webcam" EOF

It’s easy to test. Just turn flicker back on, reload the rules and watch the flicker in Cheese automatically disappear 🙂

v4l2-ctl --set-ctrl power_line_frequency=0 sudo udevadm control --reload-rules && sudo udevadm trigger

Of course I also tested with a reboot.

It’s easy to do with any webcam, just take a look on the USB bus for the vendor and product IDs. For example, here’s a Logitech C930e (which is probably the nicest webcam I’ve ever used, and also works perfectly under Fedora).

Bus 001 Device 022: ID 046d:0843 Logitech, Inc. Webcam C930e

So you would replace the following in your udev rule:

ATTRS{idVendor}==“046d”
ATTRS{idProduct}==“0843”
SYMLINK+=“c930e”

Note that SYMLINK is not necessary, it just creates an extra /dev entry, such as /dev/c930e, which is useful if you have multiple webcams.

Building a Mini-ITX NAS? Don’t buy a Silverstone DS380 case.

Edit: I made some changes which have dropped the temps to around 40 degrees at idle (haven’t tested at load yet). The case has potential, but I still think it’s slightly too cramped and the airflow is not good enough.

Here’s what I changed:

Rearranged the drives to leave a gap between each one, which basically limits the unit to 4 drives instead of 8

Inverted the PSU as per suggestion from Dan, so that it helps to draw air through the case. The default for the PSU is to draw air from outside and bypass the case.

Plugged the rear and side fans directly into the PSU molex connector, rather than through mainboard and rear of hard drive chassis

So I’m building a NAS (running Fedora Server) and thought that the Silverstone DS380 case looked great. It has 8 hot-swappable SATA bays, claims decent cooling with filters, neat form factor.

It requires an SFX PSU, but there are some that have enough juice on the 12v rail (although avoid the SilverStone SX500-LG, it’s slightly too long) so that it’s not a major problem (although I would prefer standard ATX).

So I got one to run low-power i3, C226 chipset mainboard and five HGST 3TB NAS drives. Unfortunately the cooling through the drives is pretty much non-existent. The two fans on the side draw air in but blow onto the hotswap chassis and nothing really draws air through it.

As a result, many of the drives run around 65 degrees Celsius at idle (tested overnight) which is already outside of the drives’ recommended temperature range of 0-60 degrees.

I’ve replaced the case with my second choice Fractal Design NODE 304 and the drives at idle all sit at around 35 degrees.

It has two smaller fans at the front to bring air directly over the drives and a larger one at the rear, with a manual L/M/H speed controller for all three on the rear of the case. As a bonus, it uses a standard ATX power supply and has plenty of room for it.

The only downside I’ve found so far is the lack of hot-swap, but my NAS isn’t mission-critical so that’s not a deal breaker for me.

Your mileage might vary, but I won’t buy the DS380 for a NAS again, unless it’s going to run full of SSDs or something (or I heavily mod the case). It’s OK for a small machine though without a bunch of disks (shame!) and that’s what I’ve re-purposed it for now.

-c

Btrfs RAID 6 on dm-crypt on Fedora (post updated)

Update 2016-08-26: A nasty bug was found in the RAID5/6 Btrfs parity calculation, so I recommend using RAID 10 for now. Where I use raid6 below you may want to change this to raid10. See this post for how to migrate to RAID 10.

I’m building a NAS and given the spare drives I have at the moment, thought I’d have a play with Btrfs. Apparently RAID 6 is relatively safe now (update: turns out, it’s not), so why not put it through its paces? As Btrfs doesn’t support encryption, I will need to build it on top of dm-crypt.

Boot drive:

/dev/sda

Data drives:

/dev/sdb
/dev/sdc
/dev/sdd
/dev/sde
/dev/sdf

I installed Fedora 24 Server onto /dev/sda and just went from there, opening a root shell.

# Install the btrfs and crypt packages (if not already there) so that this will actually work. dnf install -y btrfs-progs cryptsetup

WARNING WARNING WARNING
The following cryptsetup commands will wipe any drives you specify below. Please make sure you are specifying the correct drives.

# Setup dm-crypt on each data drive # and populate the crypttab file. for x in b c d e f ; do cryptsetup luksFormat /dev/sd${x} UUID="$(cryptsetup luksUUID /dev/sd${x})" echo "luks-${UUID} UUID=${UUID} none" >> /etc/crypttab done # Rebuild the initial ramdisk with crypt support echo "add_dracutmodules+=crypt" >> /etc/dracut.conf.d/crypt.conf dracut -fv # Verify that it now has my crypttab lsinitrd /boot/initramfs-$(uname -r).img |grep crypttab # Reboot and verify initramfs prompts to unlock the devices reboot # After boot, verify devices exist ls -l /dev/mapper/luks*

OK, so now I have a bunch of encrypted disks, it’s time to put btrfs into action (note the label, btrfs_data):
# Get LUKS UUIDs and create btrfs raid filesystem for x in b c d e f ; do DEVICES="${DEVICES} $(cryptsetup luksUUID /dev/sd${x}\ |sed 's|^|/dev/mapper/luks-|g')" done mkfs.btrfs -L btrfs_data -m raid6 -d raid6 ${DEVICES}

See all our current btrfs volumes:
btrfs fi show

Get the UUID of the filesystem so that we can create an entry in fstab, using the label we created before:
UUID=$(btrfs fi show btrfs_data |grep uuid |awk '{print $4}') echo "UUID=${UUID} /mnt/btrfs_data btrfs noatime,subvolid=0 0 0"\ >> /etc/fstab

Now, let’s create the mountpoint and mount the device:
mkdir /mnt/btrfs_data mount -a

Check data usage:
btrfs filesystem df /mnt/btrfs_data/

This has mounted the root of the filesystem to /mnt/btrfs_data, however we can also create subvolumes. Let’s create one called “share” for shared network data:
btrfs subvolume create /mnt/btrfs_data/share

You can mount this specific volume directly, let’s add it to fstab:
echo "UUID=${UUID} /mnt/btrfs_share btrfs noatime,subvol=share 0 0"\ >> /etc/fstab mkdir /mnt/btrfs_share mount /mnt/btrfs_share

You can list subvolumes easily by referencing our mounted Btrfs volume:
btrfs subvolume list -p /mnt/btrfs_data/

If you want to delete a subvolume, first unmount it, then remove it from fstab, delete the Btrfs subvolume and finally remove the mount point.
umount /mnt/btrfs_share sed -i /btrfs_share/d /etc/fstab btrfs subvolume delete /mnt/btrfs_data/share /mnt/btrfs_share

Now I plugged in a few backup drives and started rsyncing a few TB across to the device. It seemed to work well!

There are lots of other things you can play with, like snapshots, compression, defragment, scrub (use checksums to repair corrupt data), rebalance (re-allocates blocks across devices) etc. You can even convert existing file systems with btrfs-convert command, and use rebalance to change the RAID level. Neat!

Then I thought I’d try the rebalance command just to see how that works with a RAID device. Given it’s a large device, I kicked it off and went to do something else. I returned to an unwakeable machine… hard-resetting, journalctl -b -1 told me this sad story:
Nov 14 06:03:42 localhost.localdomain kernel: ------------[ cut here ]------------ Nov 14 06:03:42 localhost.localdomain kernel: kernel BUG at fs/btrfs/extent-tree.c:1833! Nov 14 06:03:42 localhost.localdomain kernel: invalid opcode: 0000 [#1] SMP Nov 14 06:03:42 localhost.localdomain kernel: Modules linked in: fuse joydev synaptics_usb uas usb_storage rfcomm cmac nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtab Nov 14 06:03:42 localhost.localdomain kernel: snd_soc_core snd_hda_codec rfkill snd_compress snd_hda_core snd_pcm_dmaengine ac97_bus snd_hwdep snd_seq snd_seq_device snd_pcm mei_me dw_dmac i2c_designware_platform snd_timer snd_soc_sst_a Nov 14 06:03:42 localhost.localdomain kernel: CPU: 0 PID: 6274 Comm: btrfs Not tainted 4.2.5-300.fc23.x86_64 #1 Nov 14 06:03:42 localhost.localdomain kernel: Hardware name: Gigabyte Technology Co., Ltd. Z97N-WIFI/Z97N-WIFI, BIOS F5 12/08/2014 Nov 14 06:03:42 localhost.localdomain kernel: task: ffff88006fd69d80 ti: ffff88000e344000 task.ti: ffff88000e344000 Nov 14 06:03:42 localhost.localdomain kernel: RIP: 0010:[] [] insert_inline_extent_backref+0xe7/0xf0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: RSP: 0018:ffff88000e3476a8 EFLAGS: 00010293 Nov 14 06:03:42 localhost.localdomain kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000 Nov 14 06:03:42 localhost.localdomain kernel: RDX: ffff880000000000 RSI: 0000000000000001 RDI: 0000000000000000 Nov 14 06:03:42 localhost.localdomain kernel: RBP: ffff88000e347728 R08: 0000000000004000 R09: ffff88000e3475a0 Nov 14 06:03:42 localhost.localdomain kernel: R10: 0000000000000000 R11: 0000000000000002 R12: ffff88021522f000 Nov 14 06:03:42 localhost.localdomain kernel: R13: ffff88013f868480 R14: 0000000000000000 R15: 0000000000000000 Nov 14 06:03:42 localhost.localdomain kernel: FS: 00007f66268a08c0(0000) GS:ffff88021fa00000(0000) knlGS:0000000000000000 Nov 14 06:03:42 localhost.localdomain kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 14 06:03:42 localhost.localdomain kernel: CR2: 000055a79c7e6fd0 CR3: 00000000576ce000 CR4: 00000000001406f0 Nov 14 06:03:42 localhost.localdomain kernel: Stack: Nov 14 06:03:42 localhost.localdomain kernel: 0000000000000000 0000000000000005 0000000000000001 0000000000000000 Nov 14 06:03:42 localhost.localdomain kernel: 0000000000000001 ffffffff81200176 0000000000270026 ffffffffa0925d4a Nov 14 06:03:42 localhost.localdomain kernel: 0000000000002158 00000000a7c0ba4c ffff88021522d800 0000000000000000 Nov 14 06:03:42 localhost.localdomain kernel: Call Trace: Nov 14 06:03:42 localhost.localdomain kernel: [] ? kmem_cache_alloc+0x1d6/0x210 Nov 14 06:03:42 localhost.localdomain kernel: [] ? btrfs_alloc_path+0x1a/0x20 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] __btrfs_inc_extent_ref.isra.52+0xa9/0x270 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] __btrfs_run_delayed_refs+0xc84/0x1080 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_run_delayed_refs.part.73+0x74/0x270 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] ? btrfs_release_path+0x2b/0xa0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_run_delayed_refs+0x15/0x20 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_commit_transaction+0x56/0xad0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] prepare_to_merge+0x1fe/0x210 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] relocate_block_group+0x25e/0x6b0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_relocate_block_group+0x1ca/0x2c0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_relocate_chunk.isra.39+0x3e/0xb0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_balance+0x9c4/0xf80 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_ioctl_balance+0x3c4/0x3d0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] btrfs_ioctl+0x541/0x2750 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: [] ? lru_cache_add+0x1c/0x50 Nov 14 06:03:42 localhost.localdomain kernel: [] ? lru_cache_add_active_or_unevictable+0x32/0xd0 Nov 14 06:03:42 localhost.localdomain kernel: [] ? handle_mm_fault+0xc8a/0x17d0 Nov 14 06:03:42 localhost.localdomain kernel: [] ? cp_new_stat+0xb3/0x190 Nov 14 06:03:42 localhost.localdomain kernel: [] do_vfs_ioctl+0x295/0x470 Nov 14 06:03:42 localhost.localdomain kernel: [] ? selinux_file_ioctl+0x4d/0xc0 Nov 14 06:03:42 localhost.localdomain kernel: [] SyS_ioctl+0x79/0x90 Nov 14 06:03:42 localhost.localdomain kernel: [] ? do_page_fault+0x2f/0x80 Nov 14 06:03:42 localhost.localdomain kernel: [] entry_SYSCALL_64_fastpath+0x12/0x71 Nov 14 06:03:42 localhost.localdomain kernel: Code: 10 49 89 d9 48 8b 55 c0 4c 89 7c 24 10 4c 89 f1 4c 89 ee 4c 89 e7 89 44 24 08 48 8b 45 20 48 89 04 24 e8 5d d5 ff ff 31 c0 eb ac <0f> 0b e8 92 b7 76 e0 66 90 0f 1f 44 00 00 55 48 89 e5 Nov 14 06:03:42 localhost.localdomain kernel: RIP [] insert_inline_extent_backref+0xe7/0xf0 [btrfs] Nov 14 06:03:42 localhost.localdomain kernel: RSP Nov 14 06:03:42 localhost.localdomain kernel: ---[ end trace 63b75c57d2feac56 ]---

Bummer!

Looks like rebalance has a major bug at the moment. I did a search and others have the same problem, looks like I’m hitting this bug. I’ve reported it on Fedora Bugzilla.

Anyway, so I won’t do a rebalance at the moment, but other than that, btrfs seems pretty neat. I will make sure I keep my backups up-to-date though, just in case…