Archive for the 'FOSS' Category

FreeIPA How To (Fedora)

My OpenLDAP How To (Fedora) article has proven very popular over the years but now I’ve mostly moved on to FreeIPA.

So I thought it might be good to write up a FreeIPA How To (Fedora) article (CC BY-SA 4.0) in case it’s useful for anyone else out there.

Decorating (a)kmod packages with modalias info for use with RPM

Originally posted by firnsy at Korora Project news.

Whilst developing Pharlap, our utility for easing the installation and removal of drivers, we came across a big hurdle that other distributions had seemingly solved. The hurdle was being able to identify packages that provide support for a particular piece of hardware. Our initial workarounds used a dedicated map and for a while it was sufficient but it wasn’t ideal. Over time, the frustration of it’s inelegance grew and thus began our journey to investigate a more elegant solution.

Before developing Pharlap, there was Jockey, originally ported over from Ubuntu land by Hedayat Vatankhah for his Parsidora Fedora Remix. We started to contribute to and incorporate Hedayat’s work around version 16. At this time Hedayat, proposed the integration of Jockey into the RPM Fusion repositories which was met with a level of positivity. Could this already be implemented and we just don’t know? Darn, doesn’t look like it. Let’s continue.

So I mentioned earlier that other distributions had already solved this problem and indeed they had Debian/Ubuntu decorate their kernel module packages with the modaliases that the modules provide support for. Awesome? Yes it is! That allows the higher level package utilities to query the “provides” information using a device ID of interest.

So with that in mind, we set out to identify how we could adequately decorate kmod and akmod packages with appropriate modalias information.

kmod Packages

Starting with kmod packages (more specifically those rpm packages which contain pre-compiled kernel *.ko modules) it turns out that is reasonably trivial to decorate them using the builtin pluggable fileattr decorators of RPM with some post-processed information derived from ‘modinfo’.

To achieve this first build an appropriate “what provides” decorator that can interpret our kernel module files (*.ko). Fortunately this already exists in a standard installation but for reasons I’m not entirely sure of, is not enabled. So we just copy the modalias.prov out of the /usr/lib/rpm/redhat directory as a new file /usr/lib/rpm/kmod.prov.

Here’s the file for reference.


$ cat /usr/lib/rpm/kmod.prov
#! /bin/sh
 
# heavily based upon find-suggests.ksyms by Andreas Gruenbacher .
# with modifications by Michael Brown
#
# -- added module versioning info to modalias() symbols
# -- removed code which inspects spec files.
 
IFS=$'\n'
 
#
# Initially, dont generate modalias() lines for kernel package. This needs
# additional discussion. Would like to eventually add them for
# completeness, so that we can determine when drivers are folded into
# mainline kernel.
#
case "$1" in
kernel-module-*) ;; # Fedora kernel module package names start with
# kernel-module.
kernel*) is_kernel_package=1 ;;
esac
 
if ! [ -z "$is_kernel_package" ]; then
cat > /dev/null
exit 0
fi
 
print_modaliases() {
declare class=$1 variants=$2 pos=$3
if [ -n "$variants" ]; then
echo "${class:0:pos}[$variants]${class:pos+1}"
else
[ -z "$class" ] || echo "$class"
fi
}
 
combine_modaliases() {
declare tag class variants pos n
read class
while read tag; do
for ((n=0; n<${#class}; n++)); do
if [ "*" != "${class:n:1}" -a \
"${class:0:n}" = "${tag:0:n}" -a \
"${class:n+1}" = "${tag:n+1}" ] &&
( [ -z "$pos" ] || [ $n = $pos ] ); then
variants="${variants:-${class:n:1}}${tag:n:1}"
pos=$n
break
fi
done
if [ $n -eq ${#class} ]; then
print_modaliases "$class" "$variants" "$pos"
variants=
pos=
class=$tag
fi
done
print_modaliases "$class" "$variants" "$pos"
}
 
for module in $(grep -E '/lib/modules/.+\.ko$') $*; do
# | head -n1 because some modules have *two* version tags. *cough*b44*cough*
modver=$(/sbin/modinfo -F version "$module"| head -n1)
modver=${modver// /_}
 
# only add version tag if it has a version
if [ -n "$modver" ]; then
/sbin/modinfo -F alias "$module" \
| sed -nre "s,(.+),modalias(\\1) = $modver,p"
else
/sbin/modinfo -F alias "$module" \
| sed -nre "s,(.+),modalias(\\1),p"
fi
done \
| sort -u \
| combine_modaliases

We need to plug in our new provider by adding an attribute file in the /usr/lib/rpm/fileattrs directory. To follow suit, we'll call it kmod.attr which looks like this:


$ cat /usr/lib/rpm/fileattrs/kmod.attr
%__kmod_provides %{_rpmconfigdir}/kmod.prov
%__kmod_path ^/usr/lib/modules.*\\.ko$

The two lines indicate that any kernel module captured by %__kmod_path is to be passed onto the kmod.prov decorator.

So how does it look? Here's a listing of the provides for a kmod package built with the above changes:


$ rpm -qp --provides ./kmod-wl-3.13.10-200.fc20.x86_64-6.30.223.142-5.fc20.x86_64.rpm
kernel-modules-for-kernel = 3.13.10-200.fc20.x86_64
kmod-wl-3.13.10-200.fc20.x86_64 = 6.30.223.142-5.fc20
kmod-wl-3.13.10-200.fc20.x86_64(x86-64) = 6.30.223.142-5.fc20
modalias(pci:v*d*sv*sd*bc02sc80i*)
wl-kmod = 6.30.223.142-5.fc20

Sweet, that looks exactly like what we want. So that takes care of kmods, what about akmods?

akmod Packages

Unfortunately the above method won't satisfy our initial requirement for akmods to also provide modalias information. The main reason is that akmod packages don't contain any pre-built kernel modules, they contain the source RPM from which a suitable kmod package can be built from.

Damn! Ideally, doing a "provides" search via dnf or yum should return both kmod and akmod packages.

We mentioned that an akmod package actually contains the source code and thus no directly suitable files (such as *.ko files) that can be interrogated by the fileattrs. Fortunately, akmod packages are produced by the same spec that is used to create the kmod packages and thus we have the ability to leverage some information to generate a dedicated file with sufficient information that can be interrogated for decoration at a later stage.

So looking at the typical structure of an akmod package it contains normally two files, the source RPM and a symlink to the latest source RPM (normally itself). Our proposal involves the addition of another file using similar unique naming to the source RPM (e.g. kmod-$name-$version-$release.modalias) that contains the associated modaliases that will be present when the kmod is built.

With the file in place, we can then perform a similar process to what we used for the kmod packages and ensure the decorator can process our new modalias file.

So in order to build this file, we need to unravel the complexities of how akmods are produced. I won't go into the nitty gritty but suffice to say there's a reasonable amount of magic provided by the kmodtool which does some dynamic macro creation for the kmod spec files. The final stages of these spec files, throws a call to %{?akmod_install}. It's this macro that we need to extend to create our modalias file. The following small diff is all that is needed to generate the .modalias file which we can then have picked up by an appropriate decorator.


$ diff -Nurd /usr/bin/kmodtool /tmp/kmodtool
--- /usr/bin/kmodtool 2013-12-08 04:17:24.000000000 +1100
+++ /tmp/kmodtool 2014-04-22 16:35:50.564309505 +1000
@@ -66,7 +66,14 @@
rpmbuild --define "_sourcedir %{_sourcedir}" \\\
--define "_srcrpmdir \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/" \\\
-bs --nodeps %{_specdir}/%{name}.spec ; \\\
-ln -s \$(ls \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/) \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod.latest
+ln -s \$(ls \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/) \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod.latest ; \\\
+for kernel_version in %%{?kernel_versions}; do pushd _kmod_build_\${kernel_version%%___*} ; \\\
+for module in *.ko; do \\\
+ modver=\$(modinfo -F version "\$module"| head -n1) \\\
+ modver=\${modver// /_} \\\
+ [ -n "\$modver" ] && modinfo -F alias "\$module" | sed -nre "s,(.+),modalias(\\\\1) = \$modver,p" || modinfo -F alias "\$module" | sed -nre "s,(.+),modalias(\\\\1),p" ; \\\
+done >> \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod-%{version}-%{release}.modalias ; \\\
+popd ; done
 
%package -n akmod-${kmodname}
Summary: Akmod package for ${kmodname} kernel module(s)

With our modalias file now being created, we need to build an appropriate decorator. The astute will notice that a portion of our original decorator has made it's way into the macro diff above. This in turn allows us to simplify the final decorator which in this case we call modalias.prov and place it in the /usr/lib/rpm directory.


$ cat /usr/lib/rpm/modalias.prov
#! /bin/sh
#
# heavily based upon find-suggests.ksyms by Andreas Gruenbacher .
# with modifications by Michael Brown
#
# -- modalias file already contains modalias information and just needs to
# be sorted and combined
 
print_modaliases() {
declare class=$1 variants=$2 pos=$3
if [ -n "$variants" ]; then
echo "${class:0:pos}[$variants]${class:pos+1}"
else
[ -z "$class" ] || echo "$class"
fi
}
 
combine_modaliases() {
declare tag class variants pos n
read class
while read tag; do
for ((n=0; n<${#class}; n++)); do
if [ "*" != "${class:n:1}" -a \
"${class:0:n}" = "${tag:0:n}" -a \
"${class:n+1}" = "${tag:n+1}" ] &&
( [ -z "$pos" ] || [ $n = $pos ] ); then
variants="${variants:-${class:n:1}}${tag:n:1}"
pos=$n
break
fi
done
if [ $n -eq ${#class} ]; then
print_modaliases "$class" "$variants" "$pos"
variants=
pos=
class=$tag
fi
done
print_modaliases "$class" "$variants" "$pos"
}
 
while read FILE; do
cat $FILE
done | sort -u | combine_modaliases

We hook up the modalias decorator by adding a modalias.attr attribute file in the /usr/lib/rpm/fileattrs directory, which looks like the following:


$ cat /usr/lib/rpm/fileattrs/modalias.attr
%__modalias_provides %{_rpmconfigdir}/modalias.prov
%__modalias_path ^/usr/src/akmods/.*\\.modalias$

And the final result yields:


$ rpm -qp --provides ./akmod-wl-6.30.223.142-5.fc20.x86_64.rpm
akmod-wl = 6.30.223.142-5.fc20
akmod-wl(x86-64) = 6.30.223.142-5.fc20
modalias(pci:v*d*sv*sd*bc02sc80i*)
wl-kmod = 6.30.223.142-5.fc20

Beautiful!

OK, so what now? Well this is just the results of us investigating if it was possible and how it could be done. The above mechanism would require a diff to the kmodtool and rpm-build packages to provide the auto-decorating of modalias information on kmod and akmod packages.

Knowing that this can work, we think it might be a good time to revisit the possibilities with the RPMFusion team and see if we can make this a reality that all users of RPMFusion packages can benefit from.

Stay tuned!

Disable animations in GNOME 3 for older hardware

If you love GNOME 3 but the animations are really sluggish on an older machine (driving you crazy) then you can disable them using dconf-editor (install that first).

In dconf-editor, browse to org.gnome.desktop.interface and set enable-animations=false. Hope that stops someone from un-installing GNOME on an older machine…

Find out what is using your swap, by Erik Ljungstrom

Ever wondered what it is that’s using that swap on your machine? Erik has a great post about it and a script that will help answer that question.

Here is his script:
#!/bin/bash
# Get current swap usage for all running processes
# Erik Ljungstrom 27/05/2011
SUM=0
OVERALL=0
for DIR in `find /proc/ -maxdepth 1 -type d | egrep "^/proc/[0-9]"` ; do
PID=`echo $DIR | cut -d / -f 3`
PROGNAME=`ps -p $PID -o comm --no-headers`
for SWAP in `grep Swap $DIR/smaps 2>/dev/null| awk '{ print $2 }'`
do
let SUM=$SUM+$SWAP
done
echo "PID=$PID - Swap used: $SUM - ($PROGNAME )"
let OVERALL=$OVERALL+$SUM
SUM=0
 
done
echo "Overall swap used: $OVERALL"

Fix problem updating packages in Fedora/Korora due to broken SELinux update

Unfortunately an update to the SELinux policy package in Fedora 20 (and therefore Korora 20) caused RPM scriptlets to fail when updating packages.

This bug only affects systems that have SELinux mode set to enforcing (which is the default) and were updated to version 3.12.1-116 of the selinux-policy package. If you have seen the following sort of error when updating packages, then this bug may affect you:

warning: %post(libkcompactdisc-4.12.1-1.fc20.x86_64) scriptlet failed, exit status 127
Non-fatal POSTIN scriptlet failure in rpm package libkcompactdisc-4.12.1-1.fc20.x86_64

Below are the commands to resolve this issue (which has been fixed in an updated 3.12.1-117 version of selinux-policy).

sudo setenforce 0
sudo yum clean expire-cache
sudo yum update selinux-policy\*
sudo setenforce 1

The first command disables SELinux enforcement for the current session and the subsequent commands expire the yum cache and install the SELinux policy update which fixes this issue. The last command re-enables SELinux enforcement.

If you previously installed any packages which failed with scriptlet errors like above, you can reinstall them using the following command:

sudo yum reinstall

You can find out what packages were installed after the broken update using a command like this:

sudo sed '1,/selinux-policy-3.12.1-116/d' /var/log/yum.log

If you require any assistance please don’t hesitate to ask for help using Engage or jump onto the #korora channel in IRC freenode.net servers.

Add permanent rules to FirewallD

Someone at work wanted to know how to add rules permanently to FirewallD, Fedora’s dynamic firewall (iptables), so I’m posting it in case it’s useful to someone else.

Get the default zone, this is usually “public”:
firewall-cmd --get-active-zones

List services on that zone:
firewall-cmd --zone=public --list-all

Add required TCP ports (let’s do port 80):
firewall-cmd --permanent --zone=public --add-port=80/tcp

If you need a UDP port:
firewall-cmd --permanent --zone=public --add-port=123/udp

You could restart the firewall for them to take affect, or set the rules again without the –permanent option to add them dynamically.

Restart firewall:
systemctl restart firewalld.service

You can also specify services, rather than ports if you like.

sudo firewall-cmd --permanent --zone=public --add-service=http

You’re done!

Force rsync to use delta transfer to fix corrupt remote file

We host our Korora Project ISO images on SourceForge and I (naturally) use rsync to move them there (slowly, at 100kb/sec). Sometimes though the connection drops off and that’s OK because rsync picks up where it left off.

However occasionally the ISO ends up with the wrong checksum, so something went wrong in the transfer. No amount of re-rsyncing seems to fix this up because by default it uses file size and timestamps to check whether it should skip existing files.

Fortunately, I don’t need to re-send the whole file again as rsync can perform a delta transfer instead and only send the small difference. Yay!

The way I do this is by telling rsync to use checksum. I also need to do the transfer in-place (rsync normally writes a temporary file, then moves) and not to copy the whole file (the whole-file option disables deltas), something like:
rsync -Pa --checksum --inplace --no-whole-file local.file remote.server:

Here’s a real example:
chris@x220 ~ $ rsync -Pa --checksum --inplace --no-whole-file -e ssh korora-20-i386-cinnamon-live.iso csmart,kororaproject@frs.sourceforge.net:/home/frs/project/k/ko/kororaproject/20/
 
sending incremental file list
korora-20-i386-cinnamon-live.iso
  1,715,470,336 100% 220.87MB/s 0:00:07 (xfr#1, to-chk=0/1)

So rsync just saved me 4 hours of uploading the ISO again. Thanks rsync.

Enable and test TLS 1.2 in Firefox 26

Firefox has (unfortunately) lagged behind other browsers recently when it comes to implementing the more secure TLS 1.2 and it’s only now officially landing in the upcoming release 27. It can always use more testing though and if you’re running version 26 you can still enable it and test.

Browse to:
about:config

Set the following:
security.tls.version.max:3

This is the maximum supported protocol so it doesn’t mean that the sites you visit will now be using TLS 1.2. If you want to (try and) force it, there is a security.tls.version.min but be warned that probably most of your sites will fail.

You may also wish to disable this deprecated SSL3 algorithm:
security.ssl3.rsa_fips_des_ede3_sha:false

You can test this out by browsing to http://howsmyssl.com.

Calomel is a handy addon (BSD licence) to tell you what your secure connection negotiated to when you visit a site and gives it a score.

If you notice breakage, please report upstream.

Korora 20 (Peach) released

Today we released the final images for Korora Project (Fedora1 Remix) version 20 with Cinnamon, GNOME, KDE, MATE and Xfce desktops (in 32 and 64 bit).

The release was a little delayed because we were waiting for a few bug fixes to land, as well as Christmas and New Year holidays which got in the way.

We have also been hard at work building our new open source web platform which includes a replacement for our forums which is called Engage. Anyone who had an account with our old forums can log in to the new site, you will just get an email to activate your account first. Bug reports welcome!

1 Korora is not provided or supported by the Fedora Project. Official, unmodified Fedora software is available through the Fedora Project website.

Permanently fixing permissions on a shared git repo

When creating a shared git repository (perhaps on a central server) it’s good to use the –shared option:
git init --bare --shared

If you don’t, then you may find that repository permissions get clobbered each time a different person commits and no amount of umasks, chmods and sticky bits seem to help long term.

For your next shared repo that’s fine, but if you have an existing repository you can still fix this (assuming git is your group for write access):

ssh server
chown -Rf root:git /path/to/bare/git/repo
cd /path/to/bare/git/repo
git config core.sharedRepository group
find /path/to/bare/git/repo -type f | xargs chmod 664
find /path/to/bare/git/repo -type d | xargs chmod 775
find /path/to/bare/git/repo -type d | xargs chmod g+s

Enjoy some sanity!