Tag Archive for 'jockey'

Decorating (a)kmod packages with modalias info for use with RPM

Originally posted by firnsy at Korora Project news.

Whilst developing Pharlap, our utility for easing the installation and removal of drivers, we came across a big hurdle that other distributions had seemingly solved. The hurdle was being able to identify packages that provide support for a particular piece of hardware. Our initial workarounds used a dedicated map and for a while it was sufficient but it wasn’t ideal. Over time, the frustration of it’s inelegance grew and thus began our journey to investigate a more elegant solution.

Before developing Pharlap, there was Jockey, originally ported over from Ubuntu land by Hedayat Vatankhah for his Parsidora Fedora Remix. We started to contribute to and incorporate Hedayat’s work around version 16. At this time Hedayat, proposed the integration of Jockey into the RPM Fusion repositories which was met with a level of positivity. Could this already be implemented and we just don’t know? Darn, doesn’t look like it. Let’s continue.

So I mentioned earlier that other distributions had already solved this problem and indeed they had Debian/Ubuntu decorate their kernel module packages with the modaliases that the modules provide support for. Awesome? Yes it is! That allows the higher level package utilities to query the “provides” information using a device ID of interest.

So with that in mind, we set out to identify how we could adequately decorate kmod and akmod packages with appropriate modalias information.

kmod Packages

Starting with kmod packages (more specifically those rpm packages which contain pre-compiled kernel *.ko modules) it turns out that is reasonably trivial to decorate them using the builtin pluggable fileattr decorators of RPM with some post-processed information derived from ‘modinfo’.

To achieve this first build an appropriate “what provides” decorator that can interpret our kernel module files (*.ko). Fortunately this already exists in a standard installation but for reasons I’m not entirely sure of, is not enabled. So we just copy the modalias.prov out of the /usr/lib/rpm/redhat directory as a new file /usr/lib/rpm/kmod.prov.

Here’s the file for reference.


$ cat /usr/lib/rpm/kmod.prov
#! /bin/sh
 
# heavily based upon find-suggests.ksyms by Andreas Gruenbacher .
# with modifications by Michael Brown
#
# -- added module versioning info to modalias() symbols
# -- removed code which inspects spec files.
 
IFS=$'\n'
 
#
# Initially, dont generate modalias() lines for kernel package. This needs
# additional discussion. Would like to eventually add them for
# completeness, so that we can determine when drivers are folded into
# mainline kernel.
#
case "$1" in
kernel-module-*) ;; # Fedora kernel module package names start with
# kernel-module.
kernel*) is_kernel_package=1 ;;
esac
 
if ! [ -z "$is_kernel_package" ]; then
cat > /dev/null
exit 0
fi
 
print_modaliases() {
declare class=$1 variants=$2 pos=$3
if [ -n "$variants" ]; then
echo "${class:0:pos}[$variants]${class:pos+1}"
else
[ -z "$class" ] || echo "$class"
fi
}
 
combine_modaliases() {
declare tag class variants pos n
read class
while read tag; do
for ((n=0; n<${#class}; n++)); do
if [ "*" != "${class:n:1}" -a \
"${class:0:n}" = "${tag:0:n}" -a \
"${class:n+1}" = "${tag:n+1}" ] &&
( [ -z "$pos" ] || [ $n = $pos ] ); then
variants="${variants:-${class:n:1}}${tag:n:1}"
pos=$n
break
fi
done
if [ $n -eq ${#class} ]; then
print_modaliases "$class" "$variants" "$pos"
variants=
pos=
class=$tag
fi
done
print_modaliases "$class" "$variants" "$pos"
}
 
for module in $(grep -E '/lib/modules/.+\.ko$') $*; do
# | head -n1 because some modules have *two* version tags. *cough*b44*cough*
modver=$(/sbin/modinfo -F version "$module"| head -n1)
modver=${modver// /_}
 
# only add version tag if it has a version
if [ -n "$modver" ]; then
/sbin/modinfo -F alias "$module" \
| sed -nre "s,(.+),modalias(\\1) = $modver,p"
else
/sbin/modinfo -F alias "$module" \
| sed -nre "s,(.+),modalias(\\1),p"
fi
done \
| sort -u \
| combine_modaliases

We need to plug in our new provider by adding an attribute file in the /usr/lib/rpm/fileattrs directory. To follow suit, we'll call it kmod.attr which looks like this:


$ cat /usr/lib/rpm/fileattrs/kmod.attr
%__kmod_provides %{_rpmconfigdir}/kmod.prov
%__kmod_path ^/usr/lib/modules.*\\.ko$

The two lines indicate that any kernel module captured by %__kmod_path is to be passed onto the kmod.prov decorator.

So how does it look? Here's a listing of the provides for a kmod package built with the above changes:


$ rpm -qp --provides ./kmod-wl-3.13.10-200.fc20.x86_64-6.30.223.142-5.fc20.x86_64.rpm
kernel-modules-for-kernel = 3.13.10-200.fc20.x86_64
kmod-wl-3.13.10-200.fc20.x86_64 = 6.30.223.142-5.fc20
kmod-wl-3.13.10-200.fc20.x86_64(x86-64) = 6.30.223.142-5.fc20
modalias(pci:v*d*sv*sd*bc02sc80i*)
wl-kmod = 6.30.223.142-5.fc20

Sweet, that looks exactly like what we want. So that takes care of kmods, what about akmods?

akmod Packages

Unfortunately the above method won't satisfy our initial requirement for akmods to also provide modalias information. The main reason is that akmod packages don't contain any pre-built kernel modules, they contain the source RPM from which a suitable kmod package can be built from.

Damn! Ideally, doing a "provides" search via dnf or yum should return both kmod and akmod packages.

We mentioned that an akmod package actually contains the source code and thus no directly suitable files (such as *.ko files) that can be interrogated by the fileattrs. Fortunately, akmod packages are produced by the same spec that is used to create the kmod packages and thus we have the ability to leverage some information to generate a dedicated file with sufficient information that can be interrogated for decoration at a later stage.

So looking at the typical structure of an akmod package it contains normally two files, the source RPM and a symlink to the latest source RPM (normally itself). Our proposal involves the addition of another file using similar unique naming to the source RPM (e.g. kmod-$name-$version-$release.modalias) that contains the associated modaliases that will be present when the kmod is built.

With the file in place, we can then perform a similar process to what we used for the kmod packages and ensure the decorator can process our new modalias file.

So in order to build this file, we need to unravel the complexities of how akmods are produced. I won't go into the nitty gritty but suffice to say there's a reasonable amount of magic provided by the kmodtool which does some dynamic macro creation for the kmod spec files. The final stages of these spec files, throws a call to %{?akmod_install}. It's this macro that we need to extend to create our modalias file. The following small diff is all that is needed to generate the .modalias file which we can then have picked up by an appropriate decorator.


$ diff -Nurd /usr/bin/kmodtool /tmp/kmodtool
--- /usr/bin/kmodtool 2013-12-08 04:17:24.000000000 +1100
+++ /tmp/kmodtool 2014-04-22 16:35:50.564309505 +1000
@@ -66,7 +66,14 @@
rpmbuild --define "_sourcedir %{_sourcedir}" \\\
--define "_srcrpmdir \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/" \\\
-bs --nodeps %{_specdir}/%{name}.spec ; \\\
-ln -s \$(ls \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/) \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod.latest
+ln -s \$(ls \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/) \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod.latest ; \\\
+for kernel_version in %%{?kernel_versions}; do pushd _kmod_build_\${kernel_version%%___*} ; \\\
+for module in *.ko; do \\\
+ modver=\$(modinfo -F version "\$module"| head -n1) \\\
+ modver=\${modver// /_} \\\
+ [ -n "\$modver" ] && modinfo -F alias "\$module" | sed -nre "s,(.+),modalias(\\\\1) = \$modver,p" || modinfo -F alias "\$module" | sed -nre "s,(.+),modalias(\\\\1),p" ; \\\
+done >> \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod-%{version}-%{release}.modalias ; \\\
+popd ; done
 
%package -n akmod-${kmodname}
Summary: Akmod package for ${kmodname} kernel module(s)

With our modalias file now being created, we need to build an appropriate decorator. The astute will notice that a portion of our original decorator has made it's way into the macro diff above. This in turn allows us to simplify the final decorator which in this case we call modalias.prov and place it in the /usr/lib/rpm directory.


$ cat /usr/lib/rpm/modalias.prov
#! /bin/sh
#
# heavily based upon find-suggests.ksyms by Andreas Gruenbacher .
# with modifications by Michael Brown
#
# -- modalias file already contains modalias information and just needs to
# be sorted and combined
 
print_modaliases() {
declare class=$1 variants=$2 pos=$3
if [ -n "$variants" ]; then
echo "${class:0:pos}[$variants]${class:pos+1}"
else
[ -z "$class" ] || echo "$class"
fi
}
 
combine_modaliases() {
declare tag class variants pos n
read class
while read tag; do
for ((n=0; n<${#class}; n++)); do
if [ "*" != "${class:n:1}" -a \
"${class:0:n}" = "${tag:0:n}" -a \
"${class:n+1}" = "${tag:n+1}" ] &&
( [ -z "$pos" ] || [ $n = $pos ] ); then
variants="${variants:-${class:n:1}}${tag:n:1}"
pos=$n
break
fi
done
if [ $n -eq ${#class} ]; then
print_modaliases "$class" "$variants" "$pos"
variants=
pos=
class=$tag
fi
done
print_modaliases "$class" "$variants" "$pos"
}
 
while read FILE; do
cat $FILE
done | sort -u | combine_modaliases

We hook up the modalias decorator by adding a modalias.attr attribute file in the /usr/lib/rpm/fileattrs directory, which looks like the following:


$ cat /usr/lib/rpm/fileattrs/modalias.attr
%__modalias_provides %{_rpmconfigdir}/modalias.prov
%__modalias_path ^/usr/src/akmods/.*\\.modalias$

And the final result yields:


$ rpm -qp --provides ./akmod-wl-6.30.223.142-5.fc20.x86_64.rpm
akmod-wl = 6.30.223.142-5.fc20
akmod-wl(x86-64) = 6.30.223.142-5.fc20
modalias(pci:v*d*sv*sd*bc02sc80i*)
wl-kmod = 6.30.223.142-5.fc20

Beautiful!

OK, so what now? Well this is just the results of us investigating if it was possible and how it could be done. The above mechanism would require a diff to the kmodtool and rpm-build packages to provide the auto-decorating of modalias information on kmod and akmod packages.

Knowing that this can work, we think it might be a good time to revisit the possibilities with the RPMFusion team and see if we can make this a reality that all users of RPMFusion packages can benefit from.

Stay tuned!

Recovering from 3rd party drivers, with Jockey

What happens if you enable a 3rd party driver (like ATI) with Jockey, but after a reboot your graphics is broken and you can’t log in to revert?

Answer: Use the command line version of Jockey!

Here’s how you can revert any installed 3rd party driver using Jockey from the command line:

  • At log in (perhaps with a broken X screen), switch to another terminal using the key combination:
  • Ctrl + Alt + F2

  • Log in as root
  • Get a list of enabled drivers from Jockey:
  • jockey --list

  • Disable the driver (such as the ATI driver):
  • jockey --disable=kmod:fglrx

  • Once it has finished, reboot!

When using Jockey, you should ensure that you have the latest updates applied, in particular the latest running kernel. This is because you might otherwise get a driver for a newer kernel, which might break your system upon reboot, or fail to activate.

You can also try using akmods with Jockey, which will build a new driver for you on reboot, if one does not exist for your running kernel (no matter what that is!).

-c

Using akmods with Jockey in Fedora

By default, 3rd party drivers (such as NVIDIA and ATI) are pre-compiled against a specific kernel version and delivered in a kmod package. However when there’s a kernel update, it might block because there is no updated driver package yet available. This can also break a system if you update the kernel and reboot.

Wouldn’t it be great if the system could automatically build a new driver if required? Yes, and this is where akmods come into play.

With akmods, when your system boots it will check to see if a registered driver needs to be built. If you have an updated kernel, then akmods will build a fresh driver for that kernel and install it – no need to wait for a kmod to be updated upstream. This avoids the broken system scenario above.

While akmods are nothing new, you can now use them with Jockey under Fedora 16 (rather than just kmod packages). Simply download the RPMS from a Kororaa mirror and install (or you could build your own from SRPM).

You will need:

  • jockey
  • jockey-selinux
  • jockey-modaliases
  • jockey-gtk or jockey-kde

Post install, edit the /etc/jockey.conf and set “akmods=true”. Then, you can run Jockey and install the driver you want.

Jockey driver manager

You will also need build tools like gcc, make and kernel-headers (and of course, akmods), however these should be pulled in as dependencies of the akmod package.

This is still an experimental feature and any feedback is welcome!

Jockey device manager port for Fedora (using Yum)

One of the things I wanted to do for Kororaa was make Ubuntu’s Jockey device manager work. This would replace my current dodgy script and provide a mechanism for extending support for other devices over time (if needed). I’ve put my code on github if you want to check it out.

It wasn’t meant to be a joke, but on the 1st April this year I contacted Martin Pitt (the Jockey project lead) to pitch the idea. He helpfully replied saying that they’d written Jockey to be pretty vendor agnostic and that he had already discussed the idea of porting it to Fedora with Jon Masters. Indeed, he had did have the test suite working on Fedora 12 already (using PackageKit).

So I started out trying to understand the code and seeing what was possible. The core of the work is in oslib.py which implements the package management side of things (the default code uses PackageKit). Ubuntu has their own apt implementation, so I wanted to make a native Yum implementation.

There is some detection code (which shouldn’t need to be touched) and then there are the handlers for each device you want to manage (nvidia, etc).

A few weekends ago, Jason Nielsen and I got together to work on the code. In a few hours we had ported oslib.py to Yum, so I think the majority of the work there is done (thanks for your help, Jason!). It’s a much cleaner implementation than running PackageKit and parsing the results, because Yum is written in Python :-)

That’s where it’s at now. Next I need to create some handlers and then begin some testing. I’m holding off on the next Kororaa release, because I want to get this working before I ship it. Longer-term this naturally won’t go into Fedora, but perhaps into RPMFusion.

Parsidora (another Fedora Remix) has also been working on Jockey, but using the PackageKit implementation (I’ve sent them a message to see if they are interested in the native Yum version). You can try their existing implementation for Fedora 15 if you’re interested.

Hopefully I’ll have everything ready for Kororaa in the next few weeks, before Fedora 16 comes out! :-)