Tag Archive for 'rpm'

Decorating (a)kmod packages with modalias info for use with RPM

Originally posted by firnsy at Korora Project news.

Whilst developing Pharlap, our utility for easing the installation and removal of drivers, we came across a big hurdle that other distributions had seemingly solved. The hurdle was being able to identify packages that provide support for a particular piece of hardware. Our initial workarounds used a dedicated map and for a while it was sufficient but it wasn’t ideal. Over time, the frustration of it’s inelegance grew and thus began our journey to investigate a more elegant solution.

Before developing Pharlap, there was Jockey, originally ported over from Ubuntu land by Hedayat Vatankhah for his Parsidora Fedora Remix. We started to contribute to and incorporate Hedayat’s work around version 16. At this time Hedayat, proposed the integration of Jockey into the RPM Fusion repositories which was met with a level of positivity. Could this already be implemented and we just don’t know? Darn, doesn’t look like it. Let’s continue.

So I mentioned earlier that other distributions had already solved this problem and indeed they had Debian/Ubuntu decorate their kernel module packages with the modaliases that the modules provide support for. Awesome? Yes it is! That allows the higher level package utilities to query the “provides” information using a device ID of interest.

So with that in mind, we set out to identify how we could adequately decorate kmod and akmod packages with appropriate modalias information.

kmod Packages

Starting with kmod packages (more specifically those rpm packages which contain pre-compiled kernel *.ko modules) it turns out that is reasonably trivial to decorate them using the builtin pluggable fileattr decorators of RPM with some post-processed information derived from ‘modinfo’.

To achieve this first build an appropriate “what provides” decorator that can interpret our kernel module files (*.ko). Fortunately this already exists in a standard installation but for reasons I’m not entirely sure of, is not enabled. So we just copy the modalias.prov out of the /usr/lib/rpm/redhat directory as a new file /usr/lib/rpm/kmod.prov.

Here’s the file for reference.


$ cat /usr/lib/rpm/kmod.prov
#! /bin/sh
 
# heavily based upon find-suggests.ksyms by Andreas Gruenbacher .
# with modifications by Michael Brown
#
# -- added module versioning info to modalias() symbols
# -- removed code which inspects spec files.
 
IFS=$'\n'
 
#
# Initially, dont generate modalias() lines for kernel package. This needs
# additional discussion. Would like to eventually add them for
# completeness, so that we can determine when drivers are folded into
# mainline kernel.
#
case "$1" in
kernel-module-*) ;; # Fedora kernel module package names start with
# kernel-module.
kernel*) is_kernel_package=1 ;;
esac
 
if ! [ -z "$is_kernel_package" ]; then
cat > /dev/null
exit 0
fi
 
print_modaliases() {
declare class=$1 variants=$2 pos=$3
if [ -n "$variants" ]; then
echo "${class:0:pos}[$variants]${class:pos+1}"
else
[ -z "$class" ] || echo "$class"
fi
}
 
combine_modaliases() {
declare tag class variants pos n
read class
while read tag; do
for ((n=0; n<${#class}; n++)); do
if [ "*" != "${class:n:1}" -a \
"${class:0:n}" = "${tag:0:n}" -a \
"${class:n+1}" = "${tag:n+1}" ] &&
( [ -z "$pos" ] || [ $n = $pos ] ); then
variants="${variants:-${class:n:1}}${tag:n:1}"
pos=$n
break
fi
done
if [ $n -eq ${#class} ]; then
print_modaliases "$class" "$variants" "$pos"
variants=
pos=
class=$tag
fi
done
print_modaliases "$class" "$variants" "$pos"
}
 
for module in $(grep -E '/lib/modules/.+\.ko$') $*; do
# | head -n1 because some modules have *two* version tags. *cough*b44*cough*
modver=$(/sbin/modinfo -F version "$module"| head -n1)
modver=${modver// /_}
 
# only add version tag if it has a version
if [ -n "$modver" ]; then
/sbin/modinfo -F alias "$module" \
| sed -nre "s,(.+),modalias(\\1) = $modver,p"
else
/sbin/modinfo -F alias "$module" \
| sed -nre "s,(.+),modalias(\\1),p"
fi
done \
| sort -u \
| combine_modaliases

We need to plug in our new provider by adding an attribute file in the /usr/lib/rpm/fileattrs directory. To follow suit, we'll call it kmod.attr which looks like this:


$ cat /usr/lib/rpm/fileattrs/kmod.attr
%__kmod_provides %{_rpmconfigdir}/kmod.prov
%__kmod_path ^/usr/lib/modules.*\\.ko$

The two lines indicate that any kernel module captured by %__kmod_path is to be passed onto the kmod.prov decorator.

So how does it look? Here's a listing of the provides for a kmod package built with the above changes:


$ rpm -qp --provides ./kmod-wl-3.13.10-200.fc20.x86_64-6.30.223.142-5.fc20.x86_64.rpm
kernel-modules-for-kernel = 3.13.10-200.fc20.x86_64
kmod-wl-3.13.10-200.fc20.x86_64 = 6.30.223.142-5.fc20
kmod-wl-3.13.10-200.fc20.x86_64(x86-64) = 6.30.223.142-5.fc20
modalias(pci:v*d*sv*sd*bc02sc80i*)
wl-kmod = 6.30.223.142-5.fc20

Sweet, that looks exactly like what we want. So that takes care of kmods, what about akmods?

akmod Packages

Unfortunately the above method won't satisfy our initial requirement for akmods to also provide modalias information. The main reason is that akmod packages don't contain any pre-built kernel modules, they contain the source RPM from which a suitable kmod package can be built from.

Damn! Ideally, doing a "provides" search via dnf or yum should return both kmod and akmod packages.

We mentioned that an akmod package actually contains the source code and thus no directly suitable files (such as *.ko files) that can be interrogated by the fileattrs. Fortunately, akmod packages are produced by the same spec that is used to create the kmod packages and thus we have the ability to leverage some information to generate a dedicated file with sufficient information that can be interrogated for decoration at a later stage.

So looking at the typical structure of an akmod package it contains normally two files, the source RPM and a symlink to the latest source RPM (normally itself). Our proposal involves the addition of another file using similar unique naming to the source RPM (e.g. kmod-$name-$version-$release.modalias) that contains the associated modaliases that will be present when the kmod is built.

With the file in place, we can then perform a similar process to what we used for the kmod packages and ensure the decorator can process our new modalias file.

So in order to build this file, we need to unravel the complexities of how akmods are produced. I won't go into the nitty gritty but suffice to say there's a reasonable amount of magic provided by the kmodtool which does some dynamic macro creation for the kmod spec files. The final stages of these spec files, throws a call to %{?akmod_install}. It's this macro that we need to extend to create our modalias file. The following small diff is all that is needed to generate the .modalias file which we can then have picked up by an appropriate decorator.


$ diff -Nurd /usr/bin/kmodtool /tmp/kmodtool
--- /usr/bin/kmodtool 2013-12-08 04:17:24.000000000 +1100
+++ /tmp/kmodtool 2014-04-22 16:35:50.564309505 +1000
@@ -66,7 +66,14 @@
rpmbuild --define "_sourcedir %{_sourcedir}" \\\
--define "_srcrpmdir \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/" \\\
-bs --nodeps %{_specdir}/%{name}.spec ; \\\
-ln -s \$(ls \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/) \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod.latest
+ln -s \$(ls \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/) \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod.latest ; \\\
+for kernel_version in %%{?kernel_versions}; do pushd _kmod_build_\${kernel_version%%___*} ; \\\
+for module in *.ko; do \\\
+ modver=\$(modinfo -F version "\$module"| head -n1) \\\
+ modver=\${modver// /_} \\\
+ [ -n "\$modver" ] && modinfo -F alias "\$module" | sed -nre "s,(.+),modalias(\\\\1) = \$modver,p" || modinfo -F alias "\$module" | sed -nre "s,(.+),modalias(\\\\1),p" ; \\\
+done >> \$RPM_BUILD_ROOT/%{_usrsrc}/akmods/${kmodname}-kmod-%{version}-%{release}.modalias ; \\\
+popd ; done
 
%package -n akmod-${kmodname}
Summary: Akmod package for ${kmodname} kernel module(s)

With our modalias file now being created, we need to build an appropriate decorator. The astute will notice that a portion of our original decorator has made it's way into the macro diff above. This in turn allows us to simplify the final decorator which in this case we call modalias.prov and place it in the /usr/lib/rpm directory.


$ cat /usr/lib/rpm/modalias.prov
#! /bin/sh
#
# heavily based upon find-suggests.ksyms by Andreas Gruenbacher .
# with modifications by Michael Brown
#
# -- modalias file already contains modalias information and just needs to
# be sorted and combined
 
print_modaliases() {
declare class=$1 variants=$2 pos=$3
if [ -n "$variants" ]; then
echo "${class:0:pos}[$variants]${class:pos+1}"
else
[ -z "$class" ] || echo "$class"
fi
}
 
combine_modaliases() {
declare tag class variants pos n
read class
while read tag; do
for ((n=0; n<${#class}; n++)); do
if [ "*" != "${class:n:1}" -a \
"${class:0:n}" = "${tag:0:n}" -a \
"${class:n+1}" = "${tag:n+1}" ] &&
( [ -z "$pos" ] || [ $n = $pos ] ); then
variants="${variants:-${class:n:1}}${tag:n:1}"
pos=$n
break
fi
done
if [ $n -eq ${#class} ]; then
print_modaliases "$class" "$variants" "$pos"
variants=
pos=
class=$tag
fi
done
print_modaliases "$class" "$variants" "$pos"
}
 
while read FILE; do
cat $FILE
done | sort -u | combine_modaliases

We hook up the modalias decorator by adding a modalias.attr attribute file in the /usr/lib/rpm/fileattrs directory, which looks like the following:


$ cat /usr/lib/rpm/fileattrs/modalias.attr
%__modalias_provides %{_rpmconfigdir}/modalias.prov
%__modalias_path ^/usr/src/akmods/.*\\.modalias$

And the final result yields:


$ rpm -qp --provides ./akmod-wl-6.30.223.142-5.fc20.x86_64.rpm
akmod-wl = 6.30.223.142-5.fc20
akmod-wl(x86-64) = 6.30.223.142-5.fc20
modalias(pci:v*d*sv*sd*bc02sc80i*)
wl-kmod = 6.30.223.142-5.fc20

Beautiful!

OK, so what now? Well this is just the results of us investigating if it was possible and how it could be done. The above mechanism would require a diff to the kmodtool and rpm-build packages to provide the auto-decorating of modalias information on kmod and akmod packages.

Knowing that this can work, we think it might be a good time to revisit the possibilities with the RPMFusion team and see if we can make this a reality that all users of RPMFusion packages can benefit from.

Stay tuned!

Building NVIDIA akmod package for RHEL/CentOS using Fedora and mock

Unfortunately there are no NVIDIA driver packages for RHEL/CentOS available on RPMFusion (that I could find), so this is how I built them. Disclaimer: This is just the way I did it and is probably dodgy. Suggestions welcome.

I have a dedicated build server and use mock so that all of the building is kept separate from my running system in a chroot environment. Packages are built against epel (Extra Packages for Enterprise Linux) and I’m using 64 bit.

We’re building an akmod package (rather than a static kmod) which will push the compilation of the driver on a per kernel basis to the client. The benefit of this is you only need to install the akmod once and the system will compile the kmod drivers after every kernel update.

Install Fedora
Install Fedora and then configure RPMFusion repositories (free, free-updates, non-free, non-free-updates).

Install mock and add yourself to the mock group (replace [username]). Log out and in or run newgrp:
$ sudo yum install mock
$ sudo gpasswd -a [username]
$ newgrp mock

Create a temporary location to perform the tasks below (makes copy and paste from here easier)
$ mkdir ~/building-nvidia
$ cd ~/building-nvidia

Source RPMS
Now we’re ready to get the source RPMs:
$ yumdownloader --source akmod-nvidia akmods buildsys-build-rpmfusion-kerneldevpkgs-current kmodtool libvdpau nvidia-kmod-common nvidia-settings nvidia-xconfig

Initialise mock
Let’s initialise the chroot environment for epel:
$ mock -r epel-6-x86_64 --init

Build and install dependencies
Ordinarily, building an RPM using mock will automatically install any build dependencies into the chroot. By default however it can (obviously) only automatically install any packages that are available in the target distribution. This presents a little problem with RHEL and CentOS because some of the dependencies (such as kmodtool and akmods) are not available in these distributions. So, how do we get around this? We can tell mock to install any RPM we like into the chroot, they don’t have to be available in the target distribution.

But it gets a little more tricky. If said dependency doesn’t exist for RHEL/CentOS, then we need to build it (using mock) and then install it into mock’s chroot manually. Make sense?

Once we have these dependencies manually installed the trick is telling mock not to clean the chroot environment for any post build processes, else it would remove those dependencies we manually installed (and therefore you couldn’t build the packages!).

Building kmodtool
The first dependency we need to build is kmodtool, because it does much of the grunt work for kmods (akmods requires this). Note we will add the “–no-clean” option when building packages, as we don’t want the packages we installed to be removed from the chroot (they’re dependencies).

Go for it:
$ mock -r epel-6-x86_64 --no-clean --no-cleanup-after --resultdir=/var/lib/mock/epel-6-x86_64/result/ --rebuild kmodtool*.src.rpm

Installing kmodtool
Install kmodtool into the chroot:
$ mock -r epel-6-x86_64 --install /var/lib/mock/epel-6-x86_64/result/kmodtool*.noarch.rpm

Building akmods
It gets slightly more tricky. The akmods package we are using is designed for Fedora and fails to build because it can’t detect the version of Fedora we are using and so tries to use systemd (Fedora’s boot system) which is not yet used in RHEL/CentOS. This means we need to install the akmod source rpm, modify the spec file and then re-create the source rpm for compilation (this will happen in your home directory).
$ rpm -ivh akmods*.src.rpm
$ sed -i 's/%fedora\ <=16/0%{?rhel}/g' ~/rpmbuild/SPECS/akmods.spec
$ rpmbuild -bs ~/rpmbuild/SPECS/akmods.spec
$ mock -r epel-6-x86_64 --no-clean --no-cleanup-after --rebuild --resultdir=/var/lib/mock/epel-6-x86_64/result/ ~/rpmbuild/SRPMS/akmods*.src.rpm

This will build akmods RPM and place it in mock's result dir which we will install next.

Installing akmods
Install akmods into the chroot:
$ mock -r epel-6-x86_64 --install /var/lib/mock/epel-6-x86_64/result/akmods*.noarch.rpm

Building buildsys-build-rpmfusion
The kmod package has some smarts to work out which kernel to build the kmod for. We need to build and install the buildsys-build-rpmfusion package which will give us an akmod compatible version of the package (actually it's a "provides"). This also requires kmodtool, hence we had to do this after the others:
$ mock -r epel-6-x86_64 --no-clean --no-cleanup-after --resultdir=/var/lib/mock/epel-6-x86_64/result/ --rebuild buildsys-build-rpmfusion*.src.rpm

Installing buildsys-build-rpmfusion
We only want to install the base package (not the 'current' package as that would require kernel-devel for your Fedora kernel).
$ rm /var/lib/mock/epel-6-x86_64/result/buildsys-build-rpmfusion-kerneldevpkgs-current*.x86_64.rpm
$ mock -r epel-6-x86_64 --install /var/lib/mock/epel-6-x86_64/result/buildsys-build*.x86_64.rpm

That's it for manual dependencies!

From now on, the packages we build will not be installed into the chroot, they will be copied and installed on the client machine.

Building nvidia-kmod package
The spec file for the nvidia-kmod package has a "buildforkernels" definition to tell it what kernel you want to build for, either current or latest kernel. Alternatively, it can build an akmod package (dynamically build package on boot), which is what we want. By default this option is set to "current" so we need to change this definition to "akmod" (hard-coded in the spec file).

To do this, we need to install the source RPM, change the option in its spec file, and rebuild the SRPM:
$ rpm -ivh nvidia-kmod*src.rpm
$ sed -i s/^%define\ buildforkernels\ current/%define\ buildforkernels\ akmod/ ~/rpmbuild/SPECS/nvidia-kmod.spec
$ rpmbuild -bs ~/rpmbuild/SPECS/nvidia-kmod.spec
$ mock -r epel-6-x86_64 --no-clean --rebuild ~/rpmbuild/SRPMS/nvidia-kmod*.src.rpm

If you don't want an akmod, then you need to specify the kernel that you're building for (this should install kernel-devel into the chroot). If you don't, you'll run into problems because the chroot will detect your Fedora kernel, not the kernel you need to build for. Note this is a different definition we are specifying (replace with the kernel version you want):
$ mock -r epel-6-x86_64 --define "kernels 2.6.32-220.17.1.el6.x86_64" --no-clean --rebuild nvidia-kmod*.src.rpm

Building remaining packages
Now build the rest of the packages:
$ mock -r epel-6-x86_64 --no-clean --rebuild \
libvdpau*.src.rpm nvidia-settings*.src.rpm \
nvidia-xconfig*.src.rpm xorg-x11-drv-nvidia*.src.rpm

Copy built packages
All of the build packages will be under mock's result directory, so we can now copy these out and install them on a RHEL/CentOS box!
$ mkdir {packages,source}
$ cp -r /var/lib/mock/epel-6-x86_64/result/*{noarch,x86_64}.rpm packages/
$ cp -r /var/lib/mock/epel-6-x86_64/result/*src.rpm source/

Now these can be published to a repository and pushed out to machines.

Install akmod-nvidia package
All you need to do is install the nvidia akmod package:
$ sudo yum install akmod-nvidia

This should automatically pull in all dependencies required to build the package, like kernel-devel, gcc, make, etc.

Finally, make sure that akmods is set to start on boot:
$ sudo chkconfig akmods on

Now each time a kernel update is released, the machine will automatically build the nvidia driver for you.

On becoming a Fedora maintainer

Recently, Rahul Sandaram (Fedora dev and creator of the Omega Fedora Remix) offered to sponsor me to become a Fedora maintainer, which I accepted. A day or so later I pushed my first updates into Fedora for deja-dup – the package I now co-maintain while I learn the ropes.

Previously, one would become a maintainer by first submitting a new package and thereby overtime demonstrating an understanding of the Fedora packaging process and guidelines. However recently a new system was approved whereby one could become a co-maintainer of an existing package, instead. This requires being sponsored by an existing maintainer, which Rahul was for me.

It’s quite a big task, but Rahul was very helpful and patient while I learned about the process. There is a lot of information on the Fedora Wiki but I found there wasn’t really a single place which provided a nice overview, broke down each section in detail, and then explained all the steps required. So that (along with my cautious nature) meant it took a dozen or so clarifying emails back and forth between Rahul and myself.

When I got my first understanding and taste of the system though, I’ve gotta say, I was quite impressed. The scale of the system and architecture and how it all works is amazing. The build files (RPM spec files) of all the packages in Fedora are kept under their own git repository. Hooks into this git system and special build tools manage the whole process. All in all, this automation makes it quite seamless.

So after reading and doing everything Rahul sent to me, I started on my journey.

There are two development systems to update now – rawhide (development tree) and F15 (the upcoming version). I wanted to update both of these to the latest upstream version, 17.90. I also wanted to update the stable Fedora 14 package to an upstream bug-fix release, version 16.1.1.

Cloning deja-dup from the git repo put me in the rawhide branch by default. I updated the spec file – increasing the version, adding python-cloudfiles as a new dependency, and putting in a detailed changelog. I then committed that to my local git repo with a comment linking to upstream. I then pulled those changes from rawhide into the f15 branch and updated f14.

commit bb1a5daa9feea74640fd84a0174e74e4e83ad34b
Author: Christopher Smart
Date: Sat Mar 5 21:23:07 2011 +1100
    
     Updated to upstream bugfix version 16.1.1.
     "This release fixes a bug in the just-released 16.1 that caused help
     documentation to not be translated."
     https://launchpad.net/deja-dup/+announcement/7239

Next I needed to test these packages, so I built a source RPM from the spec for each version. I discovered that one can either do a mock build on their local machine, or use Koji to do a scratch build on Fedora’s online build system. I did both, which worked well.

Using Koji meant I needed to first push my changes back to Fedora first though (or so I thought), so I was hesitant to do it without first checking with Rahul. This was the sort of thing that was emailed back and forth – I didn’t fully understand the system and didn’t want to break anything! For example, in order to test my F14 update I had to push back to the git repo – but what if I made a mistake in the process? Would my build on Koji push this broken package onto everyone’s system?

Fortunately, no. Once all the updating of the spec file and building is complete, the maintainer has to formally submit it into the update process before anything will happen. In addition, each package must then pass approval by other maintainers before it is approved and pushed out. You can see why it took some back and forth between Rahul and myself – I wanted to be sure I wasn’t going to do some damage :-)

Along the way I discovered new automated ways of doing things, such as validating a spec file, automatically downloading the source tarball to update the spec file, then uploading it to the Fedora build system.

In the end though, I think that I have a pretty good grasp of the overall process and am quite excited by the prospect of becoming a Fedora maintainer. I’m hoping that deja-dup comes out with a new update soon, so that I can do another update!

I’m still a little hesitant to go and change things – I don’t want to do the wrong thing or step on anyone’s toes. I did however create my own page on the Fedora Wiki, complete with a hackergotchi, of course.

So once again, thanks to Rahul for his patience and guidance :-)

How to upgrade Fedora (using PreUpgrade)

Coming from a Debian background, upgrading to new releases is second nature. Unfortunately, Fedora doesn’t seem to have it quite as down pat. Nevertheless, it’s possible and works quite well (but there are pitfalls, so read on).

While upgrading via PackageKit is coming, there’s a pretty decent way of doing it with PreUpgrade.

For the impatient, the steps are:

  • yum update
  • yum install preupgrade
  • preupgrade (reboot)
  • yum distro-sync
  • package-cleanup –orphans (be careful!)
  • package-cleanup –leaves (be careful!)

Now the longer version. First, update your current system:
yum update

Install preupgrade:
yum install preupgrade

Run the preupgrade tool, and follow the prompts (remote upgrades over VNC also supported):
preupgrade

PreUpgrade will prompt to reboot your computer and will update your system automatically. All of your packages should be updated and your repositories configured for you (I also had RPMFusion, Chromium and yum-rawhide which all updated as expected). On one of my systems, even the NVIDIA driver was automatically updated and configured. Now you’re booted into your new Fedora 14 system.

OK, this is where things get a little tricky. Some packages might be no-longer supported, there might be removed dependencies and the like. So, there are two neat commands that we will use to identify these packages, so that you can remove them.

package-cleanup --orphans
package-cleanup --leaves

But wait.. I discovered that there’s a more important step to do first, or you’ll cause yourself a headache!

On my machine, this wanted to remove some pretty important packages, like kdelibs.
[chris@shem ~]$ sudo package-cleanup --orphans
[sudo] password for chris:
Loaded plugins: refresh-packagekit, remove-with-leaves
akonadi-1.4.0-3.fc13.x86_64
kdeedu-marble-4.5.2-2.fc13.x86_64
kdeedu-marble-libs-4.5.2-2.fc13.x86_64
kdelibs-4.5.2-8.fc13.x86_64
kdelibs-common-4.5.2-8.fc13.x86_64
schroedinger-1.0.10-1.fc13.x86_64
xorg-x11-drv-wacom-0.10.8-2.fc13.x86_64

Fedora (unlike certain other distros I know) provides major updates to the latest versions of packages (like OpenOffice.org, KDE, GNOME, etc). Yay! It’s possible that your new Fedora system is running the same version of, say KDE, as your old one. This was indeed the case with my Fedora 13 to 14 upgrade (both run KDE 4.5.2 at time of F14 release).

Actually, the build of some important packages on Fedora 13 (like kdelibs-4.5.2-8.fc13.x86_64) were actually more recent than on the newer Fedora 14 system (kdelibs-4.5.2-5.fc14.x86_64).

The package-cleanup tool correctly lists kdelibs-4.5.2-8.fc13.x86_64 as being an orphan and if you were to remove this, you’d break your system badly. In fact, if you’re running yum’s brilliant new autoremove deps feature, as I am, you’ll lose most of your system. It makes sense – you are telling yum to remove kdelibs, so it goes and removes everything that relies on it! Yikes.

So first, we need to fix this by running the yum distro-sync command which recognises that we need to downgrade that kdelibs packages (and not remove it!).

[chris@shem ~]$ sudo yum distro-sync
Loaded plugins: refresh-packagekit, remove-with-leaves
Setting up Distribution Synchronization Process
Resolving Dependencies
--> Running transaction check
---> Package akonadi.x86_64 0:1.4.0-1.fc14 will be a downgrade
---> Package akonadi.x86_64 0:1.4.0-3.fc13 will be erased
---> Package kdeedu-marble.x86_64 0:4.5.2-1.fc14 will be a downgrade
---> Package kdeedu-marble.x86_64 0:4.5.2-2.fc13 will be erased
---> Package kdeedu-marble-libs.x86_64 0:4.5.2-1.fc14 will be a downgrade
---> Package kdeedu-marble-libs.x86_64 0:4.5.2-2.fc13 will be erased
---> Package kdelibs.x86_64 6:4.5.2-5.fc14 will be a downgrade
---> Package kdelibs.x86_64 6:4.5.2-8.fc13 will be erased
---> Package kdelibs-common.x86_64 6:4.5.2-5.fc14 will be a downgrade
---> Package kdelibs-common.x86_64 6:4.5.2-8.fc13 will be erased
---> Package schroedinger.x86_64 0:1.0.9-2.fc14 will be a downgrade
---> Package schroedinger.x86_64 0:1.0.10-1.fc13 will be erased
---> Package xorg-x11-drv-wacom.x86_64 0:0.10.8-1.20100726.fc14 will be a downgrade
---> Package xorg-x11-drv-wacom.x86_64 0:0.10.8-2.fc13 will be erased
--> Finished Dependency Resolution
--> Finding unneeded leftover dependencies
Found and removing 0 unneeded dependencies
----
Dependencies Resolved
=================================================
Package Arch Version Repository Size
=================================================
Downgrading:
akonadi x86_64 1.4.0-1.fc14 fedora 677 k
kdeedu-marble x86_64 4.5.2-1.fc14 fedora 14 M
kdeedu-marble-libs x86_64 4.5.2-1.fc14 fedora 902 k
kdelibs x86_64 6:4.5.2-5.fc14 fedora 12 M
kdelibs-common x86_64 6:4.5.2-5.fc14 fedora 1.7 M
schroedinger x86_64 1.0.9-2.fc14 fedora 276 k
xorg-x11-drv-wacom x86_64 0.10.8-1.20100726.fc14 fedora 73 k
---
Transaction Summary
=============================================================================================================================================
Downgrade 7 Package(s)
---
Total download size: 30 M
Is this ok [y/N]

Once we downgrade these packages, we can then remove any other orphans we might have, with package-cleanup –leaves and package-cleanup –orphans (I didn’t have any). One last thing to note – Fedora will not replace packages from the newer release if they are exactly the same version. For this reason, you will probably have some F13 packages still installed on your computer – don’t worry, that’s correct. They will be upgraded in time (if required).

So, now I think I know how to successfully upgrade the system without breaking it :-) If anyone has some other tips or corrections, please let me know! Hope that’s helpful.

-c

Testing yum’s autoremove orphaned deps feature

Yesterday I wrote about yum’s new feature which can automatically remove unused dependencies when a package is uninstalled.

Today I got my hands on a suitable build of yum (add the yum-rawhide repo and set clean_requirements_on_remove=1 under [main] in /etc/yum.conf) and I started to test it out on several packages, all of which introduced dependencies. My initial findings? It’s great.

Single package with dependencies
I started with a single package with several dependencies. When I uninstalled the parent package, yum also removed all of its dependencies which were not needed by any other package (i.e. orphaned dependencies).

Let’s look at an example; installing Ekiga which pulls in 5 dependencies.

yum install ekiga
...
Installed:
ekiga.x86_64 0:3.2.7-4.fc14
Dependency Installed:
evolution-data-server.x86_64 0:2.32.0-3.fc14 libgdata.x86_64 0:0.6.4-4.fc14 libgweather.x86_64 0:2.30.3-1.fc14 opal.x86_64 0:3.6.8-1.fc14
ptlib.x86_64 0:2.6.7-1.fc14

OK, so we have Ekiga, and we have 5 dependencies installed. What happens if I remove Ekiga?

yum erase ekiga
...
Removed:
ekiga.x86_64 0:3.2.7-4.fc14
Dependency Removed:
evolution-data-server.x86_64 0:2.32.0-3.fc14 libgdata.x86_64 0:0.6.4-4.fc14 libgweather.x86_64 0:2.30.3-1.fc14 opal.x86_64 0:3.6.8-1.fc14
ptlib.x86_64 0:2.6.7-1.fc14

As I was hoping, yum removed all 5 dependencies along with Ekiga itself. Ta da!

Result: Tick!

Packages with shared dependencies
I then tested two packages which have shared dependencies. What I would expect is that only the dependencies which are unique to the application being removed, are uninstalled. Other shared dependencies remain, because the other program still requires them and is not being removed.

Let’s look at an example; mplayer and vlc. Both require the libcaca library and installing both packages pulled in this library.
VLC needs libcaca:

yum deplist vlc |grep libcaca
dependency: libcaca.so.0()(64bit)
provider: libcaca.x86_64 0.99-0.10.beta17.fc14
dependency: libcaca.so.0()(64bit)
provider: libcaca.x86_64 0.99-0.10.beta17.fc14

And mplayer needs libcaca:

yum deplist mplayer |grep libcaca
dependency: libcaca.so.0()(64bit)
provider: libcaca.x86_64 0.99-0.10.beta17.fc14
dependency: libcaca.so.0()(64bit)
provider: libcaca.x86_64 0.99-0.10.beta17.fc14

When I want to remove mplayer, yum only removes those mplayer dependencies which vlc does not also require, and leaves the rest (including libcaca).


yum erase mplayer
...
Removed:
mplayer.x86_64 0:1.0-0.119.20100703svn.fc14
Dependency Removed:
libvdpau.x86_64 0:0.4.1-1.fc14.1 lzo.x86_64 0:2.03-3.fc12 mplayer-common.x86_64 0:1.0-0.119.20100703svn.fc14

Note that libcaca was not removed, which is the expected result.

Result: Tick!

Removing a package’s dependency
I then tried to remove a dependency, rather than the parent package. I would expect this to remove all parents which required that package, along with their other dependencies.

Let’s look at an example, gnash-plugin. This package pulls in three dependencies (including the actual gnash package).


yum install gnash-plugin
...
Installed:
gnash-plugin.x86_64 1:0.8.8-4.fc14
Dependency Installed:
agg.x86_64 0:2.5-9.fc13 gnash.x86_64 1:0.8.8-4.fc14 gtkglext-libs.x86_64 0:1.2.0-10.fc12

Now, let’s see what happens if I try and remove one of the dependencies.

yum erase agg
...
Removed:
agg.x86_64 0:2.5-9.fc13
Dependency Removed:
gnash.x86_64 1:0.8.8-4.fc14 gnash-plugin.x86_64 1:0.8.8-4.fc14 gtkglext-libs.x86_64 0:1.2.0-10.fc12

As you can see, it correctly removed the parent package, which could no-longer operate without the dependency.

This is such a great advancement over remove-leaves, where last time I tried to remove gnash-plugin (after testing it out, I didn’t want it) yum wanted to remove Firefox!! Fail.

Result: Tick!

Suggestion – clean up existing system
One thing I would like, is the ability to run a system wide cleanup using this code in yum, rather than other rpm orphan tools. Sort-of like an apt-get autoremove. Theoretically, if you’ve always removed any packages on your system with this new feature then you should be right, but it would be nice to sort-of do a clean up at any point (on any system) and get yum to remove any orphaned deps for you. Maybe this already exists.

If you want to clean up existing orphaned dependencies, then package-cleanup is your friend (from yum-utils).

Take a look at all the packages that are orphaned, and make sure it is sane.

package-cleanup --leaves

If you’re happy with the list, you can just remove them all (after you’ve enabled yum’s new autoremove feature, of course!):


yum erase $(package-cleanup --leaves)

You can instantly see the benefit of this new feature – on my machine this command also removes brand new orphaned dependencies, created by the orphans I’m removing! That’s 59 packages in total.

Without this yum feature, it would only remove 36 packages. You would need to run package-cleanup several times, until you’ve really removed all orphaned packages.

Similarly, this works for package-cleanup –orphans but it would be great it this was built into a single yum cleanup type function – one command to do it all.

Summary
From my initial testing, this works really well, although I’ll continue to test it and see how we go. Hopefully this will find its way into Fedora 12, 13, and 14 rather than just 15.

I’m so grateful for this work – it’s something I’ve been crying out for ever since I made the decision to stick with Fedora. Thank you, thank you, thank you!!

Yum gets autoremove dependency feature

Coming from a Gentoo/Debian background, one thing that has constantly bugged me on RPM based platforms like Fedora is the lack of decent, reliable dependency removal.

It seems so simple (and Debian has done it since the dawn of time) – if I install package x which pulls in dependencies y and z, then when I remove package x, I want to remove dependencies y and z, if they are not required by any other package.

Yes, there is the remove-leaves plugin for Yum and various RPM orphan checking tools, but in my experience, they are just not reliable.

So, I’m very happy to have discovered that Seth Vidal has merged orphaned dependency cleanup on removal into Yum. Hallelujah!

It’s in rawhide yum-3.2.28-13, and I’ll do some testing soon…

Encrypted DVDs and Fedora

The well known RPMFusion repository contains a lot of the useful software which Red Hat doesn’t ship by default (for licensing, patent or legal reasons). It is the amalgamation of several other repositories, including Dribble, Freshrpms, and Livna.

One helpful package that RPMFusion does not package however, is libdvdcss, the free software library which enables Linux to play encrypted DVDs.

Most users keep Livna around for this single purpose, but it is often offline (especially recently). So, here’s another solution – use ATrpms – another third party repository that includes lots of handy software.

The problem is, ATrpms has lots of other software which you might not want to update and which could conflict with the packages from RPMFusion. The solution? Tell yum to only include libdvdcss* packages from that repository. Easy.

To do this, simply add a new repo file (/etc/yum.repos.d/atrpms.repo) for Yum to configure it.
[atrpms]
name=Fedora Core $releasever - $basearch - ATrpms
baseurl=http://dl.atrpms.net/f$releasever-$basearch/atrpms/stable
gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms
gpgcheck=1
includepkgs=libdvdcss*

Now, just run the following to update the repo and install libdvdcss (you may wish to remove the old package first, if you have it).
sudo yum check-update
sudo yum install libdvdcss

That’s about it!

-c

Yum still on the menu?

Update: I’ve tried to post my results back to Seth’s thread but it won’t work, so I’ve emailed him instead.

In response to my article comparing Yum and Apt (at least I think it was my article, might have been someone else’s I guess), lead developer of Yum, Seth Vidal, wrote his own test script and performed some Yum benchmarks of his own.

He wrote:

Always a fun comparison. It’d be even more fun if any of the numbers seemed accurate.

His ran his test and concluded that Yum is “pretty good” and offers for others to run the test and post their results. So I did, on the same computer I used for the my article. I also compared the results to Ubuntu, as that’s really what my article was talking about :-)

So what did I find?

Continue reading ‘Yum still on the menu?’

Having Yum for Breakfast

This week I decided to write a comparative article between Yum and Apt (the package managers). Using Fedora 11 and Ubuntu 9.04, I performed various tests and benchmarked both the time and CPU usage they took. But why? Let me explain.

I really like the Fedora project. Really. I like their stance on proprietary drivers and codecs (and of course free software) and these days they seem to be pushing the technological envelope more than others. Sure Red Hat drives the direction of the project somewhat, but I don’t mind Red Hat either.

In fact, I wish I could use Fedora as my main distro! But every time I try it I just get so frustrated with Yum. Sure it’s better than up2date, but it’s so damn slow and annoying. That’s a problem for someone like me who manually updates his package database first thing every morning and checks to see what packages are available and updates the system by hand. Why do I do that? Cause I like to.

But every time I’ve tried to get into Fedora that damn package manager has stopped me. I get frustrated after a day or so. I think the longest I’ve had it on was 2 days before I switched.

Recently I installed Fedora 10 and 11 to see if there was any performance increase. Actually, to tell you the truth I was completely surprised by Yum’s agility and speed. The old Fedora I remember was not to be seen.. or so it felt like anyway.

Hence, I thought it might be good to run some tests to see.

Of course as the article points out, does any of this matter? Do we really need a fast a nimble package manager? Well for me it matters. It matters a great deal. For most users though they probably won’t care, as they just let the package manager do its thing in the background.

Still, it makes for some interesting thoughts. I think.

Yum, it’s starting to get tasty

Matt will attest the fact that I eagerly download each and every new Fedora release, hoping that this release will be one I could see myself using long term. Fedora 11 Leonidas is out and one of the marked improvements is the release of RPM 4.7.0, which has much better memory management. I wanted to see if this translates into performance gains in every day computing and have written an article for Linux Magazine with my findings.

-c