Linux on an Apple Xserve EFI only machine

We have a few of these Apple Xserve machines at work which weren’t doing much, so I thought I’d make better use of them. Naturally, this meant installing Linux on them.

These machines do not have a BIOS (or even any emulation), they use EFI and as such won’t boot the standard Linux install media. I knew that Fedora could boot EFI, so that’s where I started, with Leonidas (version 11). Unfortunately, the install media just wouldn’t work on this device, presumably as it has no BIOS emulation.

To cut a long story short, I had to learn how EFI works in order to get it booting and it wasn’t an easy thing to discover!

Booting EFI
In short, the Apple EFI boots and scans all available drives (including USB drives) for a GPT partitioned device which has a FAT formatted partition on it. Inside this partition it looks for an efi directory.

In that directory must be another from the list of approved vendor named directories, such as redhat or apple. The default is boot and this is what I used.

Inside that boot directory, EFI looks for a binary EFI file, which it can run. In my case, this is a GRUB2 EFI binary. The name of the file must match your architecture, so for me it is called bootia32.efi but if you’re running a 64bit system it should be bootx64.efi.
NOTE: Your Mac must have a 64bit EFI to load the 64bit EFI Linux installer (bootx64.efi)). Else, try choosing the 64bit options when using the 32bit EFI installer (bootia32.efi), but either way your system must have a 64bit CPU.

The GRUB2 EFI binary I am using looks for the standard GRUB config file, which is called grub.cfg. This must also be in the efi/boot/ directory of the USB device. The kernel and initramfs for the net installer should also be in that directory.

I have made a Debian Squeeze, Fedora Constantine and Ubuntu Karmic installer which you can download and extract onto your USB device (sha1sum). You will still need to partition and format your device, so more on how to do that later.

Partition USB Boot Device
So, once we understand how EFI works, next we need to get a bootable GRUB happening in order to install Linux.

Grab a USB memory stick and let’s prepare it. Please know what device it is you’re partitioning!! I’ll use sdz just for example purposes only.
Unmount it
su -c 'umount /dev/sdz1'

Partition and format it
Partition and format it
su -c 'parted -s /dev/sdz mklabel gpt mkpartfs EFI fat32 0% 100% toggle 1 boot'

It should now automatically mount. If not, unplug it and plug it in again, or mount it manually.

Prepare USB Boot Device
Next you can extract my tarball onto the device, ensuring that it maintains the efi/boot/ directory structure on it and not create an extra directories because of the archiver. If it’s not right, then re-arrange things and eject the device.

You should now be able to plug it into your Xserve and turn it on.

Hold down the Option key (Alt key on non-Apple keyboards) until you see the boot option. It should show “EFI” on the usb stick. Boot this and you should see the GRUB menu load. Choose the distro you want to install and away you go! See the “Installation” section below and skip the next few.

Manual preparation – Building GRUB2
Before we can compile GRUB, we need some dependencies. If you’re using Debian or Ubuntu, then run the following:
sudo apt-get build-dep grub2
sudo apt-get install libusb-dev

We need to check out GRUB from CVS and build it for EFI systems. Note, my Xserve is 32bit only, so I’m installing 32bit version. If you’re using 64bit then use a 64bit OS. You can cross compile, see the link above for info if you need to.

Check out source from CVS:
cvs -z3 \
-d:pserver:anonymous@cvs.savannah.gnu.org:/sources/grub \
co grub2
cd grub2

Or get source from Debian (which is more likely to compile):
apt-get source grub2
cd grub2*

Build grub2:
./configure --with-platform=efi
make

Now, make a temporary directory in which to put the GRUB EFI binary.
mkdir -p efi/boot/

Next we need to create the GRUB2 EFI program, and will specify a bunch of modules to include. This was a big stumbling block for me because someone removed the ability to read config files from module “configfile” and put it into “minicmd” requiring “sh”. The documentation I was following was not updated and I couldn’t work out why GRUB2 couldn’t parse my config (typing it in manually at the GRUB2 command prompt worked however). After lots of wasted time I emailed the GRUB list and had an answer within 20 minutes, which was a big relief.

./grub-mkimage -d . -o bootia32.efi part_gpt hfsplus fat ext2 normal sh chain boot configfile minicmd linux loadbios reboot appleldr halt search
sudo cp bootia32.efi efi/boot/

The efi/ directory now just needs to be copied onto your FAT USB stick. Don’t forget to make sure the directory structure is correct, as laid out above.

Now you just need to create a GRUB2 config file, which we’ll do below after getting the kernel and initramfs.

Manual preparation – Linux installers
I am using the netboot installers for each distro. This basically consists of a kernel, initramfs (which includes the installer). To know how to boot these in the GRUB config file, I read the isolinux configs included on the install CDs.

So, download the network installer of your choice in ISO format. Mount loop it and copy the kernel and initramfs onto the efi/boot/ directory on your USB disk.

For example (32bit), Ubuntu, Fedora, Debian Squeeze (note that Fedora does actually have an EFI boot disk which you can dd directly to a USB stick and boot, however this didn’t work on my Xserve but it might be useful to grab the kernel and initramfs from as it’s smaller).

Mount loop ISO:
mkdir /tmp/iso
sudo mount -o loop boot.iso /tmp/iso

Copy the kernel and initramfs from /tmp/iso onto your USB stick. You might need to look at the isolinux.cfg files under /tmp/iso in order to find out which is the kernel. On Debian and Ubuntu the kernel is called linux and the initramfs is called initrd.gz, both in the root directory.

GRUB Config File
Now that you have the GRUB2 EFI binary (bootia32.efi), kernel and initramfs on the USB stick, we need a config file for GRUB to read when it loads.

Here is an example for all four distros that I use in my provided tarball.
set timeout=10
set default=0
menuentry "Debian Squeeze" {
linux /efi/boot/linux-squeeze priority=low vga=normal video=efifb
initrd /efi/boot/initrd-squeeze.gz
}
menuentry "Fedora Leonidas" {
linux /efi/boot/vmlinuz-leonidas nomodeset xdriver=fbdev noselinux
initrd /efi/boot/initrd-leonidas.img
}
menuentry "Ubuntu Jaunty" {
linux /efi/boot/linux-jaunty priority=low vga=normal video=efifb
initrd /efi/boot/initrd-jaunty.gz
}
menuentry "Ubuntu Karmic" {
linux /efi/boot/linux-karmic priority=low vga=normal video=efifb
initrd /efi/boot/initrd-karmic.gz
}

Note that for Fedora I had to turn kernel based mode-setting off, and specify the Xserver driver.

The USB keyboard just wouldn’t work on either Ubuntu, but Debian Squeeze worked fine. I ended up choosing Fedora.

Manual preparation – Install Linux
Once you have all the required files, plug the stick in and turn on the Xserve. Hold down the Option key (Alt key on non-Apple keyboards) until you see the boot option. It should show “EFI” on the usb stick. Boot this and you should see the GRUB menu load where you can boot your network installer of choice.

Installation
When installing, you need to follow the same partition structure as the USB stick. I completely blew away all Apple partitions and am booting only Linux. The drives need to be GPT and to have a small FAT partition at the beginning with the efi/boot/ directory, GRUB2 EFI binary, GRUB config, kernel and initramfs.

There’s probably a better way to do this, I’m sure some distros will support GRUB2 and EFI out of the box soon. In the mean time I install the OS without a boot loader and create the efi/boot/ system manually. This means every time you update your kernel, you need to copy the kernel and initramfs, and update the GRUB config file.

The config file should be the same as the one used on the USB stick, except that the names of the kernel and initramfs will be different. For example:

set timeout=10
set default=0
menuentry "Fedora Leonidas" {
linux /efi/boot/vmlinuz-2.6.28-13-generic nomodeset xdriver=fbdev noselinux root=/dev/sda3 ro
initrd /efi/boot/initrd.img-2.6.28-13-generic
}

Conclusion
These are some rough notes to hopefully help others out there trying similar things. Please let me know how it works for you. I installed Fedora on my Xserve and it’s running quite well!

171 thoughts on “Linux on an Apple Xserve EFI only machine

  1. Maybe use your package manager to see if any packages provides */sh.mod and then install that? It’s been too long I’m afraid for me to remember. Other than that I’d be looking for bug reports in Debian.

    -c

  2. Well, i tried 12.04 ubuntu and 12.10 x64 on xserv which has EFI64 and using nomodeset in the kernel parameters ended up like JohnD, kernel loads, text appear however the moment it hits the ubuntu Anaconda installer for the graphics it gives unreadable display colors and text :(,
    I can see the mouse cursor moved but not readable! I think iam going to install it blindly like what JohnD did. And see where it gets me.

  3. Here is another video which I have tried ubuntu server 12.10 which should be console only and not graphical! Adding nomodeset boots it all the way except I cannot see anything but the system is intact and stable, what video any ideas?

    http://youtu.be/pmJRlc3DvMA

  4. Yes Chris, I have tried it and it did not boot.. I get to the grub menu then it doesn’t boot the kernel even with nomodeset.

    [updates] on the above installation of 12.10 server now after installing fglrx, I can boot to the kernel without a problem and adding random xorg.conf file and rebooting without nomodeset solved the problem and I can see Lightdm because I did install ubuntu-desktop, however ubuntu gnome loads only desktop without unity panels.

    installing xubuntu-desktop xfce loads well without a problem with menu and everything however the resolution is only 800×600! 🙁

  5. Hi Chris,

    Very helpful blog post here but I’m getting stuck at the very last step. My installed system refuses to boot.

    I started with your tarball and a USB stick, added wheezy64 entries and proceeded to install. All went well until time to install grub which didn’t seem to work so I skipped over it and completed the install.

    Now I can boot the system from the USB key by dropping to grub command line and passing set root, loading the kernel and initrd but I can’t figure out how to get grub on the internal disk.

    I’m working on an xserve 1.1 so it’s the typical 32bit EFI vs 64bit kernel mess.

    Any suggestions you can offer would be most appreciated!

  6. Hi Chris,

    Just an update. I was finally successful. Even managed a mirrored software raid.

    The Debian iso I found here: http://cdimage.debian.org/cdimage/unofficial/efi-development/upload4/

    The multiarch one in particular was a huge help. I was able to install 32bit on my old xserve then 64bit on a new PC with EFI. Then I just copied the entire root over top of the 32bit install.

    At this point the system would not boot but all I had to do was manually start my mdadm arrays. Then once up and running rebuild my initrd and now everything works like a charm.

    Thanks again for the very helpful post!

  7. Thanks for this great info.
    I have an xServe late 2006. I have installed rEFIT and can get multiple flavors to show up as a boot option, LinuxMint, Grub, and Ubuntu Natty.
    When I start the I get error screen
    Starting legacy loader
    Error: Not Found while loading legacy loader
    Please make sure you have the latest firmware update installed.

    If you could share any ideas that would be great.

  8. Kind of sounds like it’s trying to load a legacy (non-EFI) bootloader.

    These days distros like Fedora/CentOS support UEFI directly, maybe installing something like that (without even rEFInd) might be a possibility?

  9. Hi, is this an XServe1,1 that you tried this on? I’m trying to re-purpose one of these as a Linux server (with either Ubuntu Server 14.04.3, or CentOS 6.7). But I’m running into this EFI bootloader issue.

  10. I ran into a problem when running the parted command to partition and format the USB stick.

    ERROR:
    WARNING: you are attempting to use parted to operate on (mkpartfs) a file system.
    parted’s file system manipulation code is not as robust as what you’ll find in
    dedicated, file-system-specific packages like e2fsprogs. We recommend
    you use parted only to manipulate partition tables, whenever possible.
    Support for performing most operations on most types of file systems
    will be removed in an upcoming release.

    Then, it says I found a bug in GNU Parted, and that it crashed.

    I’m using Ubuntu MATE 14.04.2 to make the USB stick. Any suggestions?

  11. You probably just need to create the partition in parted and then run mkfs on the partition afterwards. What’s the command you ran?

  12. I did the exact line that you have in your “Partition and format it” section of the instructions.

    I haven’t tried to do anything with this in a couple of weeks. I’ve been hitting dead ends with all the incomplete/outdated procedures I’ve found on line.

  13. When I copied the contents of your “efi-grub-tar.bz2” and then booted my XServe with it, I just get the grub shell.

    I edited the .cfg file to give me the option of selecting CentOS 6.7 Live DVD.

    However, since it booted to the grub shell, I’m not sure what to do next.

  14. I’ve managed to install Gentoo Linux in a Xserve1,1 using initially your installer which I modified adding it the kernel, initrd and initial filesysten of ubuntu-13.04-server-amd64+mac.iso. I needed to put a CD with that distro un the DVD drive too in order to let the Ubuntu installer load the needed modules to detect the SATA controller.

    Once I had this, I opened a console and mounted the hard drive (previously partitioned in another machine), extracted the Gentoo base system and chrooted into it.

    Now I have a x86-64 Gentoo system but the MAIN PROBLEM is that it is impossible to make the Xserve1,1 boot using any modern x86-64 kernel. I’ve tried a lot of versions and many different options. Even I created a new GUB2 image (you are using a 1.9 GRUB image) in order to test if the problem is the bootloader, but there is no way for now. I can make Linux v3.10.95 to boot, that is the newest version of the kernel that can boot I’ve tried. I have enabled the option to make the kernel boot from a 32-bit EFI system in the kernel configuration, but doesn’t appear to work.

    Even booting with a 64bit kernel, there are problems with IPMI and some EFI features which can prevent the kernel from botting successfully.

    I have spent 2 weeks trying to prepare a modern (4.x) 64-bit kernel capable of booting into my Xserve1,1…

    So, since I can not find a way to do this, I’ve decided to install a 32-bit kernel and a 32-bit Gentoo system and wait for any new version of the kernel that can address this situation.

    If anyone have managed to boot a 4.x Linux kernel in a Xserve1,1 please, tell me because it would be greeat to know what is the problem.

  15. OK, maybe something’s changed upstream with 32bit EFI support. I don’t have any access to XServes anymore so I can’t help with that 🙁 Good luck!

  16. Hi Chris
    Thank you for your work on the reconditioning of the Xserves.
    I would like to get the Xserve 2.1 to work with linux on it.
    I switched to rEFIT 0.14.
    I used what you describe above on a USB flash drive.
    And I come to the choice of distributions you put in your image, efi-grub.tar.bz2.
    But once the selection is made, the screen stays on :
    Video line length: 5120
    [Initrd, addr=0x7b433000, size=0x5b80f9]

    Would you have, despite the time that has passed, an idea?
    Thank you a lot.

  17. Oh, it’s been a very long time and I don’t have any access to that hardware any more, but it sounds like it’s not booting properly, perhaps it can’t find the USB. Can you change the boot configuration to make sure you can see the kernel boot? Can you press ESC and see anything?

Leave a Reply

Your email address will not be published. Required fields are marked *