Firstly, the Mac Pro I am using has a 500GB SATA drive, so the partition sizes that I choose reflect that fact. Your machine might be different, so adjust accordingly.

Step 1
Install OS X on single partition using the whole drive (insert DVD and boot with C key held down).

Step 2
Use the diskutil terminal command to resize the OS X partition and create one for Linux and Windows.

Open “Terminal” and then run:
sudo diskutil resizeVolume disk0s2 200G "MS-DOS" "Linux" 200G "MS-DOS" "Windows" 5G
(Note that the final number for the Windows drive doesn’t really matter, it will fill all remaining available space.)

Step 3
Under OS X, install rEFIt (a graphical EFI boot loader) and then bless the drive:
sudo /efi/refit/enable-always.sh

Step 4
Insert Fedora 12 disk and reboot, holding down the C key (or boot it from the Linux CD entry in rEFIt menu).
(Note: I had to install via FTP, because installing from DVD didn’t always work on my Mac Pro. If you have the same issue then at the boot menu hit the “Tab” key and add “askmethod” to the kernel line and boot the entry.)

Step 5
During the installer, select the 3rd partition for / (choose “Custom” partition), format ext4.

Next, switch to terminal 2 (CTRL+Alt+2) and toggle boot to off for the Linux partition, like so:
parted /dev/sda
toggle 3 boot
quit

Now sync the GPT and MBR partition tables:
gptsync /dev/sda
(Or you can do this step via rEFIt menu after the install if you prefer.)

Switch back to terminal 6 (Alt+6) and continue. Install the boot loader to /dev/sda3 (this is the default).

Complete Fedora install, reboot into OS X, eject disk, insert Windows XP install CD.

For reference, the GPT partition table should look something like this:



[chris@localhost ~]$ sudo parted -l

Model: ATA ST3500630AS P (scsi)
Disk /dev/sda: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 20.5kB 210MB 210MB fat32 EFI System Partition boot
2 210MB 200GB 200GB hfs+ OS X
3 200GB 400GB 200GB ext4 Linux
4 400GB 500GB 99.9GB ntfs Windows

While the MBR table should look like this:



[chris@localhost ~]$ sudo fdisk -l

WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xcce7cce7

Device Boot Start End Blocks Id System
/dev/sda1 1 26 204819+ ee GPT
/dev/sda2 26 24325 195181428 af HFS / HFS+
/dev/sda3 * 24341 48656 195311616 83 Linux
/dev/sda4 48657 60802 97556480 7 HPFS/NTFS

Step 6
Reboot, holding down C key (or choose the Windows CD entry from rEFIt menu), install Windows XP to 4th partition, format NTFS.
If the Windows installer sees a single partition and can’t detect the file system, re-sync the GPT and MBR partition tables via rEFIt or Fedora gptsync command. Also, ensure that the Fedora partition is not set to be bootable (Step 5, part 2). The installer should see multiple drive letters, with the 4th partition being the C drive. Yes, Windows has to be installed on the last partition.

After install, the rEFIt menu should see Windows on the hard drive. Boot it and continue your Windows install. Once installed, insert the OS X Snow Leopard DVD and install the system drivers, etc.

That’s about it.

To fix this, toggle the boot flag to off using parted and then sync the GPT partition table to the master boot record partition table. It’s easiest to do this while the installer’s running (switch to TTY2) else you’ll have to install anaconda under your install to get gptsync (or run it from the rEFIt menu).

Simply run the following:
parted /dev/sda
toggle 3 boot
quit
gptsync /dev/sda

The gptsync command should spit out the “Current GPT partition table” which should show something like:
EFI System (FAT)
Mac OS X HFS+
Basic Data
Basic Data (if you have a fourth partition for Windows)

The “Proposed new MBR partition table” should show something like:
EFI Protective
Mac OS X HFS+
Linux
FAT32 (LBA) (or NTFS, depending if it's already been formated)

Now Windows should install cleanly on the fourth partition (if you’re doing that) and the partition structure under OS X should be correct.

For the life of me I can’t get Jaunty to boot on the machines. It installs just fine, but the initial reboot fails to find the root device and drops me to an “ash” shell which doesn’t ever actually appear. The keyboard also doesn’t work.

It doesn’t matter what RAID array I have, whether I’m using LVM or a standard partitioning scheme with an msdos partition table.. it just doesn’t work.

I’ve added aacraid and several other modules to the initramfs, still no joy.

Add the fact that the machine takes 10 minutes to boot each time I want to test a small change and it’s one super frustrating situation.

Oh, and 8.04 LTS works just fine.

One bug which appears to be a grub issue that I don’t have, which hasn’t been touched since April. There’s another about being unable to find the root device that also hasn’t received any love.

If anyone has some suggestions (install Debian?), let me know. The reason I’m using Ubuntu is because we have a local mirror and Jaunty because it’s a virtual machine and KVM is the way I want to go.

Update: Ahh, problem resolved and it was my fault. See comments..

These machines do not have a BIOS (or even any emulation), they use EFI and as such won’t boot the standard Linux install media. I knew that Fedora could boot EFI, so that’s where I started, with Leonidas (version 11). Unfortunately, the install media just wouldn’t work on this device, presumably as it has no BIOS emulation.

To cut a long story short, I had to learn how EFI works in order to get it booting and it wasn’t an easy thing to discover!

Booting EFI
In short, the Apple EFI boots and scans all available drives (including USB drives) for a GPT partitioned device which has a FAT formatted partition on it. Inside this partition it looks for an efi directory.

In that directory must be another from the list of approved vendor named directories, such as redhat or apple. The default is boot and this is what I used.

Inside that boot directory, EFI looks for a binary EFI file, which it can run. In my case, this is a GRUB2 EFI binary. The name of the file must match your architecture, so for me it is called bootia32.efi but if you’re running a 64bit system it should be bootx64.efi.
NOTE: Your Mac must have a 64bit EFI to load the 64bit EFI Linux installer (bootx64.efi)). Else, try choosing the 64bit options when using the 32bit EFI installer (bootia32.efi), but either way your system must have a 64bit CPU.

The GRUB2 EFI binary I am using looks for the standard GRUB config file, which is called grub.cfg. This must also be in the efi/boot/ directory of the USB device. The kernel and initramfs for the net installer should also be in that directory.

I have made a Debian Squeeze, Fedora Constantine and Ubuntu Karmic installer which you can download and extract onto your USB device (sha1sum). You will still need to partition and format your device, so more on how to do that later.

Partition USB Boot Device
So, once we understand how EFI works, next we need to get a bootable GRUB happening in order to install Linux.

Grab a USB memory stick and let’s prepare it. Please know what device it is you’re partitioning!! I’ll use sdz just for example purposes only.
Unmount it
su -c 'umount /dev/sdz1'

Partition and format it
Partition and format it
su -c 'parted -s /dev/sdz mklabel gpt mkpartfs EFI fat32 0% 100% toggle 1 boot'

It should now automatically mount. If not, unplug it and plug it in again, or mount it manually.

Prepare USB Boot Device
Next you can extract my tarball onto the device, ensuring that it maintains the efi/boot/ directory structure on it and not create an extra directories because of the archiver. If it’s not right, then re-arrange things and eject the device.

You should now be able to plug it into your Xserve and turn it on.

Hold down the Option key (Alt key on non-Apple keyboards) until you see the boot option. It should show “EFI” on the usb stick. Boot this and you should see the GRUB menu load. Choose the distro you want to install and away you go! See the “Installation” section below and skip the next few.

Manual preparation – Building GRUB2
Before we can compile GRUB, we need some dependencies. If you’re using Debian or Ubuntu, then run the following:
sudo apt-get build-dep grub2
sudo apt-get install libusb-dev

We need to check out GRUB from CVS and build it for EFI systems. Note, my Xserve is 32bit only, so I’m installing 32bit version. If you’re using 64bit then use a 64bit OS. You can cross compile, see the link above for info if you need to.

Check out source from CVS:
cvs -z3 \
-d:pserver:anonymous@cvs.savannah.gnu.org:/sources/grub \
co grub2
cd grub2

Or get source from Debian (which is more likely to compile):
apt-get source grub2
cd grub2*


Build grub2:
./configure --with-platform=efi
make

Now, make a temporary directory in which to put the GRUB EFI binary.
mkdir -p efi/boot/

Next we need to create the GRUB2 EFI program, and will specify a bunch of modules to include. This was a big stumbling block for me because someone removed the ability to read config files from module “configfile” and put it into “minicmd” requiring “sh”. The documentation I was following was not updated and I couldn’t work out why GRUB2 couldn’t parse my config (typing it in manually at the GRUB2 command prompt worked however). After lots of wasted time I emailed the GRUB list and had an answer within 20 minutes, which was a big relief.

./grub-mkimage -d . -o bootia32.efi part_gpt hfsplus fat ext2 normal sh chain boot configfile minicmd linux loadbios reboot appleldr halt search
sudo cp bootia32.efi efi/boot/

The efi/ directory now just needs to be copied onto your FAT USB stick. Don’t forget to make sure the directory structure is correct, as laid out above.

Now you just need to create a GRUB2 config file, which we’ll do below after getting the kernel and initramfs.

Manual preparation – Linux installers
I am using the netboot installers for each distro. This basically consists of a kernel, initramfs (which includes the installer). To know how to boot these in the GRUB config file, I read the isolinux configs included on the install CDs.

So, download the network installer of your choice in ISO format. Mount loop it and copy the kernel and initramfs onto the efi/boot/ directory on your USB disk.

For example (32bit), Ubuntu, Fedora, Debian Squeeze (note that Fedora does actually have an EFI boot disk which you can dd directly to a USB stick and boot, however this didn’t work on my Xserve but it might be useful to grab the kernel and initramfs from as it’s smaller).

Mount loop ISO:
mkdir /tmp/iso
sudo mount -o loop boot.iso /tmp/iso

Copy the kernel and initramfs from /tmp/iso onto your USB stick. You might need to look at the isolinux.cfg files under /tmp/iso in order to find out which is the kernel. On Debian and Ubuntu the kernel is called linux and the initramfs is called initrd.gz, both in the root directory.

GRUB Config File
Now that you have the GRUB2 EFI binary (bootia32.efi), kernel and initramfs on the USB stick, we need a config file for GRUB to read when it loads.

Here is an example for all four distros that I use in my provided tarball.
set timeout=10
set default=0
menuentry "Debian Squeeze" {
linux /efi/boot/linux-squeeze priority=low vga=normal video=efifb
initrd /efi/boot/initrd-squeeze.gz
}
menuentry "Fedora Leonidas" {
linux /efi/boot/vmlinuz-leonidas nomodeset xdriver=fbdev noselinux
initrd /efi/boot/initrd-leonidas.img
}
menuentry "Ubuntu Jaunty" {
linux /efi/boot/linux-jaunty priority=low vga=normal video=efifb
initrd /efi/boot/initrd-jaunty.gz
}
menuentry "Ubuntu Karmic" {
linux /efi/boot/linux-karmic priority=low vga=normal video=efifb
initrd /efi/boot/initrd-karmic.gz
}

Note that for Fedora I had to turn kernel based mode-setting off, and specify the Xserver driver.

The USB keyboard just wouldn’t work on either Ubuntu, but Debian Squeeze worked fine. I ended up choosing Fedora.

Manual preparation – Install Linux
Once you have all the required files, plug the stick in and turn on the Xserve. Hold down the Option key (Alt key on non-Apple keyboards) until you see the boot option. It should show “EFI” on the usb stick. Boot this and you should see the GRUB menu load where you can boot your network installer of choice.

Installation
When installing, you need to follow the same partition structure as the USB stick. I completely blew away all Apple partitions and am booting only Linux. The drives need to be GPT and to have a small FAT partition at the beginning with the efi/boot/ directory, GRUB2 EFI binary, GRUB config, kernel and initramfs.

There’s probably a better way to do this, I’m sure some distros will support GRUB2 and EFI out of the box soon. In the mean time I install the OS without a boot loader and create the efi/boot/ system manually. This means every time you update your kernel, you need to copy the kernel and initramfs, and update the GRUB config file.

The config file should be the same as the one used on the USB stick, except that the names of the kernel and initramfs will be different. For example:

set timeout=10
set default=0
menuentry "Fedora Leonidas" {
linux /efi/boot/vmlinuz-2.6.28-13-generic nomodeset xdriver=fbdev noselinux root=/dev/sda3 ro
initrd /efi/boot/initrd.img-2.6.28-13-generic
}

Conclusion
These are some rough notes to hopefully help others out there trying similar things. Please let me know how it works for you. I installed Fedora on my Xserve and it’s running quite well!

Ever wondered what was causing that big delay in bootup? Install bootchart and check!

I installed bootchart on my openSUSE box at work with sudo zypper in bootchart, rebooted and edited grub on the fly, adding init=/sbin/bootchartd to the kernel line.

After your system has loaded, bootchart creates a log of the process which you can then turn into a chart with the command bootchart. You can also specify the log location if required, or change the output format of the chart from PNG to SVG or EPS.

Of course, what you do with this information is up to you Happy hacking.