Installing XCP-NG on Apple Hardware

XCP-NG is an open source hypervisor platform; an operating system and tools to run virtual machines on a pool of computers, with a powerful web-based management interface. The hypervisor is based on Xen, with extensions to allow it to be managed by a CLI or the XenOrchestra web UI. If you’re familiar with VMware ESXi you’ll pick up XCP-NG very quickly.

XCP-NG will run on most x86 hardware, including those older Intel Apple computers you may have lying about that can be repurposed to set up a home lab or similar.

However, getting XCP-NG (either of the two long term supported versions 8.2.1 or the newer 8.3) installed and running may cause you problems, as the installer will always finish with what appears to be a fatal error. Typically it’s something like:

An unrecoverable error has occurred. The error was:Failed to run efibootmgr: Could not prepare Boot variable: No such file or directory

Not encouraging, right? But don’t worry, we can fix this. There are four steps to recover and get everything running the way it should be. But some serious things have failed that could leave you scratching your head: that root password you set during installation? Gone. The storage space for VMs you configured? Not there. Timezone and keyboard preferences? Forgotten.

First, shut down the computer and disconnect the installation USB. You won’t need it again, and having an extra disk attached might cause confusion. Then, switch on the computer and get ready with your finger hovering over the “e” key on your keyboard. As soon as the Grub loader menu appears, hit “e” and start using the cursor keys to move through the boot argument string. Look for “ro”; change that to “rw init=/sysroot/bin/sh”. What does that do? Instead of booting with the root partition in read-only mode, we’re going to mount it read-write (so we can make changes) and instead of running init to start the usual boot sequence, we’re just going to drop straight into a command shell. Hit C-x to save and continue booting.

When you get to the root shell prompt “#”, type the following commands:

chroot /sysrootpasswd

The first changes the environment for later commands to use the /sysroot jail, so that commands, shared libraries etc. can all be found. If you get strange errors that libc.so can’t be found, you forgot this step. The second lets us set the root password. Enter it twice, make a note of it and remember this process if you ever forget the root password again and need to reset it.

The usual commands to reboot won’t work as they talk to init, but there’s no init process running, just our shell. You might find Ctrl-Alt-Del lets you reboot; I usually resort to the power switch at this point. Turn it off and back on again. This time, let it boot up normally until you see the console screen:

Use the arrow keys, Enter and Escape to navigate the menus. Check the Network and Management Interface details are correct, if they are, you can probably ssh in if that’s more convenient. If you need to make changes you’ll be prompted for the root password you just set.

Next go to Keyboard and Timezone and pick the correct keyboard layout and timezone from the drop down lists.

The final problem is the trickiest bit. The system needs at least one SR (storage repository) to store VMs in. You probably remember during the installation phase being asked if you wanted it for thin provisioning (ext) or thick provisioning1. Without that, you’re a bit stuck, as you can’t set up any VMs, and the best way to manage XCP-NG is through XenOrchestra (usually XOA – the XenOrchestra Appliance), which you’d normally provision as the first VM on your server. You can create the missing SR(s) from the command line, by following the XCP-NG documentation. I did that once, years ago, and I didn’t take any notes. It’s not hard, just tedious and you’ll probably want to ssh in from another computer so you can copy and paste all the long UUID strings.

XenOrchestra is often deployed as the XOA appliance, but it’s just a web application and there’s a handy script to download the community edition sources and build it here: https://github.com/Jarli01/xenorchestra_installer

You’ll need a pre-existing x86 Debian/Ubuntu machine somewhere. If you don’t have one of those already, but you’ve got a new Apple Silicon Mac, you can install UTM and set up a Debian VM there. Just download a Debian or Ubuntu image from the Gallery, then follow the instructions for building XenOrchestra Community Edition above. I don’t think it works reliably on ARM/Apple Silicon yet, so you’ll need an Intel VM, hence using UTM.

One of the great things about XO is you can have multiple installations all talking to the same pool of hypervisor hosts, and can manage them from anywhere. It’s always useful to have a spare XO somewhere to let you manage your pool, so even though this XenOrchestra installation is a temporary bootstrapping step, I’d recommend keeping it around in case you ever need it again.

Log in to your new XenOrchestra web application, using the IP address of the computer you installed it on. The default login is “admin@admin.net” and the password is “admin”, you’ll want to change both of those at some point, but don’t worry about it just now.

Look down the list of options down the left hand side until you find “New”, and choose “Server” from the menu. Fill in the boxes – Label is the name you gave your server, but it can be anything meaningful. The address is the IP address of your server, username is “root” and the password is the root password you set above. Hit Connect, then approve the connection with the self-signed certificate. You can now manage your XCP-NG server easily.

Next go back to New and this time choose Storage. From here we are going to create the missing SR(s). Choose your Host from the pop-up list at the top, give your SR a name (eg “Primary” or literally anything meaningful – don’t worry too much about names for things in XCP-NG, they’re only for your benefit, underneath the system uses UUIDs, so you can rename everything at any time). Give the SR a description, again this is just extra information for you, so “Storage for VMs” will do. Choose the storage type; I recommend EXT for now, you can experiment with the other types later, then put in the device name to use.

If your Mac had only one internal drive the device is very likely “/dev/sda3”, but you should probably check. Log in to your server, either on the console or via ssh and use parted to check the partitions. One of them should be a very large unused one that uses up the rest of the disk. The installer usually leaves that as partition 3. If your Mac had multiple drives you’ll need to check whether you installed on sda or sdb. If your Mac had a Fusion drive (with a combined HDD and SSD) and you installed on the smaller, faster SSD, it’s probably sdb.

Click the Create button, and wait a minute. If you had a second internal drive, go back and repeat the process above for that other drive. You can just use the whole drive so “/dev/sda” or “/dev/sdb” will work.

You’ve now got a fully functional XCP-NG installation. Maybe go to Home / Pools / Patches and install any missing patches, followed by a restart before you start setting up VMs, then just follow Vates’ documentation to install an XOA appliance on your server and start setting up other VMs, access to any NFS or SMB shares you’ll be using to store installation ISOs or backups and set up your backup policy.

1. With Thin Provisioning, VMs take up as much space on disk as they are actually using, so virtual hard disks will grow as they fill up. With Thick Provisioning all the space allocated for a virtual hard disk is reserved in one go. Thin provisioning lets you pack more VMs on and makes them way more efficient to copy and move around. Thick provisioning stops VMs from growing and filling up space unexpectedly and is better suited to remote virtual disk storage solutions such as iSCSI, where you’re probably using a SAN. ↩︎