Prologue: Computers Were A Mistake

I guess it was only a matter of time until a Youtuber showed me something I wanted to buy.

I'd been thinking about building a small homelab since late last year with the goal of having a local Git repository for my open-source code, and also having a way to try out fun-looking services without paying for a VPS somewhere. [1] I was watching a review series on small business computers when I found the HP EliteDesk 805 G6 Mini and it met all my criteria: decent energy consumption, good performance, support for NICs faster than gigabit - and most importantly not powered by Intel to try and induce some competition. [2] After some research I discovered the most recent version, the 805 G8, wasn't much more expensive and I told myself the additional performance of the newer processor would help extend its lifespan; one inflated eBay auction and some upgraded RAM later I had one in my hands and my only barrier from here was my imagination. Or so I thought.

The first order of business was to determine which hypervisor I would use to run everything: ESXi was immediately dismissed because I didn't feel like dealing with licensing headaches, this also eliminated anything Microsoft had to offer as well; Proxmox is the default answer for homelab virtualization but I thought the concept of paravirtualization sounded super cool and I wanted to know how noticable its performance benefits were in the real world, so I decided to give XCP-NG a spin instead. [3]

This is when I realized the installer wouldn't run on my machine - honestly I should have given up and installed Proxmox then and there, but instead I spent five months futilely waiting for an updated LTS ISO or for the next non-LTS release. The LTS installer failed to run on my machine due to some setting in my BIOS I couldn't change, but from what I read on the forums the issue was limited to the installer - if I could get the software installed it would run just fine. To add insult to injury the alpha installer for the next release worked just fine - but I'm not going to run my homelab on an alpha release.

I don't know what posessed me to look at the installation documentation one last time, but when I noticed the instructions for creating a custom install ISO I had a new plan of attack: replace the part from the LTS installer that didn't work on AMD with the part from the alpha installer that did, which should at least allow me to find new rakes to step on further along in the process. I doubt there are many other people out there who are in this incredibly specific situation, but if you are now you don't have to waste five months waiting for nothing as well. [4]

Part 1: Installing XCP-NG

You can create a working 8.2.1 ISO by overwriting the installer kernel with one from the 8.3.0 alpha ISO; I overwrote the 8.2.1 alt installer kernel so I could have a "pure" 8.2.1 install on some other hardware if I wanted. The process is fairly straightforward:

mkdir /tmp/xcp
udisksctl loop-setup -f xcp-ng-8.2.1.iso
udisksctl loop-setup -f xcp-ng-8.3.0-alpha2.iso
cp -r /media/mischif/XCP-ng\ 8.2.1/. /tmp/xcp/
chmod -R u+w /tmp/xcp
cp /media/mischif/XCP-ng\ 8.3.0/boot/vmlinuz /tmp/xcp/boot/alt/
umount /media/mischif/XCP-ng\ 8.3.0/
umount /media/mischif/XCP-ng\ 8.2.1/
genisoimage -o /tmp/xcp-8.2.1-mod.iso -v -r -J --joliet-long -V "XCP-ng 8.2.1-mod" -c boot/isolinux/boot.cat -b boot/isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e boot/efiboot.img -no-emul-boot /tmp/xcp
isohybrid --uefi /tmp/xcp.iso

Once all this is done you can write the ISO onto a flash drive and complete the installation process like normal. The installation process will install the old kernel so you'll have a blank screen on reboot, but you should be able to SSH in and perform initial housekeeping tasks: adding users and keys, updating packages, etc. Additionally, you should yum install socat for use later in this guide.

If, like me, you have the 2.5G add-in NIC you have to install its drivers separately:

yum install igc-module

After rebooting inform XCP-NG of the new interface:

xe pif-scan host-uuid=$(xe host-list params=uuid | awk '{print $5}')

Part 2: Creating a VM

At this point the system is functional, so it's time to make it useful. The official way to control an XCP-NG server is with Xen Orchestra; if you have an instance running somewhere already you can point it at the new server and you're good to go, otherwise you'll want to create one now. Navigating to the server's IP address will show you how to install the basic version of the software, but installing from source grants all the premium features for free; I needed Docker support and I definitely wasn't going to pay $85/mo for it so I had to figure out how to build it myself.

But before you build XOA you have to create the VM it will run inside - and as a nota bene, before you start you'll want to SSH into the XCP-NG server with port forwarding enabled (e.g. ssh -L 5900:vmhost:9000 vmhost).

First you need the UUIDs relating to your server and its storage/network: [5]

HOST_UUID=$(xe host-list params=uuid | awk '{print $5}')
LOCAL_SR_UUID=$(xe sr-list params=uuid name-label="Local storage" | awk '{print $5}')
NET_UUID=$(xe network-list params=uuid bridge=xenbr0 | awk '{print $5}')

Next, you need a place to store the ISO you'll use to create the VM:

mkdir /var/isos
ISO_SR_UUID=$(xe sr-create host-uuid=$HOST_UUID name-label=ISOs type=iso device-config:location=/var/isos device-config:legacy_mode=true content-type=iso)
wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-virt-3.18.0-x86_64.iso
mv alpine-virt-3.18.0-x86_64.iso /var/isos
xe sr-scan uuid=$ISO_SR_UUID

Next you create the VM, adding storage and configuring networking:

VM_UUID=$(xe vm-install template="Other install media" new-name-label="XOA-CE")
xe vm-memory-limits-set uuid=$VM_UUID dynamic-max=2048MiB dynamic-min=2048MiB static-max=2048MiB static-min=512MiB
xe vm-cd-add uuid=$VM_UUID cd-name=alpine-virt-3.18.0-x86_64.iso device=1
xe vm-param-set uuid=$VM_UUID HVM-boot-policy="BIOS order"
xe vm-disk-add vm=$VM_UUID disk-size=10GiB sr-uuid=$LOCAL_SR_UUID device=0
VIF_UUID=$(xe vif-create vm-uuid=$VM_UUID network-uuid=$NET_UUID device=2)

After creating the VM's NIC, you can get the MAC address to set up a static DHCP lease if you want:

xe vif-param-get param-name=MAC uuid=$VIF_UUID

You could also configure the VM to automatically boot on startup:

POOL_UUID=$(xe pool-list params=uuid | awk '{print $5}')
xe pool-param-set uuid=$POOL_UUID other-config:auto_poweron=true
xe vm-param-set uuid=$VM_UUID other-config:auto_poweron=true

At this point, you're free to start the VM:

xe vm-start uuid=$VM_UUID

Part 3: Building XOA-CE

Now that the base VM is running, you can build XOA. Unfortunately XCP-NG only exposes a console over VNC so you have to use that to configure SSH access. First you expose the VNC socket to connect to:

socat TCP-LISTEN:9000 UNIX-CONNECT:/var/run/xen/vnc-$(xe vm-list params=dom-id uuid=$VM_UUID | awk '{print $5}')

Now you should be able to VNC to localhost on your computer and see the Alpine console. From here the installation process is straightforward, but I couldn't figure out how to paste in VNC so I couldn't enter my SSH key. No matter; you can enter short passwords for the root/XOA service user now and change them later.

After rebooting you can log in as your service user [6], add your SSH key, su to root and begin housekeeping:

sed -i '/community/s/^#//' /etc/apk/repositories
apk update
apk upgrade
apk add xe-guest-utilities
rc-update add xe-guest-utilities default
rc-service xe-guest-utilities start

Once the housekeeping is finished, you can download the XOA dependencies and prepare to build the software:

apk add build-base libpng-dev -t xoa-build
apk add git redis redis-openrc python3 fuse nodejs yarn
rc-update add redis default
rc-service redis start
mkdir /opt/xoa
chown xoa:xoa /opt/xoa/

Now would also be a good time to create the service file that will start the service when the VM boots. Write this service file to /etc/init.d/xoa:

#!/sbin/openrc-run

name="Xen Orchestra CE"
command_background=true
command="/usr/bin/yarn"
command_args="--cwd /opt/xoa/packages/xo-server run start"
pidfile="/run/${RC_SVCNAME}.pid"

depend() {
        need redis
}

Then set it to start automatically:

chmod +x /etc/init.d/xoa
rc-update add xoa default

At this point you can change back to the service user and complete the installation:

git clone --depth 1 https://github.com/vatesfr/xen-orchestra.git /opt/xoa
yarn --cwd /opt/xoa/
yarn --cwd /opt/xoa/ build
mkdir -p /home/xoa/.config/xo-server
cp /opt/xoa/packages/xo-server/sample.config.toml /home/xoa/.config/xo-server/config.toml
sed -i "s/#user = 'nobody'/user = 'xoa'/" /home/xoa/.config/xo-server/config.toml
sed -i "s/# group = 'nogroup'/group = 'xoa'/" /home/xoa/.config/xo-server/config.toml
sed -i "/\[http.mounts\]/a '/' = '/opt/xoa/packages/xo-web/dist'" /home/xoa/.config/xo-server/config.toml

Now you should be able to give the service a test run:

doas yarn --cwd /opt/xoa/packages/xo-server start

Log in with the default credentials and connect to the XCP-NG server; if nothing catches fire on the console you're good to go. Change back to root to start XOA as a service and clean up some no-longer-useful packages:

rc-service xoa start
apk del xoa-build

And that's it! XCP-NG 8.2.1 is now running on your HP EliteDesk 805 G8 Mini, managed by an instance of Xen Orchestra CE. Now go forth and virtualize.