This project builds operating system images that can be used for bare metal server deployments with metal-stack. Every OS image is built from a Dockerfile, exported to a lz4 compressed tarball, and uploaded to https://images.metal-stack.io/.

More information about the image store is available in IMAGE_STORE.md.

Information about our initial architectural decisions can be found in ARCHITECTURE.md.

Currently these images are supported:

1. Debian 12
2. Ubuntu 24.04
3. Firewall 3.0-ubuntu (based on Ubuntu 24.04)
4. Nvidia (based on Debian 12)

We also publish images that we need for special purposes but do not officially support. Use at your own risk.

1. Almalinux 9

With the nvidia image a worker has GPU support. Please check our official documentation on docs.metal-stack.io on how to get this running on Kubernetes.

Images are synchronized to partitions using a service called metal-image-cache-sync. The service mirrors the public operating system images to the management servers and transparently serves the metal images within a partition.

Released images are tagged with the release date and can be accessed using the following image URL pattern:

https://images.metal-stack.io/metal-os/20240913/debian/12/img.tar.lz4

Images built from the master branch are accessible at an image URL like this:

https://images.metal-stack.io/metal-os/stable/debian/12/img.tar.lz4

For other branches, the URL pattern is as follows:

https://images.metal-stack.io/metal-os/pull_requests/${CI_COMMIT_REF_SLUG}/debian/12/img.tar.lz4

These URLs can be used to define an image at the metal-api.

Please also refer to our documentation on docs.metal-stack.io on Build Your Own Images to check for the contract an OS image is expected to fulfill.

Before you can start developing changes for metal-images or even introduce new operating systems, you should install the following tools:

• docker: for sure
• golang
• kvm: hypervisor used for integration tests
• lz4: to compress tarballs
• enable docker's containerd image store
• cloud-hypervisor: virtual machine monitor running on top of KVM to spin up MicroVMs for integration tests

You can build metal-images like that:

# for debian images
make debian

# for ubuntu images
make ubuntu

# for firewall images
make firewall

# for nvidia images
make nvidia

# for almalinux images
make almalinux

IMPORTANT if you prefer the old docker build output instead of the fancy buildx srolling behind output, do the following:

BUILDKIT_PROGRESS=plain make debian

For integration testing the images are started as cloud-hypervisor VMs and basic properties like interfaces to other metal-stack components, kernel parameters, internet accessibility, DNS resolution etc. are checked with goss in a GitHub action workflow. Integration tests are also executed if you build an image locally.

In some cases it may be necessary to manually figure out the commands for provisioning a machine image. To do this in a real server environment, it is possible to hook into the metal-hammer through the machine's serial console.

You can interrupt the metal-hammer at any time by sending a keyboard interrupt. The metal-hammer takes a short break before booting into the operating system kernel, which is a good time to send the interrupt.

To prevent the machine from rebooting, you should immediately issue the following command:

while true; do echo "1" > /dev/watchdog && sleep 55; done &

If you want to enter the operating system through chroot, you need to remount some file systems that were mounted by the metal-hammer during provisioning:

# the mount points also depend on the file system layout of the machine, so please only take this as an example:
mount /dev/sda2 /rootfs
mount -t vfat /dev/sda1 /rootfs/boot/efi
mount -t proc /proc /rootfs/proc
mount -t sysfs /sys /rootfs/sys
mount -t efivarfs /sys/firmware/efi/efivars /rootfs/sys/firmware/efi/efivars
mount -t devtmpfs /dev /rootfs/dev

Finally, you can then enter the provisioned OS image.

chroot /rootfs

# maybe you can mount further file systems here, which was not possible in the u-root environment of the metal-hammer
vgchange -ay
mount /dev/csi-lvm/varlib /var/lib/

Keep in mind that you are still running on the metal-hammer kernel, which is different from the kernel that will be run in the operating system after provisioning. For further information on the metal-stack machine provisioning sequence, check out documentation on docs.metal-stack.io. The kernel used by the metal-hammer is built on our own inside the kernel repository.