Skip to content

Latest commit

 

History

History
 
 

ic-os

README.adoc

IC-OS

Introduction

IC-OS is an umbrella term for all the operating systems within the IC, including SetupOS, HostOS, GuestOS, and Boundary-guestOS.

  • SetupOS: Responsible for booting a new replica node and installing HostOS and GuestOS.

  • HostOS: The operating system that runs on the host machine. Its main responsibility is to launch and run the GuestOS in a virtual machine. In terms of its capabilities, it is intentionally limited by design to not perform any trusted capabilities.

  • GuestOS: The operating system that runs inside a virtual machine on the HostOS. The core IC protocol is executed within the GuestOS.

  • Boundary-guestOS: The operating system that runs on boundary nodes.

Building IC-OS images

All the IC-OS images can be built though Bazel.

Environment setup

Building IC-OS images locally requires environment configuration. The required packages are found in ic/gitlab-ci/container/Dockerfile.

In addition to these packages, Bazel must be installed.

As an alternative, the following script can be used to build the images in a container with the correct environment already configured:

./gitlab-ci/container/container-run.sh

Build targets

Each image has its own build targets, which are variations of the image:

  • SetupOS: prod, dev, dev-sev

  • HostOS: prod, dev, dev-sev

  • GuestOS: prod, dev, dev-malicious, dev-sev

  • BoundaryGuestOS: prod, prod-sev, dev, dev-sev

    • Note that the dev and dev-sev images use the local service worker, while the prod and prod-dev images pull the service worker from npm.

The difference between production and development images is that the console can be accessed on dev images, but not on prod images.

Note: The username and password for all IC-OS dev images are set to root

Building images

Use the following command to build images:

$ bazel build //ic-os/{setupos,hostos,guestos,boundary-guestos}/envs/<TARGET>/...

All IC-OS image build outputs are stored under /ic/bazel-bin/ic-os/{setupos,hostos,guestos,boundary-guestos}/envs/<TARGET>

Example:

$ bazel build //ic-os/guestos/envs/dev/...
# This will output a GuestOS image in /ic/bazel-bin/ic-os/guestos/envs/dev

Under the hood: Building an image

IC-OS images are first created as docker images and then transformed into "bare-metal" or "virtual-metal" images that can be used outside containerization.

Rather than installing and relying on a full-blown upstream ISO image, the system is assembled based on a minimal Docker image with the required components added. This approach allows for a minimal, controlled, and well understood system - which is key for a secure platform.

The build process is as follows:

Docker

The docker build process is split into two dockerfiles. This split is necessary to ensure a reproducible build.

Dockerfile.base

ic/ic-os/{setupos,hostos,guestos,boundary-guestos}/rootfs/Dockerfile.base

  • The Dockerfile.base takes care of installing all upstream Ubuntu packages.

  • Because the versions of these packages can change at any given time (as updates are published regularly), in order to maintain build determinism, once a week, the CI pipeline builds a new base image for each OS. The result is published on the DFINITY public Docker Hub.

Dockerfile

ic/ic-os/{setupos,hostos,guestos,boundary-guestos}/rootfs/Dockerfile

  • The Dockerfile builds off the published base image and takes care of configuring and assembling the main disk-image.

  • Any instruction in this file needs to be reproducible in itself.

Image Transformation

The docker image is then transformed into a bootable "bare-metal" or "virtual-metal" VM image for use outside containerization (either in a VM or as a physical host operating system). The resulting image is minimal, with only a few systemd services running.

Note that all pre-configuration of the system is performed using docker utilities, and the system is actually also operational as a docker container. This means that some development and testing could be done on the docker image itself, but an actual VM image is still required for proper testing.

Adding a new dependency to an ICOS image

To add a new package to an ICOS image you need to:

  • Update the list of packages to install in ic/ic-os/{setupos,hostos,guestos,boundary-guestos}/rootfs/packages.common

    • Commit the changes and wait for CI to publish the base image

  • Update the base image hash in ic/ic-os/{setupos,hostos,guestos,boundary-guestos}/rootfs/docker-base.<env>

IC-OS Directory Organization

  • bootloader/: This directory contains everything related to building EFI firmware and the GRUB bootloader image. It is configured to support the A/B partition split on upgradable IC-OS images (HostOS, GuestOS, and potentially Boundary-guestOS)

  • scripts/: This directory contains build scripts.

    • Note that GuestOS has its own scripts subdirectory that still need to be unified with the outer scripts directory.

  • rootfs/: Each rootfs subdirectory contains everything required to build a bootable Ubuntu system. Various template directories (e.g., /opt) are used, which are copied verbatim to the target system. You can add files to these directories to include them in the image.

SEV testing

Preparing DEV machine

Follow instructions here to prepare the dev machine.

Storing the SEV Certificates on the host (e.g. for test/farm machines)

Note: we are storing the PEM files instead of the DER files.

% ic/ic-os/hostos/rootfs/opt/ic/bin/get-sev-certs.sh
% sev-host-set-cert-chain  -r ark.pem -s ask.pem -v vcek.pem

Running SEV-SNP VM with virsh

Preparing image

  • cd to the root of the source tree

  • build the image: bazel build //ic-os/boundary-guestos/envs/dev-sev/…​

  • ic-os/scripts/bn-virsh/prepare-for-virsh.sh

Create, login, destroy

  • $ virsh create ./bn_sev_vm.xml

  • $ virsh console boundary_nodes_sev_snp-$USER

    • Note: control-] to exit

  • $ virsh destroy boundary_nodes_sev_snp-$USER