Skip to content
ulm0Public

About

lima-compatible image build with linuxkit

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

17 Commits

Folders and files

Repository files navigation

limakit

A LinuxKit-built guest image for Lima, with Docker as the baseline. It targets Apple Silicon (vmType: vz) and supports Lima's full (non-plain) mode: guest agent, host mounts, dynamic port forwarding, provisioning scripts and so on.

Requirements

  • macOS on Apple Silicon, Lima ≥ 2.0 (tested with 2.2.1)
  • A Docker daemon to build with (any context; the kernel build wants ≥ 6 CPUs / 8 GiB)
  • curl, make. The pinned linuxkit CLI (v1.8.2) is downloaded to bin/ on first use.

Quick start

From a release, no build needed (once the first release is out): lima.yaml points at the latest release's kernel and squashfs, and each release also ships a lima.yaml pinned to its own files and digests:

limactl start --name limakit lima.yaml   # or the lima.yaml attached to a release

Each instance keeps its data on its own disk, so you can run several (--name work, --disk 50).

To build it yourself:

make builds the lima image and pulls the kernel limakit.yml pins from GHCR.

Then build the image and start it:

make run      # builds out/limakit-{kernel,squashfs.img}, (re)creates the `limakit` instance, starts it
make shell    # limactl shell limakit
make stop     # limactl stop -f limakit
make clean    # delete the instance and build outputs (the data disk is kept)

Use Docker from the Mac through the forwarded socket:

export DOCKER_HOST=unix://$HOME/.lima/limakit/sock/docker.sock
docker run --rm -p 8080:80 nginx   # reachable on localhost:8080

make run recreates the instance on every run, because Lima copies the image when it creates an instance. That also starts Docker data and /home fresh.

How it works

The image is LinuxKit's kernel+squashfs format: Lima boots the kernel directly and mounts the squashfs read-only as the root disk, so nothing is unpacked into RAM (an idle VM uses ~150 MB). Everything runs as containerd services:

Service Role
lima (lima-init/) Alpine container that does cloud-init's job: reads Lima's cidata ISO, creates the user and keys, writes /etc/fstab, sets timezone, CA certs and DNS, and runs bootcmd. It then starts sshd and runs Lima's own boot.sh. shims/ are a minimal OpenRC stand-in, so Lima installs and starts lima-guestagent (and rosettad) the OpenRC way, and so does the k3s installer.
docker docker:29-dind, pinned by digest. Socket at /run/dockersock/docker.sock.
dhcpcd, acpid, ntpd, rngd, trim-after-delete LinuxKit packages. acpid gets a power-button action, so limactl stop shuts down cleanly.

Lima's mounts (virtiofs or reverse-sshfs) happen in the lima service. They propagate into dockerd's namespace through shared roots (/Users, /Volumes, /mnt, /home, /tmp) bound rshared → rslave, so docker run -v ~/proj:/x works.

Things to know

  • Data lives on the instance's own disk (disk: in lima.yaml, 100 GiB by default, sparse on the host), after the squashfs root: docker data, the guest /home, /root, /usr/local and the ssh host keys. It survives restarts, not limactl delete (or a recreate, which is how you move to a new release).

  • additionalDisks are Lima's: Lima formats and mounts them (format/fsType work). limakit also works around a Lima bug that re-formats disks whose names don't fit the filesystem's label limit (ext4: 11 chars, xfs: 7).

  • Docker only sees Lima mounts under /Users, /Volumes, /mnt, /home and /tmp. A mount elsewhere works in the guest shell, but docker run -v sees an empty directory. The boot log warns about such mounts.

  • containerd is off by default; containerd.system and containerd.user in lima.yaml turn it on. limakit doesn't use Lima's install, which unpacks nerdctl-full and needs systemd. It runs Alpine's containerd, buildkitd and nerdctl from the lima image, in the lima container, so nerdctl run -v sees every Lima mount. nerdctl (and nerdctl.lima on the Mac) talks to the rootless daemon (containerd.user), sudo nerdctl to the rootful one (containerd.system). Both run next to Docker. A systemctl stub exists only to pass Lima's start-up check; there is no systemd.

  • Rootless Docker is off by default; env: {LIMAKIT_DOCKER_ROOTLESS: "true"} in lima.yaml turns it on. Lima has no setting for it (its docker template sets it up with systemd), and it rejects a param: that no script uses, so limakit reads env:. Alpine's dockerd-rootless runs as the Lima user in the lima container. docker in the guest shell talks to it, and files that container root writes to mounts are owned by you. sudo docker still reaches the rootful daemon. To use it from the Mac, forward its socket:

    env:
      LIMAKIT_DOCKER_ROOTLESS: "true"
    portForwards:
      - guestSocket: "/run/user/{{.UID}}/docker.sock"
        hostSocket: "{{.Dir}}/sock/docker-rootless.sock"

    It has no memory/CPU limits: rootless dockerd uses cgroups only through systemd, so -m/--cpus only print a warning. It doesn't start alongside containerd.user, because dockerd-rootless refuses to run next to containerd-rootless.sh; the boot log warns.

  • Lima's k3s template works (base: template:k3s on top of limakit's lima.yaml). Its provision script runs get.k3s.io, which takes the OpenRC path through the shims. k3s runs in the lima container, with its state on the data disk (/var/lib/rancher, /var/lib/kubelet, /etc/rancher). Services enabled with rc-update add start again on every boot, and the shim respawns supervise-daemon services. The guest agent forwards NodePort and LoadBalancer Services to the Mac (it finds kubectl and /etc/rancher/k3s/k3s.yaml).

  • Provisioning scripts run in the lima container, not on the LinuxKit host. Put persistent state under /var/lib or the shared roots.

Lima feature support

Tested on an M4 Mac, macOS 27, Lima 2.2.1:

  • Works:
    • docker: container memory/CPU limits, IPv6 networks
    • containerd: containerd.system and containerd.user (nerdctl, nerdctl.lima, nerdctl build, port forwarding, mounts, memory/CPU limits)
    • rootless Docker (LIMAKIT_DOCKER_ROOTLESS): run, build, mounts, port forwarding, the socket on the host
    • Kubernetes in docker: k3s, k3d with flannel (default) or Cilium (BTF, tc BPF, VXLAN/GENEVE in the kernel)
    • Lima's k3s template: node Ready, kubeconfig copied to the host, Service ports forwarded by the guest agent, cluster kept across restarts
    • mounts: virtiofs and reverse-sshfs (ro/rw), all visible to docker run -v
    • port forwarding: TCP and UDP, plus the docker socket on the host; limactl tunnel
    • provisioning: system, user, boot, dependency, data, ansible, param, probes
    • additionalDisks, including fsType: btrfs; discard/trim reaches the host disk file
    • networking: networks: vzNAT, networks: lima: user-v2 (incl. lima-<name>.internal), dns:, host.lima.internal (also from containers), .local (mDNS) lookups through hostResolver (the Mac's resolver answers them; not with hostResolver.enabled: false or dns:)
    • host integration: caCerts, timezone, env:, ssh.forwardAgent, SSH over vsock
    • VM features: Rosetta (binfmt plus the rosettad cache via CDI: --device lima-vm.io/rosetta=cached), nestedVirtualization
    • lifecycle: limactl cp, limactl stop
  • Works guest-side, needs a host install to use:
    • X11 forwarding: the guest sets up DISPLAY; drawing needs XQuartz on the Mac.
    • socket_vmnet networks: need socket_vmnet plus limactl sudoers on the host (see Lima's docs). Not tested here.
  • Not supported: video.display (a console window on the Mac): the kernel has no GPU or USB input.
  • Left out on purpose: audio (the guest gets no sound device access).
  • Not supported:
    • Templates that need systemd (k8s, k0s, podman, faasd). k3s works (above).
    • stargz-snapshotter and bypass4netns for containerd.
    • Snapshots and TPM, which Lima's vz driver doesn't implement.

Testing and CI

test/run.sh boots a throwaway limakit-test instance four times (virtiofs + vzNAT; reverse-sshfs + user-v2 with containerd.system and containerd.user; rootless Docker; Lima's k3s template) and checks every feature above. Checks the host can't run are skipped: no Rosetta, a chip without nested virtualization, or no ansible-playbook.

.github/workflows/build.yml builds on GitHub's arm64 runners (ubuntu-24.04-arm) and publishes to ghcr.io/<owner>/limakit/:

  • lima:latest (what limakit.yml uses), lima:<sha> and lima:<tag>: the lima-init service image
  • the kernel and squashfs as a workflow artifact, attached to a GitHub Release on v* tags

Hosted runners can't boot the image (VZ needs nested virtualization), so test/run.sh runs on a Mac.

Kernel

The kernel is lk-kernel, shared with natkit and yagan: a complete option list over allnoconfig (config/), each option with the feature that needs it, built with LinuxKit's kernel packaging. limakit.yml pins it by version and digest; Renovate proposes new builds here.

About

lima-compatible image build with linuxkit

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages