ci(minvmd): virtio-linux kernel-fetch lane + non-gating boot E2E - #341
ci(minvmd): virtio-linux kernel-fetch lane + non-gating boot E2E#341norrietaylor wants to merge 2 commits into
Conversation
Pull the prebuilt virtio-linux kernel from the public minimal build cache (gs://minimal-shim CLI -> minimal materialize virtio-linux from minimal-staging-cache; no build, no auth) on a Linux job and hand it to a self-hosted macOS boot-e2e job. boot-e2e is continue-on-error for now: the R2.4 READY round-trip needs the R3.4 guest vsock stub (#328) and a settled marker-port contract (host VSOCK_MARKER_PORT=9799 vs guest rootfs 7350), and R2.2 rootfs staging (scripts/fetch-alpine.sh) is not yet on main. Drop continue-on-error to make it required once those land. Refs: #339 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
📝 WalkthroughWalkthroughAdds a Linux helper script to fetch a prebuilt virtio-linux kernel and updates the macOS CI workflow to produce that kernel artifact on Ubuntu and run a conditional self-hosted Apple Silicon boot-e2e job that downloads the kernel, stages a rootfs, builds/codesigns minvmd, and runs boot tests. ChangesE2E Boot Testing Setup
🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly Related PRs
Suggested Reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci-macos.yml:
- Around line 89-109: The workflow step currently wraps setup, build, codesign,
and the test run under continue-on-error; split it so the ROOTFS
download/extract (the curl | tar into ROOTFS), kernel staging, the cargo build
-p minvmd --bin minvmd, and the codesign invocation (codesign --entitlements ...
target/debug/minvmd) are separate steps that run normally (fail the job on
error), and keep continue-on-error: true only on the final step that runs
MINVMD_E2E=1 ... cargo test -p minvmd --test boot_e2e -- --include-ignored
--nocapture; ensure the final step still sets MINVMD_KERNEL_PATH,
MINVMD_ROOTFS_PATH and MINVMD_E2E as before so the test behavior is unchanged.
- Around line 49-50: The workflow’s path filter is missing the new script, so
changes to scripts/fetch-virtio-kernel.sh won't trigger the job; update the
ci-macos.yml paths filter to include "scripts/fetch-virtio-kernel.sh" (or a
matching pattern like "scripts/**") so that modifications to the fetch script
run this workflow; locate the job that runs ./scripts/fetch-virtio-kernel.sh and
add the script path to its paths inclusion list.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4eeea8ae-d2cb-4a67-86ff-18aff46c59ca
📒 Files selected for processing (2)
.github/workflows/ci-macos.ymlscripts/fetch-virtio-kernel.sh
…he non-gating test Address CodeRabbit review: - Add scripts/fetch-virtio-kernel.sh to the ci-macos path filter so a change to the fetch logic alone still runs this workflow. - Split rootfs staging, build, and codesign into their own steps that fail the job; keep continue-on-error only on the boot_e2e test invocation, so setup/build regressions are no longer masked by the known-non-gating test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@CodeRabbit review |
✅ Action performedReview finished.
|
|
closing, #344 supersedes |
#326) (#344) * feat(minvmd): add boot command and VMM child subcommand Implements R2.3 and R2.4 from the minvmd spec: - cmd/mod.rs: declares the cmd module, VSOCK_MARKER_PORT (9799) and MARKER_SOCK_ENV constants shared between parent and child. - cmd/boot.rs: the `minvmd boot [--foreground]` subcommand. On macOS: validates MINVMD_KERNEL_PATH and MINVMD_ROOTFS_PATH, creates a UNIX socket listener for the READY marker, fork-execs `minvmd __krun-vmm` with MINVMD_MARKER_SOCK set to the socket path, writes the child PID to vmm.pid, then waits up to 5 s for the guest to connect and write READY\n (R2.4). On success prints vm-up. With --foreground, blocks until the VMM child exits. On Linux: bails immediately (no-op stub). - cmd/vmm_child.rs: the hidden `minvmd __krun-vmm` subcommand. On macOS: creates a libkrun context, applies VmConfig (kernel, rootfs, 2 vcpus, 512 MiB), registers VSOCK_MARKER_PORT pointing to the host UNIX socket, then calls krun_start_enter (R2.3). On Linux: bails immediately (no-op stub). - main.rs: wires Boot and KrunVmm subcommands to the CLI. - lib.rs: exports pub mod cmd. - tests/boot_e2e.rs: READY-marker round-trip E2E test (gated on MINVMD_E2E=1 and #[ignore], macOS only) (R2.4). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(minvmd): reap child and clean up pid/socket on boot failure On the error and timeout paths in boot.rs, terminate and wait the VMM child and remove both vmm.pid and the marker socket before returning, so a failed boot does not leave stale state behind. In boot_e2e.rs, set XDG_STATE_HOME to an isolated tempdir for the spawned child so the E2E test does not clobber the developer's real state directory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(minvmd): serialize boot E2E test and add serial_test dep Add #[serial] to boot_e2e_ready_marker_round_trip so it cannot race other stateful E2E tests that mutate XDG_STATE_HOME. The test already isolates its state directory via XDG_STATE_HOME=<tempdir>, so no additional isolation is needed; serialization ensures only one such test runs at a time. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(minvmd): harden marker socket path with random nonce Use PID + 4 random bytes from /dev/urandom for the READY-marker socket path instead of PID alone, making the path unpredictable and closing the TOCTOU race a local attacker could exploit to inject a spoofed READY. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(minvmd): align READY marker vsock port to canonical 7350 The host awaited the boot READY marker on vsock 9799, but no guest emits on that port: the guest rootfs manifest documents 7350 (etc/minvmd/manifest: vsock_port_ready=7350). With 9799 the marker never arrived. Align the host to 7350. Necessary but not sufficient: krun_start_enter still returns EINVAL; root cause under investigation (see PR comment) — not a kernel-format or rootfs-format issue. Refs: #221 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minvmd): load aarch64 Image.gz with PE_GZ format The aarch64 libkrun loader implements only RAW and PE_GZ; IMAGE_GZ (=4) is x86_64-only and returns KernelFormatUnsupported, so krun_set_kernel failed before the VM could boot. Select KRUN_KERNEL_FORMAT_PE_GZ (=2) for the aarch64 Image.gz (the loader scans for the gzip magic and decompresses) and add the constant to the FFI surface. Verified against /opt/homebrew/include/libkrun.h (libkrun 1.18.1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(minvmd): set guest workload, capture console, raise boot resources The VMM child configured a kernel + rootfs but never set a workload, so libkrun's /init.krun (pid-1) fell back to /bin/sh and never emitted the READY marker. Set the guest workload via krun_set_exec (default /sbin/minvmd-stub-init, MINVMD_EXEC overrides) with an explicit minimal envp — passing None would inherit the full host env and can overflow the ~2 KiB aarch64 kernel cmdline. Add opt-in early-boot console capture (MINVMD_BOOT_LOG) for diagnosing a stuck boot, and raise the VM from 512 MiB to 2 vCPU / 1024 MiB for cheap headroom under Hypervisor.framework. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(minvmd): add VM image build scripts with guest READY writer Stage the non-EFI VM image production on main: - fetch-alpine.sh: pinned, sha256-verified Alpine 3.21.7 minirootfs. - build-rootfs.sh: overlay socat + the /sbin/minvmd-stub-init workload and the guest manifest onto the rootfs directory (consumed by krun_set_root as virtio-fs; no disk image). - fetch-virtio-kernel.sh: pull the prebuilt virtio-linux vmlinuz (Image.gz) from the public minimal build cache (carried from #341). The stub's READY writer connects OUT to the host (CID 2, port 7350) rather than listening: the host registers the marker with the plain krun_add_vsock_port (== listen=false), so the direction is guest->host. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci(minvmd): boot E2E on the real Alpine rootfs + virtio kernel Stage the guest rootfs via fetch-alpine.sh + build-rootfs.sh (was a vanilla extract with no READY writer), capture the guest console as an artifact for debugging, and scope the workflow to scripts/**. The boot E2E stays non-gating (continue-on-error) until first-green on the self-hosted runner, which also validates the virtio-linux kernel config. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(minvmd): correct kernel format, init model, and marker direction Align the spec with libkrun's verified behaviour: aarch64 loads Image.gz via PE_GZ (not IMAGE_GZ); libkrun's /init.krun is pid-1 and execs the krun_set_exec workload (no init system in the rootfs); the kernel cmdline stays unset; boot uses 2 vCPU / 1024 MiB; and the READY marker is guest-initiated (krun_add_vsock_port, listen=false), the opposite of the R3 ssh.sock bridge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minvmd): make image scripts run under macOS bash 3.2 The self-hosted runner's `/usr/bin/env bash` is the macOS system bash 3.2, which lacks associative arrays; `declare -A` made `[aarch64]=` parse as an unbound arithmetic index under `set -u`, failing build-rootfs.sh with "aarch64: unbound variable". Replace the per-arch sha256 arrays with plain vars + a case lookup. Verified under /bin/bash 3.2.57. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minvmd): add socat's runtime closure to the guest rootfs The READY-writer stub runs socat, which dynamically links libreadline.so.8 (needs libncursesw.so.6) — neither is in the Alpine minirootfs, so socat aborted with "Error loading shared library libreadline.so.8" and never wrote READY. Overlay the readline and libncursesw apks (sha256-pinned) alongside socat via a fetch_apk helper. (libssl/libcrypto/libc are already in the base.) NOTE: the library ships in `libncursesw`, not the payload-less `ncurses-libs` metapackage. Verified locally on Apple Silicon: a codesigned `minvmd boot` against the prebuilt virtio-linux kernel boots Alpine and prints `vm-up`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci(minvmd): run the boot E2E binary directly so codesign survives `cargo test` relinks target/debug/minvmd under the test profile, discarding the hypervisor entitlement applied by codesign — so krun_start_enter failed with EINVAL even though a prior step signed the binary. Build with `--no-run`, codesign, then execute the prebuilt boot_e2e test binary directly (no further cargo invocation), which preserves the signature. Stage the real rootfs via build-rootfs.sh and upload the guest console log. Verified locally: build --no-run -> codesign -> run test binary == 1 passed, binary still signed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci(minvmd): make the boot E2E a required gate The READY round-trip is green on the self-hosted Apple Silicon runner (test result: ok. 1 passed). Drop continue-on-error so a boot regression fails the workflow. This is the acceptance gate for #311 / #326. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(minvmd): expand R2.2 for the rootfs overlay and socat closure Name build-rootfs.sh (overlays /sbin/minvmd-stub-init + socat + the readline/libncursesw closure), not just fetch-alpine.sh, and state the result is a virtio-fs directory (no disk image). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minvmd): propagate /dev/urandom read failure for the marker nonce A failed open/read of /dev/urandom silently fell back to a zero nonce, making the marker socket path predictable and defeating the TOCTOU hardening the comment promises. Propagate the error with context instead of discarding it via `let _ =` (which the repo's standards ban). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: gominimal-aw-bot[bot] <281738952+gominimal-aw-bot[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What
Adds a CI lane that pulls the prebuilt virtio-linux kernel from the public minimal build cache and runs the minvmd boot E2E on the self-hosted macOS runner.
scripts/fetch-virtio-kernel.sh— fetches the promoted LinuxminimalCLI fromgs://minimal-shim, thenminimal materializesvirtio-linux'svmlinuzfrom the publicminimal-staging-cache(no kernel build, no GCS auth). Verified locally: produces a 14 MBImage.gz..github/workflows/ci-macos.yml:virtio-kerneljob (ubuntu) → uploads thevmlinuzartifact.boot-e2ejob (self-hosted macOS) → downloads the kernel, stages a rootfs, codesigns the binary (R1.4 hypervisor entitlement), runsboot_e2e.Non-gating for now
boot-e2eiscontinue-on-error: true. The R2.4 READY round-trip cannot pass yet:VSOCK_MARKER_PORT=9799vs guest rootfs7350;scripts/fetch-alpine.sh) is not on main (staged inline here as a stopgap; vanilla Alpine has no READY-writer).Drop
continue-on-errorto make it a required check once #328 and the port contract land. The kernel-fetch lane is the durable, reusable piece.Refs: #339 · kernel dep gominimal/pkgs virtio-linux
Summary by CodeRabbit