feat(minvmd,minimald): Stage 2 — minimald as pid-1 via initramfs - #368
feat(minvmd,minimald): Stage 2 — minimald as pid-1 via initramfs#368norrietaylor wants to merge 2 commits into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
43cc577 to
ce2d184
Compare
Stacks on #361 (minimald vsock guest mode). Ships minimald as the initramfs /init (a cpio of the cross-compiled static binary) and serves a full session against the GENERIC upstream microvm-rootfs — no minimald baked into the rootfs. - minimald: run as /init (detect via argv[0]); mount devtmpfs; mount /dev/vda + chroot into the rootfs so /bin/sh + socat resolve; serve over a socat vsock->UDS relay + run_on_uds with tmpfs (/run/minimal) state. The block-root guest path gains a writable data disk + format-on-first-boot. - minvmd: VmConfig.initramfs + krun_set_kernel initramfs arg (MINVMD_INITRAMFS); provision + attach the rw data disk. - scripts/build-initramfs.sh + CI: cross-compile minimald with a fast `initramfs` profile, pack the cpio, cache the aarch64 target, and run the initramfs session e2e against the generic rootfs. Validated: minimald_exec_over_bridge passes booted via initramfs — exec stdout correct, exit 0. Boot-to-READY ~76 ms median, on par with block-root. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
c1db7cd to
294c8bc
Compare
Replace the bare `KRUN_KERNEL_FORMAT_*` / `KRUN_DISK_FORMAT_*` u32 constants with `#[repr(u32)]` enums (`KernelFormat`, `DiskFormat`), so `set_kernel`/`add_disk` can only be handed a known format. The FFI boundary still passes `format as u32`. Per Evan's review on #367. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
crates/minimald/src/guest.rs (1)
82-94: 💤 Low valueConsider logging the initial mount error for diagnostics.
The function formats the disk on any mount failure, which is pragmatic for the controlled VM environment. However, logging the specific error from the first mount attempt would aid debugging when the failure isn't due to a blank disk (e.g., device missing, wrong device type).
💡 Optional improvement
pub fn mount_state_disk(device: &str, target: &str) -> std::io::Result<()> { std::fs::create_dir_all(target)?; - if raw_mount_ext4(device, target).is_ok() { + if let Err(e) = raw_mount_ext4(device, target) { + tracing::info!(device, error = %e, "state disk mount failed; formatting ext4"); + format_ext4(device)?; + raw_mount_ext4(device, target)?; + tracing::info!(device, target, "formatted + mounted state disk"); + } else { tracing::info!(device, target, "mounted state disk"); - return Ok(()); } - // Likely blank on first boot — format ext4, then retry. - tracing::info!(device, "state disk mount failed; formatting ext4"); - format_ext4(device)?; - raw_mount_ext4(device, target)?; - tracing::info!(device, target, "formatted + mounted state disk"); Ok(()) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/minimald/src/guest.rs` around lines 82 - 94, In mount_state_disk, capture the Result/error returned from the first raw_mount_ext4(device, target) attempt and include that error in the log before formatting; specifically, replace the current is_ok() check by matching or binding the Err to a variable (from the first raw_mount_ext4 call) and pass that error into tracing::info/warn so the message includes the actual mount failure (referencing mount_state_disk and raw_mount_ext4), then proceed to format_ext4(device) and retry as before.crates/minimald/src/main.rs (1)
297-303: 💤 Low valueConsider checking for specific errors when removing stale socket.
let _ = std::fs::remove_file(&guest_uds)ignores all errors, which could mask issues like permission denied or the path being a directory. While unlikely at early boot, checking forNotFoundspecifically is slightly safer.💡 Optional improvement
- let _ = std::fs::remove_file(&guest_uds); + if let Err(e) = std::fs::remove_file(&guest_uds) { + if e.kind() != std::io::ErrorKind::NotFound { + tracing::warn!(error = %e, "failed to remove stale guest UDS"); + } + }This same pattern appears at line 349 in
run_guest.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/minimald/src/main.rs` around lines 297 - 303, The removal of the stale UDS currently swallows all errors via let _ = std::fs::remove_file(&guest_uds), which can hide real issues; update the code around guest_uds (the remove_file call used before UnixListener::bind and the analogous call in run_guest) to explicitly handle errors by matching the result: treat std::io::ErrorKind::NotFound as OK (ignore) but propagate or return other errors (e.g., wrap them with MainError::IO) so permission/invalid-path errors are not silently ignored before calling UnixListener::bind or guest::spawn_vsock_relay.scripts/build-initramfs.sh (1)
27-27: 💤 Low valueConsider removing unused staging directories.
The script creates
dev,proc,sys, andnewrootdirectories but only uses the staging root to place theinitbinary. These empty directories are included in the cpio archive but aren't referenced by the initramfs code.🧹 Suggested simplification
STAGE="$(mktemp -d)" trap 'rm -rf "$STAGE"' EXIT -mkdir -p "$STAGE/dev" "$STAGE/proc" "$STAGE/sys" "$STAGE/newroot" cp "$BIN" "$STAGE/init" chmod +x "$STAGE/init"If these directories are planned for future use (e.g., pre-creating mount points), consider adding a comment explaining their purpose.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/build-initramfs.sh` at line 27, Remove or document the unused staging directories created by the mkdir call: if dev, proc, sys, and newroot under the STAGE variable are not used by the initramfs (they only add empty entries to the cpio archive), delete them from the mkdir invocation so only the necessary staging root and any required directories (e.g., where the init binary is placed) are created; alternatively, if you intend to keep them as pre-created mount points, add a short comment next to the mkdir "$STAGE/..." line explaining their purpose (reference: the mkdir invocation that creates "$STAGE/dev" "$STAGE/proc" "$STAGE/sys" "$STAGE/newroot" and the STAGE variable)..github/workflows/ci-macos.yml (1)
78-81: 💤 Low valueConsider pinning the
crossversion for reproducible CI builds.The
taiki-e/install-actionwill install the latestcrossby default. Ifcrossreleases a breaking change, CI builds could become non-reproducible or fail unexpectedly.📌 Suggested change to pin cross version
- name: Install cross uses: taiki-e/install-action@v2 with: - tool: cross + tool: cross@0.2.5 # or latest stable version🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci-macos.yml around lines 78 - 81, The GitHub Actions step named "Install cross" currently uses taiki-e/install-action@v2 with only "tool: cross", which installs the latest cross and risks non-reproducible CI; update that step to pin the cross tool to a specific release by adding a version input (e.g., add a "version: 'X.Y.Z'" field) to the taiki-e/install-action invocation so the step installs a fixed cross version instead of the latest.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/minvmd/src/bin/krun_smoke_child.rs`:
- Around line 69-73: The kernel format selection sets KernelFormat::ImageGz for
aarch64 which is unsupported; change the conditional in krun_smoke_child.rs
where you assign the variable format so that if cfg!(target_arch = "aarch64")
you use KernelFormat::Raw instead of KernelFormat::ImageGz, otherwise keep the
existing non-aarch64 branch (KernelFormat::Elf); update the assignment that
produces format (the KernelFormat enum usage) so aarch64 loads the uncompressed
RAW kernel.
---
Nitpick comments:
In @.github/workflows/ci-macos.yml:
- Around line 78-81: The GitHub Actions step named "Install cross" currently
uses taiki-e/install-action@v2 with only "tool: cross", which installs the
latest cross and risks non-reproducible CI; update that step to pin the cross
tool to a specific release by adding a version input (e.g., add a "version:
'X.Y.Z'" field) to the taiki-e/install-action invocation so the step installs a
fixed cross version instead of the latest.
In `@crates/minimald/src/guest.rs`:
- Around line 82-94: In mount_state_disk, capture the Result/error returned from
the first raw_mount_ext4(device, target) attempt and include that error in the
log before formatting; specifically, replace the current is_ok() check by
matching or binding the Err to a variable (from the first raw_mount_ext4 call)
and pass that error into tracing::info/warn so the message includes the actual
mount failure (referencing mount_state_disk and raw_mount_ext4), then proceed to
format_ext4(device) and retry as before.
In `@crates/minimald/src/main.rs`:
- Around line 297-303: The removal of the stale UDS currently swallows all
errors via let _ = std::fs::remove_file(&guest_uds), which can hide real issues;
update the code around guest_uds (the remove_file call used before
UnixListener::bind and the analogous call in run_guest) to explicitly handle
errors by matching the result: treat std::io::ErrorKind::NotFound as OK (ignore)
but propagate or return other errors (e.g., wrap them with MainError::IO) so
permission/invalid-path errors are not silently ignored before calling
UnixListener::bind or guest::spawn_vsock_relay.
In `@scripts/build-initramfs.sh`:
- Line 27: Remove or document the unused staging directories created by the
mkdir call: if dev, proc, sys, and newroot under the STAGE variable are not used
by the initramfs (they only add empty entries to the cpio archive), delete them
from the mkdir invocation so only the necessary staging root and any required
directories (e.g., where the init binary is placed) are created; alternatively,
if you intend to keep them as pre-created mount points, add a short comment next
to the mkdir "$STAGE/..." line explaining their purpose (reference: the mkdir
invocation that creates "$STAGE/dev" "$STAGE/proc" "$STAGE/sys" "$STAGE/newroot"
and the STAGE variable).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7f948338-bc9f-4dc8-b91d-85738654c7de
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (17)
.github/workflows/ci-macos.ymlCargo.tomlcrates/minimald/src/guest.rscrates/minimald/src/main.rscrates/minimald/src/server.rscrates/minvmd/Cargo.tomlcrates/minvmd/README.mdcrates/minvmd/src/bin/krun_smoke_child.rscrates/minvmd/src/cmd/vmm_child.rscrates/minvmd/src/image.rscrates/minvmd/src/krun/ctx.rscrates/minvmd/src/krun/mod.rscrates/minvmd/src/krun/raw.rscrates/minvmd/src/state.rscrates/minvmd/src/vm.rscrates/minvmd/tests/minimald_session_e2e.rsscripts/build-initramfs.sh
| let format = if cfg!(target_arch = "aarch64") { | ||
| KRUN_KERNEL_FORMAT_IMAGE_GZ | ||
| KernelFormat::ImageGz | ||
| } else { | ||
| KRUN_KERNEL_FORMAT_ELF | ||
| KernelFormat::Elf | ||
| }; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify aarch64 kernel format usage across the codebase
echo "=== Checking image.rs kernel_format() implementation ==="
rg -n -A 3 -B 3 'target_arch = "aarch64"' crates/minvmd/src/image.rs
echo ""
echo "=== Checking raw.rs documentation on ImageGz support ==="
rg -n -C 2 'ImageGz.*x86_64' crates/minvmd/src/krun/raw.rs
echo ""
echo "=== Checking for other aarch64 kernel format usage ==="
rg -n 'KernelFormat::(Raw|ImageGz|Elf|PeGz)' crates/minvmd/src/ --type rustRepository: gominimal/minimal
Length of output: 1897
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Inspect krun_smoke_child.rs around lines 55-90 ==="
nl -ba crates/minvmd/src/bin/krun_smoke_child.rs | sed -n '55,95p'
echo
echo "=== Search for usage of variable `format` in this file ==="
rg -n "let format|\\bformat\\b" crates/minvmd/src/bin/krun_smoke_child.rsRepository: gominimal/minimal
Length of output: 177
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Inspect crates/minvmd/src/bin/krun_smoke_child.rs around lines 55-95 (cat -n) ==="
cat -n crates/minvmd/src/bin/krun_smoke_child.rs | sed -n '55,95p'
echo
echo "=== Search for usages of variable `format` in this file ==="
rg -n --no-heading "\\bformat\\b" crates/minvmd/src/bin/krun_smoke_child.rsRepository: gominimal/minimal
Length of output: 2009
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Inspect crates/minvmd/src/bin/krun_smoke_child.rs (top + kernel selection) ==="
cat -n crates/minvmd/src/bin/krun_smoke_child.rs | sed -n '1,160p'
echo
echo "=== Inspect crates/minvmd/src/image.rs (kernel_format + how kernel file is prepared) ==="
cat -n crates/minvmd/src/image.rs | sed -n '1,220p'
echo
echo "=== Locate KernelFormat enum in crates/minvmd/src/krun ==="
rg -n "enum KernelFormat|KernelFormat::" crates/minvmd/src/krun -SRepository: gominimal/minimal
Length of output: 9809
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Inspect crates/minvmd/src/krun/raw.rs around KernelFormat + ImageGz docs ==="
rg -n "ImageGz" crates/minvmd/src/krun/raw.rs
echo
cat -n crates/minvmd/src/krun/raw.rs | sed -n '1,120p'Repository: gominimal/minimal
Length of output: 6565
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Find where MINVMD_KERNEL_PATH is set/used (tests + minvmd) ==="
rg -n "MINVMD_KERNEL_PATH" -S .
echo
echo "=== Inspect tests/krun_smoke.rs ==="
ls -la tests || true
cat -n tests/krun_smoke.rs 2>/dev/null || trueRepository: gominimal/minimal
Length of output: 4146
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Inspect crates/minvmd/tests/krun_smoke.rs ==="
cat -n crates/minvmd/tests/krun_smoke.rs | sed -n '1,220p'
echo
echo "=== Search in crates/minvmd/tests for Image.gz / vmlinuz / uncompressed kernel ==="
rg -n "Image\.gz|vmlinuz|uncompressed|gunzip|ImageGz|KernelFormat::Raw|KernelFormat::ImageGz" crates/minvmd/tests crates/minvmd/src -SRepository: gominimal/minimal
Length of output: 6102
Fix aarch64 kernel format selection in krun_smoke_child.rs
File: crates/minvmd/src/bin/krun_smoke_child.rs (lines 69-73)
The smoke child selects KernelFormat::ImageGz for aarch64, but minvmd/src/image.rs states the aarch64 kernel is shipped uncompressed and should be loaded with KernelFormat::Raw. Also, minvmd/src/krun/raw.rs documents that ImageGz (=4) is x86_64-only and returns KernelFormatUnsupported on aarch64 (aarch64 loader implements only RAW and PE_GZ). This mismatch can make the smoke test fail on aarch64.
let format = if cfg!(target_arch = "aarch64") {
KernelFormat::ImageGz
} else {
KernelFormat::Elf
};🔧 Proposed fix
let format = if cfg!(target_arch = "aarch64") {
- KernelFormat::ImageGz
+ KernelFormat::Raw
} else {
KernelFormat::Elf
};🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/minvmd/src/bin/krun_smoke_child.rs` around lines 69 - 73, The kernel
format selection sets KernelFormat::ImageGz for aarch64 which is unsupported;
change the conditional in krun_smoke_child.rs where you assign the variable
format so that if cfg!(target_arch = "aarch64") you use KernelFormat::Raw
instead of KernelFormat::ImageGz, otherwise keep the existing non-aarch64 branch
(KernelFormat::Elf); update the assignment that produces format (the
KernelFormat enum usage) so aarch64 loads the uncompressed RAW kernel.
|
Closing in favor of #373 + #374, which re-slice this work along capability lines instead of delivery-mechanism lines:
The block-root boot path ( |
Stacks on #367 (Stage 1, generic upstream rootfs). Supersedes #362 (the local-rootfs integration).
Approach
minimald runs the guest as pid-1, shipped as the initramfs
/init(a ~27 MB cpio of the cross-compiled static binary) — not baked into the rootfs. The guest root stays the generic upstreammicrovm-rootfs. minimald mounts/dev, mounts the rootfs (/dev/vda),chroots in so/bin/sh+socatresolve, then serves a session over the bridge (socat vsock→UDS relay +run_on_uds, tmpfs state at/run/minimal).Why this over the local-rootfs approach (#362)
minvmd-rootfspackagemicrovm-rootfskrun_set_kernelValidated locally
minimald_exec_over_bridgebooted via initramfs against the generic upstreammicrovm-rootfs(no minimald, no mke2fs in it):CreateSession+ exec →stdout="MINIMALD_SESSION_OK" exit=0. Boot-to-READY ~76 ms median (n=10), on par with block-root.CI
artifactsjob (ubuntu) builds the kernel (cache pull) + the initramfs (cross-compile minimald → cpio) and uploads both;boot-e2eruns the initramfs session e2e on the self-hosted runner.Follow-up (not blocking, flagged)
Session state is on tmpfs (ephemeral). A persistent host key + session store needs the data disk, which needs
mke2fsto format (the generic rootfs excludes e2fsprogs). Options: ship a smallmke2fsin the initramfs, a Rust ext4 formatter, or host-side pre-format. The session itself is proven.Draft until CI is green on the runner and the persistence decision is made.
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
New Features
Bug Fixes
Improvements
Documentation