ci,test: auto-discover integration harnesses by name convention across every lane - #732
Conversation
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (20)
💤 Files with no reviewable changes (3)
📝 WalkthroughWalkthroughThe PR standardizes integration-test naming, moves native/macOS/KVM CI to nextest-based discovery, updates minvmd networking ownership and lifecycle validation, removes obsolete relay coverage, and synchronizes README and specification references. ChangesIntegration migration
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant NextestArchive
participant IntegrationHarnesses
participant Minvmd
GitHubActions->>NextestArchive: build or load archived harnesses
NextestArchive->>IntegrationHarnesses: select *_integration binaries
IntegrationHarnesses->>Minvmd: run VM integration tests
Minvmd-->>GitHubActions: report test and lifecycle status
Possibly related issues
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Comment |
VM e2e harnesses are now selected by binary-name convention instead of a hardcoded per-test list, so adding a test never requires a CI edit: - Harnesses in crates/minvmd/tests/ end in `_e2e` (non-root) or `_root_e2e` (root, run under sudo for CAP_NET_ADMIN). - The KVM lane's three enumerated steps collapse to two convention filtersets: `binary(/_e2e$/) and not binary(/_root_e2e$/)` and `binary(/_root_e2e$/)`. A new `*_e2e.rs` is picked up with no YAML. - Rename krun_smoke -> krun_smoke_e2e and vsock_relay_e2e -> vsock_relay_root_e2e; delete the retired bridge_e2e stub. - A unit test in minvmd (runs in core-tests, no libkrun) guards the suffix so a typo fails CI instead of silently dropping a harness. This newly runs volume_quiesce_e2e (added in #705 but never wired into a lane) in the KVM lane. The macOS lane tracks the rename only; its filterset conversion is coupled to the nextest-archive-on-mac work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
…discovery The netns-integration job hardcoded `cargo test -p minimald --test netns`. Rename netns.rs -> netns_root_e2e.rs and select it by the same convention the KVM lane uses: the renamed `minimald-root-e2e` job runs `cargo nextest run -p minimald -E 'binary(/_root_e2e$/)'`, so a new crates/minimald/tests/*_root_e2e.rs runs with no workflow edit. A minimald unit test guards the suffix; mesh_uc7 is allowlisted because its networking-wg feature is off by default and no lane runs it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
The mac e2e job hardcoded two test names and ran them as direct binaries. Build a nextest archive before the codesign (`nextest run --archive-file` is offline, so the signature survives) and select harnesses by the `_e2e` convention, so a new crates/minvmd/tests/*_e2e.rs runs on mac with no workflow edit. This newly runs boot_e2e and volume_quiesce_e2e on mac. The full-boot harnesses spawn the codesigned minvmd as a subprocess (MINVMD_BIN). krun_smoke_e2e stays a separate kernel-less FFI-smoke step, excluded from the archive run: it boots in-process through krun_smoke_child, which cannot take the hypervisor entitlement from within the archive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
Reflect the auto-discovery migration in the frozen unit specs: krun_smoke -> krun_smoke_e2e in R1.5, and note that bridge_e2e was removed (superseded by minimald_session_e2e). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
minvmd's net::relay (open_tap/attach_to_switch/SwitchRelay) has had no production caller since #581 moved the per-PTask tap+relay into the guest minimald over the vsock shuttle; it was a frozen byte-for-byte fork of minimald's net::switch that never received the ingress gate the live copy grew. Its only caller was vsock_relay_root_e2e, which booted no VM and re-proved the relay against a hand-made netns — coverage already provided, with real PTask-to-PTask traffic, by minimald's netns_root_e2e UC6. Delete net/relay.rs, its module wiring, and the test; correct the net.rs module doc (the tap + relay run in the guest, not here). The KVM lane's root step existed only for that test, so it is removed along with the gvproxy materialize step and the vendor/gvproxy pin filter it needed; the lane keeps a single auto-discovered harness step (a future minvmd *_root_e2e would need its own sudo-wrapped step). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
…ally Drop the internal use-case numbers (UC1/UC4/UC6/UC7) from the netns test function names, its module doc, the native-lane comments, and the minimald naming guard. Describe each proof by what it does — a no-network task cannot reach the internet, a static ingress mapping exposes then removes a task's port, two own-IP tasks reach each other over the gvproxy switch — and state why they need root: they create tap devices and configure network namespaces, which requires CAP_NET_ADMIN, so the unprivileged CI runner grants it via sudo. No behavior change; the harness is selected by binary name, not function name, so the renames are self-contained. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
The harness-driven nextest proofs are component/subsystem INTEGRATION tests, not full-system e2e: each stands up one subsystem against real resources (boot a microVM, round-trip the vsock bridge, drive the libkrun FFI, check a real ext4 volume, exercise the tap/netns stack) rather than driving the `minimal` CLI through the whole system. The only true end-to-end proof is the CLI script (scripts/session-e2e.sh). Rename the auto-discovery suffix _e2e / _root_e2e -> _integration / _root_integration across the five harnesses, the KVM/native/mac lane filtersets, and both naming guards; the guard docs now state when to reach for an integration harness vs. a full-system e2e script. Rename the minvmd-scoped lifecycle script (lifecycle-e2e.sh -> minvmd-lifecycle.sh) and the native root job (minimald-root-e2e -> minimald-root-integration). Also scrub the workflow YAMLs of internal plan jargon (UC/N/R references) in favour of functional descriptions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
48c5290 to
1652938
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/minimald/src/lib.rs`:
- Around line 62-67: Update the test-file discovery flow building offenders so
directory-entry errors from std::fs::read_dir are not discarded by
filter_map(Result::ok). Propagate the error or fail explicitly with context,
while preserving the existing filtering of regular Rust files and the
fail-closed naming guard behavior.
In `@crates/minvmd/tests/krun_smoke_integration.rs`:
- Around line 69-83: Update the environment-variable branching around kernel_set
and rootfs_set to reject partial configuration: require both MINVMD_KERNEL_PATH
and MINVMD_ROOTFS_PATH to be set or neither to be set, and fail the test
explicitly when only one is present. Preserve the existing full-path start_enter
assertions and bring-up-only behavior for valid configurations.
In `@docs/specs/01-spec-minvmd-host-daemon/01-spec-minvmd-host-daemon.md`:
- Around line 250-253: Update the preceding test-reference bullet in the active
proof artifact to replace tests/bridge_e2e.rs with
tests/minimald_session_integration.rs, keeping the entry presented as a required
passing test rather than referencing the retired bridge test.
In `@docs/specs/02-spec-minvmd-linux-kvm/02-spec-minvmd-linux-kvm.md`:
- Around line 218-239: Remove the obsolete bridge_e2e.rs entry from the affected
areas and delete requirement R3.3, while retaining
minimald_session_integration.rs as the active Linux session integration proof.
Leave R3.1 and R3.2 unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3efc569d-b8f6-482d-bec3-6d0d77ecf687
📒 Files selected for processing (20)
.github/workflows/ci-linux-kvm.yml.github/workflows/ci-linux-native.yml.github/workflows/ci-macos.yml.github/workflows/release.ymlcrates/minimald/src/lib.rscrates/minimald/tests/netns_root_integration.rscrates/minvmd/README.mdcrates/minvmd/src/lib.rscrates/minvmd/src/net.rscrates/minvmd/src/net/relay.rscrates/minvmd/tests/boot_integration.rscrates/minvmd/tests/bridge_e2e.rscrates/minvmd/tests/krun_smoke_integration.rscrates/minvmd/tests/minimald_session_integration.rscrates/minvmd/tests/volume_quiesce_integration.rscrates/minvmd/tests/vsock_relay_e2e.rsdocs/specs/01-spec-minvmd-host-daemon/01-spec-minvmd-host-daemon.mddocs/specs/02-spec-minvmd-linux-kvm/02-spec-minvmd-linux-kvm.mddocs/specs/02-spec-minvmd-linux-kvm/architecture.mdscripts/minvmd-lifecycle.sh
💤 Files with no reviewable changes (3)
- crates/minvmd/tests/vsock_relay_e2e.rs
- crates/minvmd/src/net/relay.rs
- crates/minvmd/tests/bridge_e2e.rs
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/minimald/src/lib.rs`:
- Around line 62-67: Update the test-file discovery flow building offenders so
directory-entry errors from std::fs::read_dir are not discarded by
filter_map(Result::ok). Propagate the error or fail explicitly with context,
while preserving the existing filtering of regular Rust files and the
fail-closed naming guard behavior.
In `@crates/minvmd/tests/krun_smoke_integration.rs`:
- Around line 69-83: Update the environment-variable branching around kernel_set
and rootfs_set to reject partial configuration: require both MINVMD_KERNEL_PATH
and MINVMD_ROOTFS_PATH to be set or neither to be set, and fail the test
explicitly when only one is present. Preserve the existing full-path start_enter
assertions and bring-up-only behavior for valid configurations.
In `@docs/specs/01-spec-minvmd-host-daemon/01-spec-minvmd-host-daemon.md`:
- Around line 250-253: Update the preceding test-reference bullet in the active
proof artifact to replace tests/bridge_e2e.rs with
tests/minimald_session_integration.rs, keeping the entry presented as a required
passing test rather than referencing the retired bridge test.
In `@docs/specs/02-spec-minvmd-linux-kvm/02-spec-minvmd-linux-kvm.md`:
- Around line 218-239: Remove the obsolete bridge_e2e.rs entry from the affected
areas and delete requirement R3.3, while retaining
minimald_session_integration.rs as the active Linux session integration proof.
Leave R3.1 and R3.2 unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3efc569d-b8f6-482d-bec3-6d0d77ecf687
📒 Files selected for processing (20)
.github/workflows/ci-linux-kvm.yml.github/workflows/ci-linux-native.yml.github/workflows/ci-macos.yml.github/workflows/release.ymlcrates/minimald/src/lib.rscrates/minimald/tests/netns_root_integration.rscrates/minvmd/README.mdcrates/minvmd/src/lib.rscrates/minvmd/src/net.rscrates/minvmd/src/net/relay.rscrates/minvmd/tests/boot_integration.rscrates/minvmd/tests/bridge_e2e.rscrates/minvmd/tests/krun_smoke_integration.rscrates/minvmd/tests/minimald_session_integration.rscrates/minvmd/tests/volume_quiesce_integration.rscrates/minvmd/tests/vsock_relay_e2e.rsdocs/specs/01-spec-minvmd-host-daemon/01-spec-minvmd-host-daemon.mddocs/specs/02-spec-minvmd-linux-kvm/02-spec-minvmd-linux-kvm.mddocs/specs/02-spec-minvmd-linux-kvm/architecture.mdscripts/minvmd-lifecycle.sh
💤 Files with no reviewable changes (3)
- crates/minvmd/tests/vsock_relay_e2e.rs
- crates/minvmd/src/net/relay.rs
- crates/minvmd/tests/bridge_e2e.rs
🛑 Comments failed to post (4)
crates/minimald/src/lib.rs (1)
62-67: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Do not silently discard directory-entry errors.
filter_map(Result::ok)can omit an unreadable*.rsentry, allowing the naming guard to pass without inspecting every test file. Propagate the error or fail with context so this guard remains fail-closed.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/minimald/src/lib.rs` around lines 62 - 67, Update the test-file discovery flow building offenders so directory-entry errors from std::fs::read_dir are not discarded by filter_map(Result::ok). Propagate the error or fail explicitly with context, while preserving the existing filtering of regular Rust files and the fail-closed naming guard behavior.crates/minvmd/tests/krun_smoke_integration.rs (1)
69-83: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Reject partially configured full-path environments.
If only one of
MINVMD_KERNEL_PATHorMINVMD_ROOTFS_PATHis set, this test enters the bring-up-only branch and can pass without exercisingstart_enter. Require both variables or neither so CI misconfiguration cannot silently reduce coverage.Proposed fix
let kernel_set = std::env::var("MINVMD_KERNEL_PATH").is_ok(); let rootfs_set = std::env::var("MINVMD_ROOTFS_PATH").is_ok(); +assert_eq!( + kernel_set, rootfs_set, + "MINVMD_KERNEL_PATH and MINVMD_ROOTFS_PATH must be set together" +);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.let kernel_set = std::env::var("MINVMD_KERNEL_PATH").is_ok(); let rootfs_set = std::env::var("MINVMD_ROOTFS_PATH").is_ok(); assert_eq!( kernel_set, rootfs_set, "MINVMD_KERNEL_PATH and MINVMD_ROOTFS_PATH must be set together" ); if kernel_set && rootfs_set { // Full path: helper called start_enter. Accept the libkrun-defined // exit code set per the function's own error-code documentation. assert!( stderr.contains("STAGE: start_enter"), "missing start_enter marker (full path)\n--- stderr ---\n{stderr}", ); assert!( matches!(code, 0 | 2 | 125 | 126 | 127), "unexpected child exit code {code} on full path; expected 0 | 2 | 125 | 126 | 127", ); } else {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/minvmd/tests/krun_smoke_integration.rs` around lines 69 - 83, Update the environment-variable branching around kernel_set and rootfs_set to reject partial configuration: require both MINVMD_KERNEL_PATH and MINVMD_ROOTFS_PATH to be set or neither to be set, and fail the test explicitly when only one is present. Preserve the existing full-path start_enter assertions and bring-up-only behavior for valid configurations.docs/specs/01-spec-minvmd-host-daemon/01-spec-minvmd-host-daemon.md (1)
250-253: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Remove the retired bridge test from the active proof artifact.
The preceding bullet still presents
tests/bridge_e2e.rsas a required passing test, while this paragraph says it was removed. Replace it withminimald_session_integration.rsor explicitly mark the entire entry as historical.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/specs/01-spec-minvmd-host-daemon/01-spec-minvmd-host-daemon.md` around lines 250 - 253, Update the preceding test-reference bullet in the active proof artifact to replace tests/bridge_e2e.rs with tests/minimald_session_integration.rs, keeping the entry presented as a required passing test rather than referencing the retired bridge test.docs/specs/02-spec-minvmd-linux-kvm/02-spec-minvmd-linux-kvm.md (1)
218-239: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Remove the obsolete
bridge_e2erequirement.The affected-area list and R3.3 still require
tests/bridge_e2e.rs, but the same block says that harness was removed. Delete this requirement and retainminimald_session_integration.rsas the active session proof.🧰 Tools
🪛 LanguageTool
[uncategorized] ~226-~226: The operating system from Apple is written “macOS”.
Context: ...s/boot_integration.rsshall remove the#![cfg(target_os = "macos")]` file-level attribute. The READY-m...(MAC_OS)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/specs/02-spec-minvmd-linux-kvm/02-spec-minvmd-linux-kvm.md` around lines 218 - 239, Remove the obsolete bridge_e2e.rs entry from the affected areas and delete requirement R3.3, while retaining minimald_session_integration.rs as the active Linux session integration proof. Leave R3.1 and R3.2 unchanged.
Resolve the conflicts from the commits that landed on main since this branch's merge-base (#721, #732, #734, #735, #722), keeping main's content and re-applying the `min` binary-target rename on top. - justfile: main folded `up` into `dm1` (#722), so the branch's older `up` recipe is dropped rather than resurrected. Main's `dm1` invoked the `{{minimal}}` variable this branch renames, which would have left `just` unable to resolve it; it now invokes `{{min-bin}}`. - CI lanes: keep main's rewritten jobs and steps, renaming only the CLI build flags and built-binary paths (`--bin min`, `target/debug/min`). Also point the sessions example project at `./target/debug/min`; the binary path it documented no longer exists after the rename. Published release asset names (`minimal-linux-amd64`, ...), the macOS `minimal` shim, and the `minimal` crate and lib target are deliberately left alone.
What
Establishes two clear, self-documenting test categories and makes the heavier
one auto-discovered on every lane:
crates/<crate>/tests/*_integration.rs(add_root→*_root_integration.rsif it needsCAP_NET_ADMIN). Rust nextestproofs that stand up one subsystem against real resources a unit test
can't give it — boot a microVM, round-trip the vsock bridge, drive the libkrun
FFI, check a real ext4 volume, exercise the tap/netns stack. Selected by
binary-name suffix, so a new file runs with no CI edit.
minimalCLI endto end (activate → session → exec → destroy). These live as scripts under
scripts/(e.g.session-e2e.sh) and are wired per lane.The harness proofs were previously mislabeled
_e2e; none of them drive the CLIor the whole system, so this renames them to
_integrationand reserves "e2e"for the CLI scripts. Both naming guards now carry the when-to-use-which rule.
Convention
*_integration.rs→ run bybinary(/_integration$/) and not binary(/_root_integration$/).*_root_integration.rs→ run bybinary(/_root_integration$/), undersudo.#[cfg(test)]unit guard in each crate (minvmd,minimald) asserts thesuffix, so a typo fails CI loudly instead of silently dropping the test.
Per lane
test-kvmbinary(/_integration$/)minimald-root-integrationcargo test -p minimald --test netnscargo nextest run -p minimald -E 'binary(/_root_integration$/)'e2e(mini)--testnames, run as direct binariesbinary(/_integration$/)Which tests need root, and why
Only the network-namespace proofs (native lane) need root. They create tap
devices (
/dev/net/tun) and configure network namespaces + interfaces,which requires
CAP_NET_ADMIN; the daemon holds it in production, so theunprivileged CI runner grants it via
sudo. Booting a VM does not need root(guest traffic rides vsock;
/dev/kvmis opened via a udev group rule), so everyKVM/mac harness is unprivileged.
Renames and removals
boot_e2e→boot_integration,minimald_session_e2e→minimald_session_integration,volume_quiesce_e2e→volume_quiesce_integration,krun_smoke_e2e→krun_smoke_integration,netns_root_e2e→netns_root_integration.scripts/lifecycle-e2e.sh→scripts/minvmd-lifecycle.sh(it drivesminvmdlifecycle, not the whole system); native job
minimald-root-e2e→minimald-root-integration.bridge_e2e.rs(retired Stage-1 socat-echo stub, superseded byminimald_session_integration) and the dead host-side tap relay inminvmd(
net/relay.rs+ its test) — no production caller since the tap+relay movedinto the guest daemon; its coverage is already provided by the netns own-IP
proof.
New coverage
volume_quiesce_integration(added in #705,wired into no lane) is now auto-discovered and runs on both KVM and Mac.
Notes
krun_smoke_integrationstays a separate kernel-less FFI-smoke step on Mac(its in-process helper can't take the hypervisor entitlement from inside the
archive); the full-boot harnesses boot via the codesigned
minvmdsubprocess.networking-wg, off, mothballed) stays un-run —allowlisted in the minimald guard.
favour of functional descriptions.
Proof
All three lanes green on the branch, with the auto-discovered harnesses running:
volume_quiesce×3.binary(/_root_integration$/), plus the guard.volume_quiesce×3).🤖 Generated with Claude Code
https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
Summary by CodeRabbit
New Features
Bug Fixes
Documentation