Skip to content

fix(minvmd): resource-surface hardening from post-merge review of #775 - #793

Closed
norrietaylor wants to merge 5 commits into
mainfrom
fix/minvmd-resource-hardening
Closed

fix(minvmd): resource-surface hardening from post-merge review of #775#793
norrietaylor wants to merge 5 commits into
mainfrom
fix/minvmd-resource-hardening

Conversation

@norrietaylor

@norrietaylor norrietaylor commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary

Follow-up to #775: fixes the four confirmed findings from CodeRabbit's post-merge review of the minvmd resource surface, plus one adjacent bug the new tests exposed. Each section below states the failure as state + action → observed wrong outcome.

1. status could report resources the VM did not boot with

Failure state: A VM is booting (the ~30–60 s window between VMM-child spawn and the guest's READY marker). A user runs minvmd config set --ram-mib 8192 inside that window. → The child boots the VM with the old RAM value, but the parent re-resolves the config after READY and records the new value as State.booted_ram_mib; minvmd status then reports a RAM size the running VM does not have, silencing or fabricating any reasoning built on the booted values (R2.6).

Also: the child resolved vcpus and ram_mib as two separate config.toml reads — a config set between them boots a VM with vcpus from one config version and RAM from another.

Fix: the parent resolves the pair once, pre-spawn, hands it to the child via MINVMD_BOOTED_VCPUS/MINVMD_BOOTED_RAM_MIB, and persists that same snapshot at the Running transition. The child boots from the snapshot (local resolution only as a mixed-version fallback). Both spawn sites (run, boot) pass it.

2. A malformed config.toml broke the one command that could diagnose it

Failure state: config.toml contains invalid TOML (hand edit, partial write). → Boot silently falls back to defaults, but minvmd config show — the command a user would run to see what boot will use — exits with an error instead. Additionally, run_show's three independent config reads could label values with sources from a different file version under a concurrent config set.

Fix: config show now uses the same single-read resolution pass as boot (resolve_resources()): one snapshot supplies values and source labels, and a malformed file reports the same default fallback boot would use (with a warning on stderr).

3. Hand-edited invalid values rode straight into the VMM

Failure state: config.toml contains vcpus = 0 or ram_mib = 64 (values config set rejects — only the write path validated). → Boot resolution hands a 0-vcpu VmConfig to libkrun, or boots a guest with too little RAM to reach userspace, which then fails as an opaque READY timeout.

Fix: persisted values that config set would reject are treated as unset with a warning; boot falls back to the built-in defaults. Covered by new CLI integration tests driving the real binary against planted config.toml files.

4. atomic_write_toml was atomic but not durable

Failure state: A state/config write completes (config set prints saved:), then the host crashes before the directory entry reaches disk. → The rename lives in the parent directory, which was never fsynced, so the old file resurfaces after reboot — an acknowledged write is lost.

Fix: fsync the parent directory after the rename.

5. Lifecycle proof erased the developer's real config (script)

Failure state: A developer with persisted VM settings runs scripts/minvmd-lifecycle.sh locally. → The EXIT-trap teardown rm -fs the default provider's config.toml unconditionally, permanently deleting their real resource configuration. Separately, an ambient MINVMD_VM_RAM_MIB in the caller's shell out-ranks the persisted layer and fails the proof's ram_mib_source == "config" assertion for reasons unrelated to the code under proof.

Fix: back up any pre-existing config.toml into the proof's scratch dir and restore it on exit (CI runners have none, so teardown still just drops the proof's own file); unset MINVMD_VM_RAM_MIB MINVMD_VM_VCPUS before the assertions.

Bonus: tracing diagnostics corrupted piped JSON

Failure state (exposed by the new tests): any code path emits a tracing warning while minvmd config show --json or minvmd status --json runs (e.g. the malformed-config fallback). → fmt::layer() wrote to stdout, so the warning lands in front of the JSON document and minvmd … --json | jq fails to parse. In run --detach, diagnostics went to /dev/null (stdout) instead of the run.log that captures stderr as the documented boot-failure diagnosis channel.

Fix: route the tracing subscriber to stderr.

Not addressed (from the same review)

  • The suggestion to lower the CLI test's RAM value to 512 MiB rests on a wrong premise — over-allocation warns, it never rejects, so 3072 passes on any runner.
  • The standalone doc findings (stale reactive-warning user story, undefined R9.1 anchors) are left for a separate docs pass; this PR only updates spec prose the code changes made stale.

Testing

  • cargo test -p minvmd — 127 lib + 6 CLI-integration tests pass (3 new resolution/sanitize unit tests; 2 new CLI tests for malformed / hand-edited-invalid config.toml).
  • cargo clippy -p minvmd --all-targets -- -D warnings clean; cargo fmt applied; bash -n on the script.
  • The snapshot handoff on a real boot needs the gated VM lanes (MINVMD_E2E=1 / scripts/minvmd-lifecycle.sh), which CI exercises on the KVM lane.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • VM boots now use a consistent snapshot of configured CPU and memory resources throughout startup.
    • config show reports the resolved resource values and whether they come from environment settings, saved configuration, or defaults.
  • Bug Fixes

    • Invalid or malformed saved resource settings now safely fall back to defaults.
    • Runtime status remains aligned with the resources used at boot.
    • Diagnostic output no longer interferes with JSON command results.
  • Reliability

    • Configuration state is written more durably to reduce data loss after interruptions.

norrietaylor and others added 4 commits July 16, 2026 12:17
fmt::layer() defaults to stdout, where diagnostics corrupt the
machine-readable surfaces: a warning emitted during `config show --json`
or `status --json` lands in front of the JSON document, so a pipe
consumer's parse fails. In `run --detach` the supervisor redirects child
stderr to run.log as the boot-failure diagnosis channel — with tracing
on stdout those diagnostics went to /dev/null instead.

Refs: #775

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… VMM child

Three related gaps in the #747 resource surface, from the post-merge
CodeRabbit review of #775:

- The parent recorded State.booted_* by re-resolving the config after
  the guest reached READY, while the child had resolved it independently
  at spawn — a `config set` landing in the boot window made `status`
  report resources the VM did not boot with. The parent now resolves
  the pair once pre-spawn, hands it to the child via MINVMD_BOOTED_*,
  and persists that same snapshot at the Running transition. A missing
  snapshot (older parent binary) falls back to local resolution.
- The child's two separate effective_* calls could tear vcpus and
  ram_mib across two config.toml versions; resolution is now a single
  read (resolve_resources), shared by boot, status, and `config show`,
  which also reports value sources from that one snapshot instead of
  three independent reads — and no longer fails on a malformed
  config.toml that boot would silently fall back from.
- Boot resolution trusted persisted values that `config set` would
  reject: a hand-edited vcpus = 0 or sub-floor ram_mib went straight
  into VmConfig. Such values are now treated as unset with a warning.

Refs: #775

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tmp-file fsync persists the file contents, but the rename lives in
the directory: after a crash between the rename and the directory entry
reaching disk, the old state/config file resurfaces even though the
caller was told the write succeeded.

Refs: #775

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…proof

Teardown deleted the default provider's config.toml unconditionally, so
running the proof on a developer machine erased the user's persisted VM
resource settings. Back up any pre-existing file into the proof's
scratch dir and restore it on exit; CI runners have none, so teardown
still drops the proof's own config. Also unset MINVMD_VM_RAM_MIB/VCPUS
up front: an ambient override out-ranks the persisted layer and would
fail the ram_mib_source == "config" assertion for reasons unrelated to
the code under proof.

Refs: #775

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 60b17d5a-b37d-4030-b808-34ae5bf2a313

📥 Commits

Reviewing files that changed from the base of the PR and between 9bcb073 and a7d290d.

📒 Files selected for processing (1)
  • crates/minvmd/src/cmd/mod.rs
📝 Walkthrough

Walkthrough

Changes

Resource resolution and boot snapshots

Layer / File(s) Summary
Resource resolution and validation
crates/minvmd/src/cmd/mod.rs, crates/minvmd/src/cmd/config.rs, crates/minvmd/tests/*, docs/specs/09-spec-minvmd-resource-monitoring/*
Persisted values are sanitized, vCPU/RAM are resolved together with source tracking, and malformed or invalid configuration falls back to defaults.
Pre-spawn resource handoff
crates/minvmd/src/cmd/{boot,run,vmm_child}.rs, docs/specs/09-spec-minvmd-resource-monitoring/*
Boot and run pass pre-spawn resource snapshots to the VMM child and record the same values in runtime state.
Durable writes and lifecycle diagnostics
crates/minvmd/src/{state,main,config}.rs, scripts/minvmd-lifecycle.sh
TOML replacement fsyncs the parent directory, diagnostics use stderr, and lifecycle checks preserve provider configuration while clearing resource overrides.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

Possibly related PRs

  • gominimal/minimal#775 — Shares the minvmd resource-resolution and booted-state pipeline, with this PR adding the pre-spawn environment snapshot handoff.

Suggested reviewers: twitchyliquid64

Poem

A rabbit packed two values tight,
vCPUs and RAM for booting right.
Through env vars they safely hop,
While state records each snapshot.
Configs heal, logs drift to stderr—
Durable hops make systems steadier.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main theme of the change: hardening minvmd's resource surface.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/minvmd/src/cmd/mod.rs`:
- Around line 164-198: Update sanitize_persisted to validate persisted vcpus
against the host ceiling returned by max_vm_vcpus(...), clearing cfg.vcpus and
warning when it exceeds that limit so boot resolution uses the default value and
source instead of clamping it. Inject or calculate the ceiling through the
existing configuration path, and add coverage verifying an oversized persisted
value resolves to the default.

In
`@docs/specs/09-spec-minvmd-resource-monitoring/09-spec-minvmd-resource-monitoring.md`:
- Around line 189-198: Update requirement R2.2 to state that status and
vmm_child consume the pre-spawn resource-resolution snapshot defined by
R2.5–R2.6, rather than the current resolution directly; retain the existing
single-pass resolution and effective_resources requirements.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9414988a-6114-4734-9a21-dfdb49b9bd23

📥 Commits

Reviewing files that changed from the base of the PR and between 80fa818 and 9bcb073.

📒 Files selected for processing (12)
  • crates/minvmd/src/cmd/boot.rs
  • crates/minvmd/src/cmd/config.rs
  • crates/minvmd/src/cmd/mod.rs
  • crates/minvmd/src/cmd/run.rs
  • crates/minvmd/src/cmd/vmm_child.rs
  • crates/minvmd/src/config.rs
  • crates/minvmd/src/main.rs
  • crates/minvmd/src/state.rs
  • crates/minvmd/tests/config_cli_integration.rs
  • docs/specs/09-spec-minvmd-resource-monitoring/09-spec-minvmd-resource-monitoring.md
  • docs/specs/09-spec-minvmd-resource-monitoring/architecture.md
  • scripts/minvmd-lifecycle.sh

Comment on lines +164 to +198
/// the default, so an unreadable file never blocks boot (R9.7). Values `config
/// set` would reject are sanitized out (see [`sanitize_persisted`]).
pub(crate) fn persisted_resource_config() -> crate::config::ResourceConfig {
match crate::config::ResourceConfig::read(&crate::state::provider_dir()) {
let cfg = match crate::config::ResourceConfig::read(&crate::state::provider_dir()) {
Ok(cfg) => cfg,
Err(e) => {
tracing::warn!(error = %e, "failed to read persisted resource config; using defaults");
crate::config::ResourceConfig::default()
}
};
sanitize_persisted(cfg)
}

/// Drop persisted values that `config set` would reject — it validates only
/// the write path, so a hand-edited `config.toml` can carry `vcpus = 0` or a
/// sub-floor `ram_mib` that boot resolution would otherwise hand straight to
/// the VMM (a 0-vcpu VmConfig, or a guest that cannot reach userspace). Each
/// offending field is warned about and treated as unset, falling back to the
/// built-in default.
fn sanitize_persisted(mut cfg: crate::config::ResourceConfig) -> crate::config::ResourceConfig {
if cfg.vcpus == Some(0) {
tracing::warn!("persisted vcpus = 0 is invalid; falling back to the default");
cfg.vcpus = None;
}
if let Some(m) = cfg.ram_mib
&& m < config::MIN_RAM_MIB
{
tracing::warn!(
ram_mib = m,
min = config::MIN_RAM_MIB,
"persisted ram_mib is below the boot floor; falling back to the default"
);
cfg.ram_mib = None;
}
cfg

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Sanitize persisted vCPU values above the host ceiling.

A hand-edited vcpus above max_vm_vcpus(...) is rejected by config set, but this path retains it, clamps it, and reports "config" rather than falling back to the default as documented.

Please inject or calculate the ceiling here, clear the field when exceeded, and add coverage for the resulting default value/source.

Based on PR objectives, hand-edited invalid persisted values must fall back to defaults with warnings.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/minvmd/src/cmd/mod.rs` around lines 164 - 198, Update
sanitize_persisted to validate persisted vcpus against the host ceiling returned
by max_vm_vcpus(...), clearing cfg.vcpus and warning when it exceeds that limit
so boot resolution uses the default value and source instead of clamping it.
Inject or calculate the ceiling through the existing configuration path, and add
coverage verifying an oversized persisted value resolves to the default.

On the stock-Linux lane (libkrun feature on, library not found) the
test build compiled booted_resources_from_env via the cfg(test) arm
with no caller — vmm_child's real impl, its only caller, is configured
out — so clippy's -D dead-code failed CI. Gate it to minvmd_libkrun;
parse_booted_snapshot stays test-visible since the unit tests drive it
directly.

Refs: #793

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant