Skip to content

docs(arch-vm-ext4-volume): pkg cache on the in-VM ext4 volume (volume-aware) - #674

Merged
norrietaylor merged 3 commits into
mainfrom
arch/vm-ext4-volume-e853f21e6bab3fb5
Jul 8, 2026
Merged

docs(arch-vm-ext4-volume): pkg cache on the in-VM ext4 volume (volume-aware)#674
norrietaylor merged 3 commits into
mainfrom
arch/vm-ext4-volume-e853f21e6bab3fb5

Conversation

@gominimal-aw-bot

@gominimal-aw-bot gominimal-aw-bot Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Architecture for #583: per-VM writable ext4 /dev/vdb volume.

Chosen approach

Two-disk RAW topology: /dev/vda (base rootfs, read-only, shared, unchanged) and /dev/vdb (per-VM writable ext4, carries minimal_state_dir = /var/lib/minimal). Host creates a sparse raw file (ensure_sparse_raw); guest formats on first boot via mkfs.ext4 keyed on superblock detection (idempotent, platform-portable). Clean shutdown driven by minvmd stop invoking the Shutdown RPC before SIGTERM; failed volume mount is loud (MOUNT_FAILED marker, no silent fallback).

ADR promoted

None — all decisions are feature-scoped (minvmd / minimald / sessions crates only).

Assumption ledger

All six load-bearing assumptions are settled; no needs-spike residue. See docs/specs/08-spec-vm-ext4-volume/architecture.md for the full ledger.

Next step

Merging this PR advances tracking issue #583 to the plan phase, where one Unit sub-issue per demoable unit is previewed. Merging closes the architecture sub-issue (Closes #<architecture-sub-issue> is added to this body by sdd-pr-sanitize).

Generated for issue #583

Generated by sdd-triage-arch for issue #583 ·

Closes #673

Summary by CodeRabbit

  • New Features
    • Added a per-VM persistent writable ext4 data volume, replacing the previous per-boot temporary approach.
    • Strengthened session and provider indexing so state can self-recover after corrupted session data.
    • Improved workspace uploads with safer staging behavior and atomic swaps when forcing updates.
  • Bug Fixes
    • Enforced clearer, loud failures if the data volume cannot be mounted (no silent fallback), improving overall consistency.
    • Improved shutdown quiescing to better preserve state.
  • Documentation
    • Added an architecture specification covering storage layout, boot/shutdown flow, and recovery expectations.

Design the architecture for #583: two-disk RAW topology (/dev/vda RO,
/dev/vdb writable ext4), host sparse-raw provisioning, guest first-boot
mkfs, shutdown quiesce, crash safety, session persistence, and provider
index. Records FFI surface (krun_add_disk3, SyncMode), crate-level
interface changes across minvmd/minimald/sessions, and a fully-settled
assumption ledger with no needs-spike residue.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 41d715ea-5cd3-4cd3-a97e-bede524752ac

📥 Commits

Reviewing files that changed from the base of the PR and between 15c29e1 and e64f92a.

📒 Files selected for processing (1)
  • docs/specs/08-spec-vm-ext4-volume/architecture.md
✅ Files skipped from review due to trivial changes (1)
  • docs/specs/08-spec-vm-ext4-volume/architecture.md

📝 Walkthrough

Walkthrough

This PR adds an architecture spec for a per-VM writable ext4 volume on /dev/vdb, covering host provisioning, guest mount and shutdown behavior, session-store recovery, workspace upload semantics, provider indexing, and design alternatives.

Changes

Architecture Specification Document

Layer / File(s) Summary
Intro and chosen approach
docs/specs/08-spec-vm-ext4-volume/architecture.md
Front matter and the two-disk layout with per-VM writable ext4 state on /dev/vdb, plus mount and failure semantics.
Host FFI and volume provisioning
docs/specs/08-spec-vm-ext4-volume/architecture.md
Adds the host-side FFI, sparse image helpers, path resolution, and VmConfig wiring for the data disk.
Guest mount, quiesce, and boot gating
docs/specs/08-spec-vm-ext4-volume/architecture.md
Defines guest mount detection, failure handling, shutdown marker emission, READY gating, and the in-VM state paths.
Shutdown, stop command, and session store resilience
docs/specs/08-spec-vm-ext4-volume/architecture.md
Specifies shutdown quiescing, stop-command timing, and corrupt session-store recovery.
Provider reset and workspace upload
docs/specs/08-spec-vm-ext4-volume/architecture.md
Updates provider reset timing and workspace upload behavior for force and non-force paths.
Provider index and rootfs closure
docs/specs/08-spec-vm-ext4-volume/architecture.md
Introduces the session-index concept, lifecycle event streaming, and rootfs build closure updates.
Alternatives and assumptions
docs/specs/08-spec-vm-ext4-volume/architecture.md
Lists rejected alternatives, assumptions, and knowledge gaps.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related issues

Possibly related PRs

  • gominimal/pkgs#365: Adds guest tooling and rootfs support aligned with the ext4-on-/dev/vdb volume requirements described here.

Suggested reviewers: norrietaylor

Poem

A rabbit packed a disk so neat,
With /dev/vdb for burrowed feet.
No tmpfs sighs, no silent fall,
Just ext4 dreams inside the wall.
🐇✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately describes the main change: a volume-aware in-VM ext4 cache architecture.
Linked Issues check ✅ Passed The spec matches #673 by documenting the architecture for moving the pkg cache onto a per-VM writable ext4 volume.
Out of Scope Changes check ✅ Passed No clear out-of-scope code changes are present; the PR is confined to the architecture spec for the ext4 volume work.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Comment @coderabbitai help to get the list of available commands.

@gominimal-aw-bot

This comment has been minimized.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/specs/08-spec-vm-ext4-volume/architecture.md`:
- Around line 110-116: In the mount_state_volume function, modify the logic that
checks for the ext4 magic word at byte offset 1080. Currently the function calls
mkfs.ext4 when the signature is "absent or corrupt", but this should be narrowed
to only call mkfs.ext4 when the signature is truly absent. If the ext4 signature
is already present (indicating a valid ext4 filesystem), the function should
attempt to mount or repair the volume instead of reformatting it. This prevents
wiping session data on persistent session disks after an unclean shutdown.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7ca6334b-d1dc-4059-abb2-7e54ca419c53

📥 Commits

Reviewing files that changed from the base of the PR and between 248f869 and 12a9bdb.

📒 Files selected for processing (1)
  • docs/specs/08-spec-vm-ext4-volume/architecture.md

Comment thread docs/specs/08-spec-vm-ext4-volume/architecture.md
…ture

mount_state_volume mkfs'd on "absent or corrupt", which would wipe a
persistent session disk after an unclean shutdown. Narrow mkfs to the
truly-blank case (no ext4 magic); when a signature is present, mount and
let the journal replay, and on mount failure run e2fsck + retry, failing
closed rather than reformatting. Note e2fsck in the e2fsprogs closure.

Refs #583

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gominimal-aw-bot

Copy link
Copy Markdown
Contributor Author

sdd-validate — Architecture boundary findings

Boundary resolved: Architecture (PR adds docs/specs/08-spec-vm-ext4-volume/architecture.md under docs/specs/).

Gate set applied: Architecture gates 1–4.

Tracking issue: #583 (sdd:triage, full-path feature).


Gate 1 — Decision and rationale present

Result: Pass.

The "Chosen approach" section clearly states the chosen design (two-disk RAW virtio-blk topology: read-only shared /dev/vda + per-VM writable ext4 /dev/vdb) and the reasoning behind each element: the EXDEV hardlink constraint is resolved by construction; guest-side first-boot mkfs.ext4 keyed on superblock detection is idempotent and platform-portable (macOS has no mke2fs); loud failure on /dev/vdb mount is required once user data lives on the volume. Decision and rationale are both present.

Gate 2 — Alternatives considered

Result: Pass.

The "Alternatives considered" section enumerates five alternatives, each with a named rejection rationale grounded in concrete evidence:

  • qcow2 data disk / backing-file overlay — rejected: no Qcow2 DiskFormat variant exists (raw.rs:40-44); no host-side overlay-creation path in libkrun; metadata-corruption surface on hard-kill.
  • VolumeProvisioner trait abstraction — rejected (YAGNI): Phase-2 reflink provisioner is a pure host-side strategy swap with no guest/boot-path change; three speculative provisioners do not justify a trait.
  • Host-side mkfs.ext4 before disk attach — rejected: macOS has no mke2fs/mkfs.ext4.
  • Silent tmpfs fallback on mount failure — rejected: session worktrees are user data with no host copy; a ghost READY on a fallback volume is a data-loss posture.
  • full sync mode as default — rejected: macOS/HVF benchmarks (Proof Artifact 4, 2026-07-07) show full is ≈12× slower than relaxed (3.1 GB/s vs 254 MB/s) while relaxed still honours VIRTIO_BLK_F_FLUSH.

Gate 3 — Consistent with existing decisions

Result: Pass.

No decisions/ directory exists in this repository, so there are no accepted ADRs to contradict. The document references ADR-0001 (error-handling strategy) and its application is consistent with it — VolumeError in volume.rs uses thiserror (library crate); stop.rs uses anyhow context (application crate) — and with the coding standards in CLAUDE.md.

Gate 4 — No implementation detail masquerading as a decision

Result: Warning.

The "Chosen approach" and "Alternatives considered" sections capture genuine cross-cutting decisions (disk topology, provisioning strategy, failure-mode semantics, sync-mode default). The "Data and interface changes" section, however, intermingles interface-level design with task-level specifics that sit below the cross-cutting threshold for an architecture record:

  • Specific Rust function signatures with full parameter lists and return types (e.g. krun_add_disk3, mount_state_volume, quiesce_state_volume).
  • File:line-number references into the current codebase (raw.rs:39, main.rs:302-305, store.rs:353-367, rpc.rs:384-413).
  • Byte-offset constants (0x53EF at offset 1080) and shell-out command flags (mkfs.ext4 -F, e2fsck -p).

These are not coding preferences dressed as architecture decisions (the gate's core concern), and their presence does not obscure the architectural choices. They are more naturally home in a task sub-issue's "files in scope" or implementation notes. Worth trimming in future architecture records to keep the cross-cutting signal clear. No hand-off required.


Summary

Gate Result
1 — Decision and rationale present ✅ Pass
2 — Alternatives considered ✅ Pass
3 — Consistent with existing decisions ✅ Pass
4 — No implementation detail masquerading as a decision ⚠️ Warning

No Blockers. No needs-human applied. No lifecycle label move (architecture boundary; lifecycle moves are at the implementation boundary only).

Generated by sdd-validate for issue #674 ·

Generated by sdd-validate for issue #674 ·

@norrietaylor
norrietaylor enabled auto-merge (squash) July 8, 2026 19:41
…ning

Downgrade `lifecycle-vsock-persistent` from settled to needs-spike: a
held-open guest->host lifecycle socket collides with the host->guest SSH
bridge under libkrun's concurrent-vsock wedge (#588) — the same failure
that red-lit autospawn-e2e on #672. Record the spike under knowledge gaps.

Fold #672's empirical mkfs hardening into the guest-boot section: the
undersize guard, the trailing margin for libkrun's backing-file trailer
shave, and eager inode/journal init that avoids a lazy-init storm.

Refs: #583, #588, #672
@norrietaylor
norrietaylor merged commit a53dd2a into main Jul 8, 2026
3 checks passed
@norrietaylor
norrietaylor deleted the arch/vm-ext4-volume-e853f21e6bab3fb5 branch July 8, 2026 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

architecture: pkg cache on the in-VM ext4 volume (volume-aware)

1 participant