Skip to content

docs(arch-minvmd-networking-gvproxy): minvmd networking — gvproxy userspace net for VM sessions - #445

Merged
norrietaylor merged 3 commits into
mainfrom
arch/minvmd-networking-gvproxy-06efc65792cab481
Jun 17, 2026
Merged

docs(arch-minvmd-networking-gvproxy): minvmd networking — gvproxy userspace net for VM sessions#445
norrietaylor merged 3 commits into
mainfrom
arch/minvmd-networking-gvproxy-06efc65792cab481

Conversation

@gominimal-aw-bot

@gominimal-aw-bot gominimal-aw-bot Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Architecture for tracking issue #404.

Chosen approach

minvmd will spawn one gvproxy child process per VM, connected to the VM's virtio-net device via libkrun's krun_set_passt_fd API. The gvproxy child is owned by the VM supervisor and is reaped when the VM stops or crashes.

The mechanism uses a Unix socketpair to wire gvproxy to the guest's virtio-net device. The supervisor creates the socketpair, spawns gvproxy with one end (via --fd <n>), exports the other end's FD to the VMM child via MINVMD_NET_FD, and the VMM child calls krun_set_passt_fd(ctx, fd) before krun_start_enter.

TSI remains available as a fallback via MINVMD_NETMODE=tsi or automatic fallback when gvproxy is not on PATH.

The network allowlist enforcement hook is wired (check_network_policy) but its body is a no-op (policy default: open) for this spec.

Data and interface changes

  • New module: crates/minvmd/src/net.rs (NetworkMode, spawn_gvproxy, NetworkPolicy, check_network_policy)
  • State: gains gvproxy_pid: Option<u32>
  • VmConfig: gains net_mode field; apply() grows net_fd parameter and removes macOS gate
  • FFI: krun_set_passt_fd in krun/raw.rs + safe wrapper set_passt_fd in krun/ctx.rs
  • run_foreground: spawns gvproxy, reaps it on all exit paths

Alternatives considered

  • gvproxy as sibling process: rejected (orphan risk on supervisor crash)
  • TSI exclusively: rejected (~62 connection cap vs 200 target, informed by minimal#204)
  • tap devices / vmnet: rejected (requires privileges)

Knowledge gaps

All assumptions are settled (no spikes required).

  • Platform parity confirmed: krun_set_passt_fd is platform-agnostic (informed by arch-minvmd-linux-kvm)
  • Process model established: reusing supervisor-spawns-child pattern (informed by arch-minvmd-host-daemon)
  • Network allowlist enforcement deferred: deliberate phasing decision per spec's Open Questions

Next step

Merging this PR will close the architecture sub-issue and advance the tracking issue to phase B (Unit decomposition and plan comment).

Related: minimal#311 (macOS minvmd), minimal#396 (Linux minvmd), minimal#204 (TSI EMFILE root cause)

Generated by sdd-triage for issue #404 · sonnet45 1.9M ·

Closes #444

Summary by CodeRabbit

Release Notes

  • Documentation
    • Added a new architecture specification for gvproxy-backed userspace networking per VM
    • Documented network mode selection (including fallback behavior when gvproxy is unavailable)
    • Described the startup handoff flow, required lifecycle/cleanup behavior, and state persistence
    • Included integration and verification expectations for the new networking approach

…erspace networking

Add architecture document for minimal#404 (minvmd networking — gvproxy
userspace net for VM sessions).

## Chosen approach

- minvmd spawns one gvproxy child per VM, owned by the supervisor
- Unix socketpair wires gvproxy to the VM's virtio-net device via
  libkrun's krun_set_passt_fd API
- TSI remains selectable as fallback via MINVMD_NETMODE=tsi
- Network allowlist enforcement hook wired (no-op for this spec)

## Data and interface changes

- New module: crates/minvmd/src/net.rs (NetworkMode, spawn_gvproxy,
  NetworkPolicy, check_network_policy)
- State gains gvproxy_pid: Option<u32>
- VmConfig gains net_mode field and apply() grows net_fd parameter
- New FFI: krun_set_passt_fd in krun/raw.rs + safe wrapper in krun/ctx.rs
- run_foreground spawns gvproxy and reaps it on all exit paths

## Alternatives considered

- gvproxy as sibling process (rejected: orphan risk)
- TSI exclusively (rejected: ~62 connection cap vs 200 target)
- tap devices / vmnet (rejected: requires privileges)

All assumptions settled (no spikes). Platform-agnostic design (macOS + Linux).

Related: minimal#311 (macOS minvmd), minimal#396 (Linux minvmd), minimal#204 (TSI EMFILE)
Informed by arch-minvmd-host-daemon, arch-minvmd-linux-kvm

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8486375a-d7b1-4927-9df0-f8904fff9030

📥 Commits

Reviewing files that changed from the base of the PR and between 9b23f52 and 70d6925.

📒 Files selected for processing (1)
  • docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
✅ Files skipped from review due to trivial changes (1)
  • docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md

📝 Walkthrough

Walkthrough

Adds a new architecture specification document (docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md) defining per-VM gvproxy-backed userspace networking for minvmd. The spec covers NetworkMode selection, Unix socketpair FD handoff via MINVMD_NET_FD, krun_set_passt_fd FFI, supervisor lifecycle/cleanup, network policy hook, rejected alternatives, assumptions, and verification steps.

Changes

minvmd gvproxy networking architecture spec

Layer / File(s) Summary
Chosen approach and NetworkMode selection
docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
Front-matter and high-level design: per-VM gvproxy spawning, socketpair FD handoff via MINVMD_NET_FD, NetworkMode (GvProxy vs Tsi) selection via MINVMD_NETMODE with TSI fallback, and boot-path network policy hook placement.
Implementation surface: net.rs, supervisor wiring, libkrun FFI, VMM config
docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
Specifies the new net.rs module, cmd/run.rs socketpair/env/FD/cleanup changes, gvproxy_pid supervisor state serialization, krun_set_passt_fd raw/ctx FFI APIs, conditional set_passt_fd in VmConfig.apply(), VMM child MINVMD_NET_FD parsing, and removal of the macOS/Linux platform gate.
Rejected alternatives
docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
Documents three rejected alternatives (gvproxy as separate process tree, TSI-only, tap/vmnet) with listed constraints and the selection rationale.
Assumptions, knowledge gaps, and deferred items
docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
Assumption ledger, constraints, thin areas, and explicitly deferred items: policy enforcement details, DNS/port-forwarding, and Linux namespace sandbox integration uncertainty.
Verification steps and test expectations
docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
Verification commands including MINVMD_NETMODE=tsi fallback behavior and architecture tracking/issue linkage expectations.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related issues

  • minimal#404: This PR is the architecture specification document directly implementing the design scope described in that tracking issue — per-VM gvproxy spawning, TSI fallback, lifecycle management, and the network policy enforcement hook.
  • minimal#403: The architecture document specifies the design and implementation details for the userspace networking feature tracked in that issue.
  • inbox#225 / inbox#242: The PR's architecture spec directly addresses the gvproxy-backed userspace networking for minvmd VMs described in those inbox issues, including the same lifecycle, policy hook, and FFI integration scope.

Possibly related PRs

  • gominimal/minimal#405: Both PRs touch VmConfig::apply() — this spec describes removing macOS-only platform gating and conditionally calling set_passt_fd based on MINVMD_NET_FD, which is what that PR implements.
  • gominimal/minimal#407: Both PRs are part of the same minvmd gvproxy networking documentation effort covering socketpair/MINVMD_NET_FD/krun_set_passt_fd/NetworkMode/fallback lifecycle.

Suggested reviewers

  • norrietaylor

Poem

🐇 Hoppity-hop through the network stack,
A socketpair handed, no orphan comes back.
gvproxy wakes up, eth0 takes a bow,
TSI waits on standby — just in case, meow.
The spec is now written, the design is clear,
Two hundred connects? No EMFILE to fear! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding a minvmd networking architecture spec for gvproxy-backed userspace networking per VM.
Linked Issues check ✅ Passed The PR comprehensively addresses issue #444 (architecture deliverable for #404) with detailed specifications for gvproxy networking, lifecycle management, fallback behavior, and network policy enforcement.
Out of Scope Changes check ✅ Passed The PR contains only documentation changes to architecture.md, all directly related to specifying gvproxy-backed networking as required by the linked issue.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md (1)

177-184: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Verify test artifact existence and clarify issue-tracking language.

Lines 180–182 reference a test binary minimald_session_e2e, but the spec does not indicate whether this test already exists in the codebase or is expected to be created as part of this PR (or a follow-up). Since this is an architecture-only PR, the test likely does not exist yet and will be added in a later implementation PR; the spec should clarify this expectation. Additionally, line 182 states "The architecture sub-issue is created as a child of tracking issue minimal#404," but this document is itself an architecture specification document, not an issue tracker entry. The phrasing should be adjusted to reflect the actual artifact (e.g., "This architecture document is linked from minimal#404 and will remain open for follow-up implementation PRs").

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md` around lines
177 - 184, In the Verification section, clarify that the minimald_session_e2e
test binary referenced in lines 180-182 does not yet exist and will be created
as part of a future implementation PR (since this is an architecture-only
specification). Additionally, replace the statement in line 182 that says "The
architecture sub-issue is created as a child of tracking issue minimal#404" with
language that accurately reflects that this is an architecture specification
document (not an issue) that is linked from or referenced in the tracking issue
minimal#404, and will remain open for follow-up implementation work.
🧹 Nitpick comments (1)
docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md (1)

91-91: 💤 Low value

Fix capitalization of "macOS" in code references.

Per capitalization conventions, the operating system should be written as "macOS" (not "macos"), even in code-quoted contexts where the actual code uses lowercase. On lines 91 and 97, the capitalization should be corrected in the surrounding prose. For example, line 91 should read "…removes the macOS gate…" (not "…removes the #[cfg(target_os = "macos")] gate…").

However, note that when citing actual code symbols like #[cfg(target_os = "macos")], the exact code spelling is correct; the issue applies to the English text descriptions.

Also applies to: 97-97

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md` at line 91,
Correct the capitalization of the operating system name in the English prose
descriptions on lines 91 and 97. Change references from "macos" to "macOS" when
describing the operating system in your text (for example, "removes the macOS
gate" instead of "removes the macos gate"), but preserve the exact lowercase
spelling when citing actual code symbols like #[cfg(target_os = "macos")], which
should remain unchanged to match the actual source code.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md`:
- Line 38: The function signature check_network_policy references a PolicyError
type that is not defined in the net.rs module specification. Add the PolicyError
type definition to the net.rs module API section in the architecture.md file,
defining it as an enum or struct that includes at least one variant for
representing policy violation errors. If PolicyError is intended to be a type
alias or re-export from another module, explicitly document this relationship in
the module specification instead.
- Line 48: The specification at line 48 describing the spawn_gvproxy(gvproxy_fd)
call before the VMM child does not document the synchronization requirements
between gvproxy and VMM initialization. Add documentation clarifying whether the
supervisor must wait for gvproxy to signal readiness (such as by listening on
the socketpair FD) before spawning the VMM child, or whether they can be spawned
concurrently. Include explanation of how the synchronization prevents race
conditions where the VMM's virtio-net device might attempt to connect before
gvproxy is fully initialized, and specify any guarantees about initialization
order and timing.
- Line 47: The current specification at line 47 only documents clearing
FD_CLOEXEC on the gvproxy-side FD but omits the requirement for the VMM-side FD.
Add explicit documentation to clarify that FD_CLOEXEC must also be cleared on
the VMM-side FD in the supervisor before spawning the VMM child process. This
ensures the VMM-side FD survives the child's fork()+exec() boundary and prevents
unintended closure during the exec operation. The requirement should be stated
alongside the existing gvproxy-side FD_CLOEXEC clearing requirement.
- Around line 52-53: Expand the exit paths description to specify failure
handling semantics when calling gvproxy_child.kill() and gvproxy_child.wait(),
including how the supervisor should handle EINTR during reaping, whether there
are timeout constraints on wait operations, and what happens if reaping fails.
Additionally, provide a definition or reference for StartingGuard (e.g., cite
where it is defined in the codebase), explain its role as a RAII guard, and
clarify how its drop implementation integrates with the gvproxy reaping
requirement so implementers understand the complete lifecycle hook point.
- Around line 88-91: The specification for the conditional logic in the apply()
method is incomplete because it does not define the error handling behavior when
net_mode is set to NetworkMode::GvProxy but net_fd is None (indicating gvproxy
failed to spawn). Add documentation to the spec clarifying the intended behavior
in this scenario: specify whether apply() should treat this as a hard error that
causes the function to fail and log an error, log a warning and silently fall
back to TSI mode, or handle it in some other defined way. This clarification
should be added to the section describing the conditional branches in apply() to
prevent ambiguity during implementation and ensure consistent behavior across
platforms.

---

Outside diff comments:
In `@docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md`:
- Around line 177-184: In the Verification section, clarify that the
minimald_session_e2e test binary referenced in lines 180-182 does not yet exist
and will be created as part of a future implementation PR (since this is an
architecture-only specification). Additionally, replace the statement in line
182 that says "The architecture sub-issue is created as a child of tracking
issue minimal#404" with language that accurately reflects that this is an
architecture specification document (not an issue) that is linked from or
referenced in the tracking issue minimal#404, and will remain open for follow-up
implementation work.

---

Nitpick comments:
In `@docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md`:
- Line 91: Correct the capitalization of the operating system name in the
English prose descriptions on lines 91 and 97. Change references from "macos" to
"macOS" when describing the operating system in your text (for example, "removes
the macOS gate" instead of "removes the macos gate"), but preserve the exact
lowercase spelling when citing actual code symbols like #[cfg(target_os =
"macos")], which should remain unchanged to match the actual source code.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 54993300-6bb1-4333-9bc0-35efeb0c5622

📥 Commits

Reviewing files that changed from the base of the PR and between 99c0a81 and 9b23f52.

📒 Files selected for processing (1)
  • docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md

Comment thread docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
Comment thread docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md Outdated
Comment thread docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md Outdated
Comment thread docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md Outdated
Comment thread docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md
@gominimal-aw-bot

This comment has been minimized.

@norrietaylor

Copy link
Copy Markdown
Member

/revise address Code rabbit concerns. validate first

@norrietaylor

Copy link
Copy Markdown
Member

/revise address Code rabbit concerns. validate first

@gominimal-aw-bot gominimal-aw-bot Bot mentioned this pull request Jun 17, 2026
Address CodeRabbit review on the gvproxy networking architecture spec:

- Define PolicyError as a thiserror enum in the net.rs module API so
  the check_network_policy signature is complete and implementable.
- Document that FD_CLOEXEC must be cleared on both socketpair ends, not
  just the gvproxy side, so the VMM child inherits its FD across exec.
- State the gvproxy/VMM startup ordering: no readiness handshake is
  needed because the SOCK_STREAM socketpair is connected at creation.
- Specify gvproxy reaping semantics (best-effort kill+wait, EINTR
  retry, warn-not-abort) and cite StartingGuard in state.rs.
- Specify apply() behavior when GvProxy mode is selected with no FD:
  warn and fall back to TSI rather than hard-error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor

Copy link
Copy Markdown
Member

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@gominimal-aw-bot

This comment has been minimized.

@gominimal-aw-bot

Copy link
Copy Markdown
Contributor Author

Validation complete — Architecture boundary

Boundary: Architecture (adds docs/specs/03-spec-minvmd-networking-gvproxy/architecture.md)

Gate set applied: Architecture gates (4 gates)

Results

All gates passed. No findings.

Gate 1: Decision and rationale present

  • Chosen approach clearly stated (gvproxy child process per VM, wired via krun_set_passt_fd)
  • Rationale provided throughout, including process ownership model and TSI fallback strategy

Gate 2: Alternatives considered

  • Alternative A (gvproxy as sibling): rejected (orphan risk)
  • Alternative B (TSI exclusively): rejected (~62 connection cap vs 200 target)
  • Alternative C (tap/vmnet): rejected (privilege requirements, platform-specific)

Gate 3: Consistent with existing decisions

  • Correctly references arch-minvmd-host-daemon for process model (supervisor-spawns-child pattern)
  • Correctly references arch-minvmd-linux-kvm for platform-agnostic API design
  • No contradictions with existing ADR 0001 (error handling)

Gate 4: No implementation detail masquerading as a decision

  • Captures cross-cutting architectural choices (process lifecycle, platform abstraction, network transport selection)
  • "Data and interface changes" section appropriately documents API boundaries and module structure

The architecture record is ready for review.

Generated by sdd-validate for issue #445 · sonnet45 1.1M ·

@norrietaylor
norrietaylor merged commit b1ea627 into main Jun 17, 2026
14 of 17 checks passed
@norrietaylor
norrietaylor deleted the arch/minvmd-networking-gvproxy-06efc65792cab481 branch June 17, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

architecture: minvmd networking — gvproxy userspace net for VM sessions

1 participant