Skip to content

feat(minimald,minvmd,minimal2): end-to-end VM/guest/session networking integration - #581

Merged
norrietaylor merged 50 commits into
mainfrom
feat/networking-host-exposure
Jun 29, 2026
Merged

feat(minimald,minvmd,minimal2): end-to-end VM/guest/session networking integration#581
norrietaylor merged 50 commits into
mainfrom
feat/networking-host-exposure

Conversation

@norrietaylor

@norrietaylor norrietaylor commented Jun 26, 2026

Copy link
Copy Markdown
Member

End-to-end networking across the VM, guest-daemon, and session layers: a
session opened through minimal2 now gets working egress, managed DNS, an mTLS
reverse proxy, and static ingress, with the host-side proxies reachable on the
host loopback.

What this wires together

The networking primitives existed in pieces (gvproxy switch #561,
mTLS reverse proxy #554, static
ingress + session policy #556).
This connects them into one path that works from minimal2 down to an
in-sandbox shell.

VM layer (minvmd)

  • Spawns the one host gvproxy switch and registers a single vsock shuttle
    port (add_vsock_port2(.., listen=false), port 1024) that libkrun dials
    through to the host gvproxy -listen socket. Registered for every boot (not
    only own-ip), so the daemon's root-netns tap and each own-ip PTask tap can
    attach. There is no in-VM bridge: each tap opens its own vsock connection
    over that one port and relays its raw L2 frames to the host gvproxy, which is
    the switch (so two own-ip PTasks on the same host talk via the host switch).
  • Own-ip boot now hard-errors if the gvproxy binary is missing instead of
    silently booting a session with no network (MINVMD_GVPROXY_BIN); a
    present-but-unlaunchable binary only degrades a non-own-ip boot.
  • Guest RAM default is arch-conditional — 2048 MiB on x86_64 (hole-safe: a 4 GiB
    guest straddles the 32-bit MMIO hole and panics the kernel before /init),
    4096 on aarch64 — and overridable via MINVMD_VM_RAM_MIB; sized so the guest
    tmpfs rootfs does not hit StorageFull.

Guest layer (minimald)

  • Root-netns egress (guest.rs): brings up eth0, configures the interface
    over AF_INET ioctls (no ip dependency), installs resolv.conf, and attaches
    to the gvproxy switch over vsock — this is what gives the daemon itself egress
    (e.g. to clone an upstream repo during scaffold).
  • In-sandbox interactive session now works inside the microVM: switch_root
    via MS_MOVE+chroot(".") instead of a bare chroot, fixing the
    userns-vs-chroot EPERM (hakoniwa exit 125); devpts mounted with a ptmx
    instance; /tmp tmpfs; shell exec'd at /usr/bin/bash.
  • Own-ip ingress applied over the gvproxy control channel
    (ControlChannel::Unix / ::Vsock).
  • Host-side proxies started in-VM and published on the host loopback
    (server.rs): egress/DNS proxy (:7654) always, mTLS reverse proxy (:7655)
    under networking-proxy. In-VM they bind 0.0.0.0 (decoupled from eth0
    readiness) and only a port whose listener actually bound is exposed on the
    host via the gvproxy forwarder. DM2 (native) binds host loopback directly.

Session / CLI layer (minimal2, sessions)

  • activate --network {no-net|host-net|own-ip} and --ingress HOST:GUEST[/proto].
  • Session policy round-trips ingress mappings; validation rejects ill-formed
    policies at the boundary.

Networking abstraction (sandbox2)

Following review feedback, the per-sandbox network wiring lives behind a
sandbox2::Network trait (HostNet/NoNet built-ins + a GvproxyNetwork impl
for own-ip), configured on the sandbox Config and applied by sandbox2 itself,
so tasks — not only interactive sessions — get networking through one seam; a
task now runs under its session's network mode (a no-net session's task no longer
leaks host egress). Teardown of the switch attach is explicit. The shared switch
primitives (subnet/MAC/-config rendering) moved into a dedicated switch crate,
and the two same-named GvproxySwitch types were renamed GvproxySupervisor
(minvmd, owns the host gvproxy process) / SwitchClient (minimald, the
allocator + switch client) to make the per-model ownership clear. The gvproxy
process lifecycle stays in minimald (DM2) / minvmd (DM1) — sandbox2 only
wires an already-running switch into a sandbox netns.

Temporary scaffolding (call out)

crates/mctx/src/scaffold.rs is a temporary, self-contained copy of the
minimal.toml generator (generate_mfile / scaffold_default_mfile, packages
["base"]), used to scaffold a default minimal.toml for an empty session so an
attach does not fail with "minimal.toml not found". It is marked temporary in the
code. The intended end state is to share the real generator rather than copy it;
this copy avoids a larger refactor inside this integration PR and should be
removed when that refactor lands.

The scaffold clone runs inside the async session-actor; the blocking clone is
fenced with tokio::task::block_in_place on the daemon's multi-thread runtime
(guarded by runtime flavor so it falls back to a plain call rather than panicking
on a current-thread caller).

Test plan & status

Full plan + executable runner committed under
docs/specs/03-spec-networking/ (test-plan.md, test-plan.sh). Every
per-session assertion runs inside the sandbox via the full interactive session
path
(minimal2 attach <sess>, no -c); attach -c is never used because it
runs on the daemon host in the guest root netns and bypasses the sandbox.

Deployment Mode (DM) Key

  • DM1 — macOS (Apple Silicon) + Linux VMs (minimald in each VM, UDS-over-vsock)
  • DM2 — native Linux, a single minimald running directly on the host (no microVM), reachable over a UDS or TCP-localhost; multi-user hosts may run one per user
  • DM3 — native Linux + Linux VM(s) with minimald in each
  • DM4 — DM2 + DM3 combined (host minimald and VM minimald)
  • DM5 — any of the above, network-exposed with authN/authZ

Re-run on the pinned rootfs that now carries ip/nsenter (own-ip PTasks
launch). Verdicts are per deployment model: DM1 (macOS + VM, HVF), DM2
(native Linux, host-native daemon, no VM), DM3 (native Linux + VM, KVM).

TC DM1 (macOS+VM) DM2 (native, no VM) DM3 (Linux+VM)
TC1 no-net isolation PASS PASS PASS ²
TC1b host-net egress PASS PASS PASS ²
TC2 own-ip egress PASS FAIL ³ PASS ²
TC2 same-host peer BLOCKED ¹ FAIL ³ BLOCKED ¹
TC3 managed DNS :7654 BLOCKED ¹ FAIL ³ BLOCKED ¹
TC4 static ingress PARTIAL PARTIAL ³ PARTIAL ²
TC5 policy validation PASS PASS PASS
TC6 policy round-trip PASS PASS PASS
TC7 mTLS :7655 PARTIAL PARTIAL ³ PARTIAL ²
TC8 ssh-forward BLOCKED ¹ FAIL ³ BLOCKED ¹
TC9 mesh CLI PASS PASS PASS

DM1 is the authoritative macOS interactive run (evidence: TC1 no ifaces;
TC1b/TC2 in-sandbox curl→200; TC4/TC6 ingress mapping round-trips; TC5 all 3
rejected; TC7 no-cert→401, with-cert data path on ¹).

¹ Blocked by #588 — the
first own-ip attach per boot succeeds, every subsequent attach wedges. Hits
the microVM models (DM1/DM3) identically; the same BLOCKED/PARTIAL rows are
the own-ip data path (TC2-peer, TC3, TC4-data, TC7-data, TC8).

² DM3 shares DM1's microVM/own-ip path (minimald-in-VM over the vsock shuttle);
the first own-ip egress completes and the rest hit ¹, reproduced on Linux/KVM.
Not re-run TC-by-TC — identical guest/session code to DM1.

³ DM2 own-ip before this fix. A host-native daemon is unprivileged, so own-ip
failed three independent ways: tap setup EPERM (ip/nsenter don't inherit
setcap +ep); with setcap, sandbox uid_map EPERM (dumpable→SUID_DUMP_ROOT);
and no in-netns DNS (127.0.0.53 stub is unreachable). Fixed in the stacked
#589 (rootless in-process tap +
dumpable reset + gateway DNS): DM2 own-ip egress and curl example.com→200 now
pass. TC4/TC7 are PARTIAL here for the same reason as DM1 (CLI/policy half
passes; own-ip data path was the failing half).

⁴ DM2/DM3 CLI & policy rows (TC5/6/9) and the no-net / host-net sandbox (TC1/1b)
share the host-agnostic / shared sandbox path; the mTLS-401 and managed-DNS
proxies bind host loopback directly on DM2 (no gvproxy forwarder).

Known gap — own-ip PTask attach reliability (#588)

The earlier rootfs gap is resolved: .minimal/minimal.toml locked_commit is
pinned to a rootfs build that carries ip/nsenter, so move_tap_into_netns
succeeds, an own-ip PTask launches, and in-sandbox egress over the gvproxy shuttle
returns 200 (TC2).

The remaining own-ip data-path TCs are blocked by a separate reliability bug
(#588): the first own-ip
attach per boot succeeds, but every subsequent own-ip attach in the same boot
wedges (the vsock shuttle / switch per-tap state is not released on session
destroy). Because each data-path TC destroys and re-activates a fresh own-ip
session, only the first (TC2 egress) completes; TC2-peer, TC3, TC4-data,
TC7-data, and TC8 hang. This is not introduced by this PR — the vsock shuttle
landed in #561 — and is tracked
separately. The proxy infrastructure itself is verified (:7655 → 401, :7654
reachable on host loopback). The same attach issue is why own-ip tasks
currently fall back to HostNet.

Platform caveat

The full interactive test plan was run on macOS (HVF, DM1 / microVM path).
Linux/KVM (DM1 on Linux) is exercised in CI: the arch-conditional RAM fix
clears the x86_64 MMIO-hole boot panic, the minvmd-linux-kvm-e2e lane boots the
VM, and netns-integration exercises the Linux sandbox-netns path. DM2 (native
Linux, no VM) own-IP is delivered separately by the stacked
#589 (rootless host-native
own-IP), where it is verified end to end. The host-specific bit (mTLS cert dir)
is already OS-aware in the runner.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added minimal2 Activate networking controls: --network and repeatable --ingress EXT:INT[/PROTO] for static TCP/UDP mappings.
    • Introduced configurable “own-IP” guest networking with host gvproxy integration and guest egress support; micro-VM mode now applies it automatically.
  • Bug Fixes
    • Auto-scaffolds a default minimal.toml for newly attached empty session workspaces.
    • Improved guest rootfs prep (writable /tmp, better devpts/PTY handling) and interactive SSH attach (ssh -tt).
  • Documentation
    • Added a networking CLI test plan and companion script.

norrietaylor and others added 24 commits June 24, 2026 15:38
`minimal activate` hardcoded NetworkMode::default() (HostNet) with empty
policy, so NoNet/OwnIp and ingress port mappings had no CLI surface and were
reachable only via the netns proofs constructing a Record directly.

- `--network <no-net|host-net|own-ip>` (default host-net; no-flag behavior
  unchanged) via a local CliNetworkMode ValueEnum, keeping the sessions crate
  free of a clap dependency.
- `--ingress EXT:INT[/PROTO]` (repeatable; PROTO defaults tcp) parsed into
  sessions::PortMapping; non-tcp/udp and malformed specs rejected at parse time.
- Surface the daemon's typed CreateSession validation error (e.g. ingress on a
  non-own-ip session) instead of a generic failure line.

Daemon-side validate_policy is unchanged and remains the enforcement point.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0143kv2BRrRqGxmVwwHskQtS
On DM1/3/4 (a libkrun VM) gvproxy must run on the host, owned by
minvmd, not inside the guest where it has no host uplink (issue #572).
Wire up the previously-dead host-side switch:

- HostGvproxy: an RAII supervisor that spawns the host gvproxy switch
  (reusing GvproxyConfig::spawn) on a dedicated current-thread tokio
  runtime, so minvmd's synchronous boot/run supervisor can own an async
  switch lifecycle and tear it down on drop. Started with an empty
  static-lease table — the guest configures each PTask's switch IP
  statically, so minvmd need not own the per-PTask address book.
- run --foreground spawns + supervises the host gvproxy for an own-IP VM
  before booting the VMM child, so the -listen switch socket exists when
  libkrun dials it; the handle is held for the VM's lifetime.
- VmConfig::apply registers the per-PTask shuttle vsock bridge for an
  own-IP VM: add_vsock_port2(VSOCK_GVPROXY_SHUTTLE_PORT, switch_sock,
  listen = false) — the guest shuttle connects to AF_VSOCK CID 2 and
  libkrun splices it to the host gvproxy -listen socket, carrying raw L2
  frames (no second TCP/IP stack).
- net::shuttle: the shuttle vsock port + host switch-socket path
  resolver (placed beside the minimald bridge socket).
- image::resolve_gvproxy_path reads MINVMD_GVPROXY_BIN (fallback to the
  fixed install path); cmd::own_ip_requested reads MINVMD_VM_OWN_IP so
  the parent's gvproxy-spawn decision and the VMM child's VM network mode
  stay in lock-step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In a libkrun VM (DM1/3/4) minimald-in-guest must NOT spawn gvproxy
locally — gvproxy runs on the host (owned by minvmd) and the guest only
relays raw L2 frames to it (issue #572). Previously the DM2 spawn-local
path ran verbatim in the guest, so an OwnIp PTask got a 100.64 switch IP
but no egress (gvproxy NAT'd into a dead-end netns).

- net::SwitchTransport enum (LocalSpawn vs HostShuttle { cid, port })
  makes "spawn locally" and "relay to the host switch" mutually
  exclusive in the type system. GvproxySwitch::with_transport selects
  it; in HostShuttle mode attach() skips the local gvproxy spawn/config
  and only tracks the attach count.
- switch::attach_to_switch_vsock relays a PTask tap to the host gvproxy
  over AF_VSOCK (CID 2 : shuttle port); it shares a transport-agnostic
  spawn_relay with the DM2 UDS attach_to_switch — same HyperKit-framed
  raw-L2 relay, so exactly one gVisor stack stays in the path.
- session_host::attach_own_ip branches on the switch transport: DM2 uses
  the local control socket, DM1/3/4 the vsock shuttle. Static ingress on
  a VM own-IP PTask is skipped with a warning (gvproxy's port-forward API
  is host-side and unreachable over the frame-only shuttle); egress, the
  #572 focus, needs no such call.
- DM signal: the `vsock` listen-arg is exactly the libkrun-VM boundary
  (DM1/3/4) vs the UDS daemon (DM2); threaded through Config::in_microvm
  into the transport selection. DM2 behaviour is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`just up` materializes the guest kernel + rootfs, builds the
networking-enabled guest initramfs, builds minvmd (codesigned on
macOS), builds the minimal CLI, then runs `minimal ls` to autospawn
the supervised minvmd and reach minimald over the UDS bridge.

Recipes branch on $(uname -s):
- macOS: `minimal materialize`, Homebrew libkrun, codesign.
- Linux: fetch-artifact.sh / fetch-libkrun.sh with LIBKRUN_PREFIX +
  LD_LIBRARY_PATH exported so build.rs links the real implementation;
  no codesign; requires durable kvm group membership.

build-initramfs.sh gains a FEATURES env hook so the guest minimald is
compiled with networking-proxy,networking-wg (the HTTPS mTLS proxy and
WireGuard mesh peer); it previously passed no --features.

artifacts is idempotent (skips a present artifact) and clean removes
only the bring-up files, never the shared .scratch directory.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract `generate_mfile` from `minimal init` (cmd_init) into
mctx::scaffold and add `scaffold_default_mfile`, which resolves the
latest gominimal/pkgs @main commit and writes a default shell-stack
minimal.toml into a directory that has none.

minimald's session `context()` now scaffolds such a file when the
session workspace lacks one, so `attach` no longer fails with
"minimal.toml not found" on an empty workspace. Scaffolding is lazy
(on first attach / context build), not on activate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the issue #572 per-PTask vsock shuttle to the guest's root
network namespace, so minimald itself has egress (default route + DNS)
and can fetch the upstream pkgs repo — e.g. the clone that scaffolding
a session's minimal.toml performs.

minvmd:
- spawn the host gvproxy switch for every VM (best-effort: skip with a
  warning when the gvproxy binary is absent, so the boot/session e2e
  lanes still boot without egress), not just own-IP VMs.
- register the gvproxy shuttle vsock port unconditionally; the root tap
  and any own-IP PTask taps are all L2 clients on the one switch.

minimald:
- bring up a primary `eth0` tap in the root netns at boot, attached to
  the host gvproxy over the vsock shuttle. Configure address/up/default
  route via AF_INET ioctls (the generic rootfs ships no iproute2), and
  install /etc/resolv.conf by bind-mounting from the writable /run tmpfs
  (the rootfs is read-only). Hold the relay for the server's lifetime.
- reserve a daemon switch address (`broadcast - 2`) from the top of the
  subnet so the PTask range still starts at `network + 2`.

justfile: `up` fetches gvproxy and exports MINVMD_GVPROXY_BIN so the
host switch comes up; gvproxy fetch is idempotent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ary copy

The session-workspace scaffolding is a stop-gap for the workspace-upload
gap, so keep it isolated rather than refactoring shared code. Revert the
extraction of `generate_mfile` from `minimal::cmd_init` (cmd_init keeps
its own copy) and instead duplicate the generator inside mctx::scaffold,
marked TEMPORARY throughout. Deleting one module then removes the
workaround cleanly, with no cmd_init unpicking.

mctx now exports only `scaffold_default_mfile`; `generate_mfile` and its
constants are private to the module.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… on DM1

Unify the forwarder API behind a `ControlChannel { Unix | Vsock }`:
- DM2 keeps the local unix control socket.
- DM1/3/4 reach the host gvproxy's `-listen` socket over the same vsock
  shuttle port the L2 `/connect` relay uses (it serves
  `/services/forwarder/*` too), so `post_json` rides vsock there.

Replaces the prior "skip ingress with a warning on the VM path" branch in
`attach_own_ip`: own-IP PTasks in a libkrun VM now get their static
ingress forwards published on the host loopback (gvproxy binds
127.0.0.1:ext -> ptask-switch-ip:int), torn down on session end.

`post_json` is refactored into a transport-generic `exchange` over any
AsyncRead+AsyncWrite stream, shared by the unix and vsock channels.

Verification of the live data path is pending a scriptable interactive
attach: `attach -c` runs the exec path (daemon host netns) and never
mints the own-IP session host, so it cannot exercise this path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…HG1)

Interactive `minimal2 attach` panicked the daemon and never delivered a
working shell. Root causes, fixed here:

- rustls had no process-default CryptoProvider, so the first rustls user
  on the attach path (the remote-cache HTTPS client during a session
  build) panicked ("no process-level CryptoProvider"), killing the
  session actor; the caller then panicked at session.rs:398 on the lost
  reply. Install ring as the default provider at minimald startup, and
  make `rustls` a normal dep (it is always linked via reqwest anyway).
- The session actor dropping its reply now surfaces as an AttachError
  instead of `expect`-panicking the worker.
- `minimal2 attach` (interactive) requested no PTY, so the daemon's
  shell_request rejected it whenever stdin was not a tty (automation).
  Force one with `ssh -tt` for the interactive path.
- The read-only guest rootfs left `/tmp` unwritable, so remote-cache
  staging (tempfile) failed with EROFS; mount a tmpfs at `/tmp`.

With these, attach reaches the in-sandbox session build. Remaining
blocker is cache capacity (the session package set overflows the
tmpfs cache) — the seeded-cache-disk work, tracked separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two in-sandbox-shell bring-up fixes uncovered while exercising the
interactive attach path:

- Bump the microVM from 1024 to 4096 MiB. The in-VM session build stages
  packages into a RAM-backed tmpfs cache (/run/minimal/cache); 1024 MiB
  overflowed (StorageFull) unpacking large packages. Stop-gap until the
  persistent seeded cache disk lands.
- Exec the session shell at /usr/bin/bash, not /bin/bash: the bash
  package installs to --prefix=/usr and the generic rootfs has no
  /bin/bash, so the old path failed with ENOENT.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The interactive session shell's server-side `Pty::open` does posix_openpt
then opens `/dev/pts/N`, which ENOENT'd because the guest mounted devtmpfs
but never mounted devpts at `/dev/pts`. Mount it in `enter_rootfs`
(best-effort, so a pty failure degrades to a warning rather than turning
boot into READY-only). Matches the "devpts+/tmp" guest-compose fix.

Also label the previously-bare error sites in the session launch
(container build / pty open / build command) so a future failure names
the failing step instead of surfacing as an opaque
"session spawn: No such file or directory".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Park point for the HG1 in-sandbox-shell investigation.

- devpts: mount with ptmxmode=0666 and repoint /dev/ptmx -> pts/ptmx so
  posix_openpt and /dev/pts/N share one instance (verified correct in the
  guest). This removed the pty-creation failure.
- log the session process exit code in the host mainloop.

Finding (parked): the in-sandbox shell on minvmd fails with the sandbox
process exiting 125 — hakoniwa cannot construct the container inside the
libkrun guest (works natively on DM2). Not the pty (devpts fixed), not
bash, not the package closure (125 != 127). Next: capture hakoniwa's
exact setup error (debug logging / sandbox stderr); suspect unprivileged
userns or pivot_root for pid-1 in the guest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uest

Two bugs blocked the interactive sandbox shell (HG1) on minvmd while it
worked natively on DM2:

1. exit 125 — guest pid-1 entered the rootfs with a bare chroot(/newroot),
   leaving its root dir != the mount-namespace root. The kernel then
   refused hakoniwa's unshare(CLONE_NEWUSER) with EPERM (a "chroot
   environment" per user_namespaces(7)), so every sandbox build failed.
   Enter the rootfs the switch_root(8) way: MS_MOVE /newroot onto / then
   chroot("."), so the new root is the namespace root and the unprivileged
   userns unshare succeeds. (pivot_root can't move the initramfs source.)

2. vt100 panic — the attach client requests a 0x0 PTY, so WinSize::from
   built a 0-cell grid and vt100-ctt panicked on bash's first write,
   killing the session worker. Clamp 0 rows/cols to 24x80 (corrected by
   the first window_change).

Also log hakoniwa's exit reason on a non-zero sandbox exit (the reason
field was discarded, which made the 125 opaque).

Verified via the full interactive session path (no attach -c): a host-net
session runs an in-sandbox command and curls example.com -> 200 with
eth0+route; a no-net session runs in-sandbox but has no egress (curl 000)
and only lo — namespace isolation holds. No exit 125 / EPERM / panic in
the guest log.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Formatting-only; the ingress-over-vsock commit (ControlChannel) shipped
unformatted closures. No behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… host loopback

Slice 2 of the networking host-exposure unit (unblocks TC3/TC7 on DM1).

Start minimald's two host-side proxies for the daemon's lifetime in
Server::run (both DM1 and DM2), consolidating the startup that previously
only bound (never served) the egress proxy on DM2:
- B5 egress/DNS proxy on :7654 (always);
- B8 mTLS reverse proxy on :7655 (networking-proxy feature), using the
  CertAuthority's server config.

Both route by Host header through the sessions manager's PTask hostname
registry, now shared as Arc<RwLock<HostnameRegistry>> (a HostRoute impl on
the RwLock) and reachable via a new ManagerHandle::hostnames() accessor.

Bind address: in a microVM (DM1) the proxies bind the daemon switch IP
(100.64.255.253) so the host gvproxy forward can reach them, then minimald
publishes 127.0.0.1:{7654,7655} -> daemon-ip:{7654,7655} on the macOS host
via the gvproxy forwarder over the vsock control channel (reusing
policy::post_json / ControlChannel::Vsock). On native Linux (DM2) they bind
host loopback directly and need no expose. All startup/expose is
best-effort: a bind or expose failure warns and the daemon keeps serving.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Executable test plan (test-plan.sh) + companion doc (test-plan.md) for the
networking epic (#478). Every per-session networking assertion runs INSIDE
the sandbox via the full interactive session path (expect-driven
`minimal2 attach`, never `attach -c`, which bypasses the sandbox). One
reusable in-session driver; host->PTask TCs keep an in-session socat
backend alive across the host check; switch-IP via /proc/net/fib_trie (no
`ip` in-session); idempotent cleanup. Cert dir is OS-aware (macOS vs XDG).

Last run on this branch: TC1/TC1b/TC5/TC6/TC9 PASS, TC7 no-cert->401 PASS;
TC2/3/4/7(200)/8 blocked on own-ip PTask attach, which shells out to
ip/nsenter (absent from the guest rootfs) in move_tap_into_netns.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A panic in one session task could poison the hostname registry RwLock,
after which every proxy route resolution silently returned 502. Recover
the guard via into_inner() so a poisoned lock still resolves routes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the microVM the host-side proxies bound the guest switch IP, coupling
listener startup to eth0 readiness. Bind 0.0.0.0 instead so the listeners
always come up, and only publish a port on the host loopback once its
listener has actually bound.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scaffold_default_mfile does a blocking git clone inside the async
session-actor task; wrap it in block_in_place so it moves other futures
off the worker thread instead of stalling them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A clean session exit (code 0) is routine; log it at debug and reserve the
warning for non-zero exits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An own-IP VM has no network without the gvproxy switch. Rather than warn
and boot a session that silently has no egress, hard-error when the
binary is absent and own-ip was requested; point at MINVMD_GVPROXY_BIN.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
apply_ingress/remove_ingress now take a &ControlChannel; wrap the switch
control socket in ControlChannel::Unix at the call sites.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR wires own-IP networking across minimald and minvmd, adds guest egress setup and host gvproxy shuttle support, routes ingress control over Unix or vsock, and updates build, test, and session bootstrap tooling.

Changes

Own-IP networking stack

Layer / File(s) Summary
Workspace scaffold and activation
crates/mctx/src/lib.rs, crates/mctx/src/scaffold.rs, crates/minimal2/src/main.rs, crates/minimald/src/session.rs, crates/mctx/src/error.rs
mctx scaffolding adds default minimal.toml support for empty workspaces, minimal2 activation parses --network/--ingress, and session context handling now scaffolds missing workspace files and surfaces create-session errors.
Sandbox networking API
crates/sandbox2/src/network.rs, crates/sandbox2/src/config.rs, crates/sandbox2/src/error.rs, crates/sandbox2/src/lib.rs
sandbox2 adds a pluggable per-sandbox network contract, custom error wrapping, and execution-time attach and teardown handling.
Shared switch primitives
crates/switch/src/lib.rs, Cargo.toml, crates/switch/Cargo.toml, crates/minimald/Cargo.toml, crates/minvmd/Cargo.toml
The shared switch crate adds subnet arithmetic, MAC derivation, gvproxy YAML rendering, and workspace wiring for the new package.
Switch transport and ingress control
crates/minimald/src/net/*, crates/minimald/tests/netns.rs
minimald::net adds shared switch re-exports, transport selection, transport-agnostic ingress control, and vsock relay attachment for gvproxy connectivity.
Session and host runtime
crates/minimald/src/sessions.rs, crates/minimald/src/net/proxy.rs, crates/minimald/src/session.rs, crates/minimald/src/session_host.rs
minimald wires own-IP sandbox networking into session launch, teardown, terminal handling, hostname routing, and session-manager state.
MicroVM startup and guest egress
crates/minimald/src/main.rs, crates/minimald/src/server.rs, crates/minimald/src/guest.rs, crates/minimald/src/test_harness.rs
minimald sets microVM mode, adjusts host proxy binding, prepares the guest rootfs for PTYs, and brings up guest root egress for the vsock path.
Host gvproxy bootstrap and OwnIp mode
crates/minvmd/src/cmd/*, crates/minvmd/src/image.rs, crates/minvmd/src/net/*, crates/minvmd/src/vm.rs
minvmd resolves and supervises host gvproxy, selects OwnIp VM mode from environment, and registers the gvproxy shuttle vsock bridge in VM setup.
Repo build tasks and docs
justfile, scripts/build-initramfs.sh, docs/specs/03-spec-networking/*
justfile adds bring-up tasks and artifact paths, build-initramfs.sh accepts optional Cargo features, and the networking test plan and script describe and exercise the TC1–TC9 flows.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~90 minutes

Possibly related PRs

  • gominimal/minimal#373: Both PRs modify the guest rootfs/initramfs boot path in crates/minimald/src/guest.rs.
  • gominimal/minimal#375: Both PRs touch attachable-session scaffolding in crates/minimald/src/session.rs and session_host.rs.
  • gominimal/minimal#525: Both PRs build on the earlier minimald::net gvproxy switch and relay implementation.
  • gominimal/minimal#546: Both PRs touch host-side egress proxy routing in crates/minimald/src/net/proxy.rs.
  • gominimal/minimal#547: Both PRs refactor the OwnIp sandbox launch path and its gvproxy wiring.

Suggested labels

needs-human

Suggested reviewers

  • evanspearman

Poem

A rabbit thumped through code at dawn,
With own-IP hops and tunnels drawn.
🐇 The gvproxy stars aligned just right,
And little packets sang all night.
A carrot-bright network grew warm and strong.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: end-to-end networking integration across minimald, minvmd, and minimal2.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (1)
crates/mctx/src/scaffold.rs (1)

38-42: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Unbounded blocking network checkout.

Context::sub_setup + vcs.checkout_of perform a blocking git clone/fetch with no timeout. Since session.rs invokes this synchronously (via block_in_place) on the session actor, an unreachable or slow remote will stall the call—and the worker thread—indefinitely. Given this is explicitly temporary scaffolding, a deadline/bound is optional, but worth tracking before it leaves the daemon path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/mctx/src/scaffold.rs` around lines 38 - 42, The checkout path in
scaffold setup currently performs a blocking remote fetch/clone without any
deadline, which can stall the session actor indefinitely. Update
`Context::sub_setup` and the `vcs.checkout_of` call site in `scaffold.rs` to use
a bounded checkout strategy or wrap the network operation in a timeout, and make
sure the caller can handle timeout failures cleanly. Keep the fix localized
around the `Context::sub_setup` / `checkout_of` flow so the temporary
scaffolding does not block the daemon path forever.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/minimald/src/net/switch.rs`:
- Around line 314-319: Bound the host-side connect/write sequence in
attach_to_switch_vsock so it cannot hang indefinitely. Wrap both the
tokio_vsock::VsockStream::connect call and the AsyncWriteExt::write_all of
CONNECT_REQUEST with an appropriate timeout, and propagate the timeout error so
the OwnIp launch/root-egress setup can fail fast and roll back cleanly if the
shuttle or gvproxy stalls.

In `@crates/minimald/src/sessions.rs`:
- Around line 230-233: RenameSession currently updates only the session
store/host record and leaves HostNet routing stale, so the hostname registry
must be updated as well. In the session rename flow in sessions.rs, make sure
the existing HostNet entry is removed for the old name and re-registered under
the new name using the same hostname registry paths used by register_host_net
and DestroySession deregistration. Apply the same fix anywhere the session name
changes so the hostname routes always stay in sync with the current session
name.

In `@docs/specs/03-spec-networking/test-plan.md`:
- Around line 37-63: Update the per-test notes in the test plan so they match
the current status summary without contradictions: TC1 and TC1b should remain
marked as passing, and the proxy-related notes for TC3/TC7 should reflect that
the host-side proxy plumbing on :7654 and :7655 is already verified. Use the
existing test identifiers (TC1, TC1b, TC3, TC7) and the summary block as the
source of truth, then rewrite the later per-test sections to remove any stale
“blocked” or “not wired” language that conflicts with the current branch state.
- Around line 172-173: The companion script reference is pointing to the wrong
name, so update the documentation to match the actual script added in this PR.
In the test-plan spec entry that mentions the executable companion, replace the
`net-test-plan.sh` reference with the real script name `test-plan.sh` so readers
can locate and run the correct file.

In `@docs/specs/03-spec-networking/test-plan.sh`:
- Around line 31-33: Add trap-based cleanup to the test-plan.sh harness so
interrupted or failing runs still remove created sessions and stop the TC8
ssh-forward process. Update the existing banner/destroy flow and add a shell
trap in the main control path to invoke cleanup on exit signals, and make sure
the TC8 forwarder PID is captured in a variable that the trap can terminate. Use
the existing destroy helper and the TC8 expect/forwarder launch points as the
anchors for the fix.

In `@justfile`:
- Line 19: The binary path used by the signing task is pointing to the debug
build instead of the release build, so `codesign-minvmd` may sign the wrong
artifact or fail if the debug binary is missing. Update the `minvmd-bin`
variable in the justfile to reference the release binary built by
`codesign-minvmd`, and make sure any related signing steps use that same
`minvmd-bin` symbol so the release artifact is the one being signed.

---

Nitpick comments:
In `@crates/mctx/src/scaffold.rs`:
- Around line 38-42: The checkout path in scaffold setup currently performs a
blocking remote fetch/clone without any deadline, which can stall the session
actor indefinitely. Update `Context::sub_setup` and the `vcs.checkout_of` call
site in `scaffold.rs` to use a bounded checkout strategy or wrap the network
operation in a timeout, and make sure the caller can handle timeout failures
cleanly. Keep the fix localized around the `Context::sub_setup` / `checkout_of`
flow so the temporary scaffolding does not block the daemon path forever.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 45be770d-86f8-478e-b453-cf0a68c0456e

📥 Commits

Reviewing files that changed from the base of the PR and between bf05ff4 and 9da27a0.

📒 Files selected for processing (27)
  • crates/mctx/src/lib.rs
  • crates/mctx/src/scaffold.rs
  • crates/minimal2/src/main.rs
  • crates/minimald/Cargo.toml
  • crates/minimald/src/guest.rs
  • crates/minimald/src/main.rs
  • crates/minimald/src/net/mod.rs
  • crates/minimald/src/net/policy.rs
  • crates/minimald/src/net/proxy.rs
  • crates/minimald/src/net/switch.rs
  • crates/minimald/src/server.rs
  • crates/minimald/src/session.rs
  • crates/minimald/src/session_host.rs
  • crates/minimald/src/sessions.rs
  • crates/minimald/src/test_harness.rs
  • crates/minimald/tests/netns.rs
  • crates/minvmd/src/cmd/mod.rs
  • crates/minvmd/src/cmd/run.rs
  • crates/minvmd/src/cmd/vmm_child.rs
  • crates/minvmd/src/image.rs
  • crates/minvmd/src/net.rs
  • crates/minvmd/src/net/shuttle.rs
  • crates/minvmd/src/vm.rs
  • docs/specs/03-spec-networking/test-plan.md
  • docs/specs/03-spec-networking/test-plan.sh
  • justfile
  • scripts/build-initramfs.sh

Comment thread crates/minimald/src/net/switch.rs Outdated
Comment thread crates/minimald/src/sessions.rs
Comment thread docs/specs/03-spec-networking/test-plan.md
Comment thread docs/specs/03-spec-networking/test-plan.md Outdated
Comment thread docs/specs/03-spec-networking/test-plan.sh
Comment thread justfile
That work is folded into this integration; remove the now-stale issue
references from code comments. Comment-only, no behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor
norrietaylor force-pushed the feat/networking-host-exposure branch from 23ff364 to 2b55134 Compare June 26, 2026 08:03
norrietaylor and others added 2 commits June 26, 2026 01:06
Pre-existing formatting drift flagged by CI `cargo fmt --check`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the hand-written Default impl with #[derive(Default)] +
#[default] on LocalSpawn; fixes clippy::derivable_impls under -D warnings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/switch/src/lib.rs`:
- Around line 57-61: `SwitchSubnet::new` currently stores the provided `base`
address without masking off host bits, which makes equivalent subnets compare
differently via `Eq` and `Hash`-like identity semantics. Update the constructor
and any related initialization path in `SwitchSubnet` so `base` is canonicalized
to the network address before storing it, using the same normalization logic
already relied on by `network()` and `Display`, and keep the `base` field
normalized for all downstream methods.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4d867f45-bf40-408f-a3e0-63b43051780f

📥 Commits

Reviewing files that changed from the base of the PR and between 03b0be6 and 11c909d.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • Cargo.toml
  • crates/minimald/Cargo.toml
  • crates/minimald/src/net/mod.rs
  • crates/minvmd/Cargo.toml
  • crates/minvmd/src/net.rs
  • crates/minvmd/src/net/shuttle.rs
  • crates/switch/Cargo.toml
  • crates/switch/src/lib.rs
✅ Files skipped from review due to trivial changes (2)
  • Cargo.toml
  • crates/switch/Cargo.toml
🚧 Files skipped from review as they are similar to previous changes (3)
  • crates/minvmd/src/net/shuttle.rs
  • crates/minvmd/src/net.rs
  • crates/minimald/src/net/mod.rs

Comment thread crates/switch/src/lib.rs
…rable

`just up` / boot e2e could not bring up the VM for two distinct,
environment-dependent reasons; both are now configurable with safe,
arch-aware defaults.

1. Timeouts too short. A cold multi-GiB VM on the generic guest kernel
   takes ~22-28s to reach userspace (longer under load), but the
   READY-marker wait (boot/run) was a fixed 5s and the autospawn UDS wait
   8s -- both firing before the kernel reached /init.
   - minvmd: cmd::ready_timeout() (env MINVMD_READY_TIMEOUT_SECS, default
     60s), shared by boot and run; the timeout error names the override.
   - minimal2: autospawn UDS wait via MINIMAL_SPAWN_TIMEOUT_SECS (default
     75s). All three helpers reject 0 -- a zero timeout would give the
     boot no readiness window / make recv_timeout fail instantly.

2. x86_64 boot panic. A 4096 MiB guest boots on aarch64 but on x86_64
   exactly 4 GiB straddles the 32-bit MMIO/PCI hole (~3-4 GiB),
   mis-placing the initramfs so the kernel finds no /init and panics in
   prepare_namespace ("VFS: cannot open root device"). Guest RAM is now
   cmd::vm_ram_mib() (env MINVMD_VM_RAM_MIB; status reports the resolved
   value), and DEFAULT_VM_RAM_MIB is arch-conditional -- 2048 on x86_64
   (CI-proven, hole-safe), 4096 on aarch64 -- so every boot path (CI,
   `just up`, any host) is safe without per-lane pinning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
norrietaylor added a commit that referenced this pull request Jun 26, 2026
run_with_cancel now kills+reaps the spawned child if Network::attach fails,
and tears the NetGuard down before propagating any reader/writer/wait error,
so an own-IP switch attachment is never leaked on an error path. No-op for
HostNet/NoNet. Refs #581 review (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
norrietaylor added a commit that referenced this pull request Jun 26, 2026
Zero the host bits so two subnets with the same network compare equal
(Display and address math already normalize through network()). Refs #581
review (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
norrietaylor added a commit that referenced this pull request Jun 26, 2026
- Bound the host-shuttle vsock connect + `/connect` upgrade with a 5s
  timeout so an unresponsive/absent host gvproxy fails the OwnIp attach fast
  instead of stalling bring-up.
- Guard the scaffold `block_in_place` by runtime flavor: it panics on a
  current-thread runtime, so fall back to a plain blocking call there.
- RenameSession now re-points the HostNet hostname route to the new name
  (was stranded under the launch name).

Refs #581/#585 review (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
norrietaylor added a commit that referenced this pull request Jun 26, 2026
Add mctx::make_env_with_network and have the task-exec path pass the
session's NetworkMode instead of a hardcoded HostNet, so a task gets the
same netns isolation as its session through the sandbox2 Network seam (a
no-net session's task no longer leaks host egress). OwnIp falls back to
HostNet pending the per-PTask tap/switch attach for the task producer
(blocked by the guest rootfs ip/nsenter gap). Refs #581 review
(twitchyliquid64) [C3].

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
norrietaylor added a commit that referenced this pull request Jun 26, 2026
- justfile codesign-minvmd signs the release binary it builds (was debug).
- OWN_IP env doc lists all accepted truthy values (1/true/yes/on).
- test-plan: fix companion script name (test-plan.sh), refresh stale
  hakoniwa-125 statuses to the real own-ip ip/nsenter blocker (TC1/TC1b
  now PASS), and add trap-based cleanup so an interrupted run tears down
  sessions + the TC8 ssh-forward.

Refs #581 review (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor

Copy link
Copy Markdown
Member Author

CodeRabbit review — all findings addressed in the latest push. Triage + dispositions:

Fixed (correctness):

  • switch.rs host-shuttle connect/write had no timeout → 5s bound on the vsock connect + /connect upgrade (daff5d91).
  • sessions.rs HostNet hostname route stranded on rename → RenameSession re-points it (daff5d91).
  • sandbox2 run_with_cancel leaked the sandbox/switch on Network::attach/writer/wait failure → kill+reap the child on attach failure and tear the NetGuard down before propagating any error (a202aa38). Covers the gvproxy_network.rs:63, lib.rs:786, and lib.rs:947 findings.
  • switch::SwitchSubnet::new stored host bits verbatim → canonicalize base to the network address (f12f48bc).

Fixed (docs/tooling):

  • justfile codesign-minvmd signed the debug binary while building release → signs target/release/minvmd (79accea7).
  • OWN_IP env doc now lists all accepted truthy values (1/true/yes/on) (79accea7).
  • test-plan: companion script name corrected to test-plan.sh; stale per-TC statuses refreshed (TC1/TC1b now PASS; own-ip TCs cite the real ip/nsenter rootfs blocker); trap-based cleanup added so an interrupted run tears down sessions + the TC8 ssh-forward (79accea7).

Note: the mctx::scaffold unbounded-blocking-clone nitpick is the explicitly-temporary scaffold (removed once sessions receive their project files); its block_in_place is now flavor-guarded so it can't panic on a current-thread runtime (daff5d91).

Verified: cross-clippy (minimald+mctx, aarch64-musl), native clippy (minvmd+switch), switch + minvmd net tests, fmt — all clean.

norrietaylor and others added 5 commits June 26, 2026 15:11
run_with_cancel now kills and reaps the spawned child when Network::attach
fails, and tears the NetGuard down before propagating any reader/writer/wait
error, so an own-IP switch attachment is never leaked on an error path.
No-op for HostNet/NoNet. From the CodeRabbit review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Zero the host bits so two subnets with the same network compare equal
(Display and address math already normalize through network()). From the
CodeRabbit review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Bound the host-shuttle vsock connect and /connect upgrade with a 5s
  timeout so an unresponsive or absent host gvproxy fails the OwnIp attach
  fast instead of stalling bring-up.
- Guard the scaffold block_in_place by runtime flavor: it panics on a
  current-thread runtime, so fall back to a plain blocking call there.
- RenameSession re-points the HostNet hostname route to the new name (it
  was left stranded under the launch name).
- Un-gate SessionHandle::record so the task-exec path can read the
  session network mode in production.

From the CodeRabbit and twitchyliquid64 reviews.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HostGvproxy::spawn now polls gvproxy's -listen socket (5s) before sending
Ok(pid), so a switch that died during startup is surfaced as an error
rather than reported network-ready. Also factor switch-socket derivation
into a pure helper so its test no longer mutates process-global
XDG_RUNTIME_DIR. From the CodeRabbit review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add mctx::make_env_with_network and have the task-exec path pass the
session's NetworkMode instead of a hardcoded HostNet, so a task gets the
same netns isolation as its session through the sandbox2 Network seam (a
no-net session's task no longer leaks host egress). OwnIp falls back to
HostNet pending the per-PTask tap/switch attach for the task producer,
which is blocked by the guest rootfs ip/nsenter gap. Addresses the
twitchyliquid64 review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor
norrietaylor force-pushed the feat/networking-host-exposure branch from 79accea to 5b75f63 Compare June 26, 2026 22:14
- justfile codesign-minvmd signs the release binary it builds (was debug).
- OWN_IP env doc lists all accepted truthy values (1, true, yes, on).
- test-plan: fix companion script name (test-plan.sh), refresh stale
  hakoniwa-125 statuses to the real own-ip ip/nsenter blocker (TC1/TC1b
  now PASS), and add trap-based cleanup so an interrupted run tears down
  sessions and the TC8 ssh-forward.

From the CodeRabbit review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor
norrietaylor force-pushed the feat/networking-host-exposure branch from 5b75f63 to aeee85f Compare June 26, 2026 22:18
norrietaylor and others added 3 commits June 26, 2026 15:32
Point locked_commit at 23193970569b93e5e7a980aa598c3342c386d33f, the new
upstream rootfs build. If it carries ip/nsenter, this unblocks the own-ip
data-path test cases that the guest rootfs gap was blocking.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The M6 dead-code removal dropped GvproxySupervisor::attach_ptask /
detach_ptask / PtaskAttachment and SwitchSubnet::host, and gateway() now
returns Ipv4Addr instead of Option. vsock_relay_e2e.rs still used the old
API. The test is #[ignore]-gated and only compiled by the
minvmd-linux-kvm-e2e lane (cargo test -p minvmd vsock --include-ignored),
so the normal test/clippy lanes never built it and the break surfaced only
on that lane.

Rewrite against the current flow: open_tap + attach_to_switch returning a
SwitchRelay (Drop tears it down), the seeded first-PTask lease via
SwitchSubnet::first_ptask(), and gateway() as a plain Ipv4Addr.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eout

A flaky own-ip attach can wedge an expect block with no internal bound
(see #588), stalling the whole runner. Wrap each per-session expect
invocation in gtimeout/timeout (STEP_TIMEOUT, default 360s) so one hang
caps at the step instead of the plan. Degrades to no guard when neither
binary is present.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor
norrietaylor merged commit 3b28f9f into main Jun 29, 2026
43 checks passed
@norrietaylor
norrietaylor deleted the feat/networking-host-exposure branch June 29, 2026 16:04
norrietaylor added a commit that referenced this pull request Jul 13, 2026
minvmd's net::relay (open_tap/attach_to_switch/SwitchRelay) has had no
production caller since #581 moved the per-PTask tap+relay into the guest
minimald over the vsock shuttle; it was a frozen byte-for-byte fork of
minimald's net::switch that never received the ingress gate the live copy
grew. Its only caller was vsock_relay_root_e2e, which booted no VM and
re-proved the relay against a hand-made netns — coverage already provided,
with real PTask-to-PTask traffic, by minimald's netns_root_e2e UC6.

Delete net/relay.rs, its module wiring, and the test; correct the net.rs
module doc (the tap + relay run in the guest, not here). The KVM lane's
root step existed only for that test, so it is removed along with the
gvproxy materialize step and the vendor/gvproxy pin filter it needed; the
lane keeps a single auto-discovered harness step (a future minvmd
*_root_e2e would need its own sudo-wrapped step).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
norrietaylor added a commit that referenced this pull request Jul 13, 2026
minvmd's net::relay (open_tap/attach_to_switch/SwitchRelay) has had no
production caller since #581 moved the per-PTask tap+relay into the guest
minimald over the vsock shuttle; it was a frozen byte-for-byte fork of
minimald's net::switch that never received the ingress gate the live copy
grew. Its only caller was vsock_relay_root_e2e, which booted no VM and
re-proved the relay against a hand-made netns — coverage already provided,
with real PTask-to-PTask traffic, by minimald's netns_root_e2e UC6.

Delete net/relay.rs, its module wiring, and the test; correct the net.rs
module doc (the tap + relay run in the guest, not here). The KVM lane's
root step existed only for that test, so it is removed along with the
gvproxy materialize step and the vendor/gvproxy pin filter it needed; the
lane keeps a single auto-discovered harness step (a future minvmd
*_root_e2e would need its own sudo-wrapped step).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU
norrietaylor added a commit that referenced this pull request Jul 14, 2026
…s every lane (#732)

* test(minvmd,ci): auto-discover VM e2e harnesses by name suffix

VM e2e harnesses are now selected by binary-name convention instead of a
hardcoded per-test list, so adding a test never requires a CI edit:

- Harnesses in crates/minvmd/tests/ end in `_e2e` (non-root) or
  `_root_e2e` (root, run under sudo for CAP_NET_ADMIN).
- The KVM lane's three enumerated steps collapse to two convention
  filtersets: `binary(/_e2e$/) and not binary(/_root_e2e$/)` and
  `binary(/_root_e2e$/)`. A new `*_e2e.rs` is picked up with no YAML.
- Rename krun_smoke -> krun_smoke_e2e and vsock_relay_e2e ->
  vsock_relay_root_e2e; delete the retired bridge_e2e stub.
- A unit test in minvmd (runs in core-tests, no libkrun) guards the
  suffix so a typo fails CI instead of silently dropping a harness.

This newly runs volume_quiesce_e2e (added in #705 but never wired into a
lane) in the KVM lane. The macOS lane tracks the rename only; its
filterset conversion is coupled to the nextest-archive-on-mac work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU

* test(minimald,linux-native): fold the netns track into root e2e auto-discovery

The netns-integration job hardcoded `cargo test -p minimald --test netns`.
Rename netns.rs -> netns_root_e2e.rs and select it by the same convention
the KVM lane uses: the renamed `minimald-root-e2e` job runs
`cargo nextest run -p minimald -E 'binary(/_root_e2e$/)'`, so a new
crates/minimald/tests/*_root_e2e.rs runs with no workflow edit.

A minimald unit test guards the suffix; mesh_uc7 is allowlisted because
its networking-wg feature is off by default and no lane runs it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU

* ci(macos): auto-discover VM e2e harnesses from a nextest archive

The mac e2e job hardcoded two test names and ran them as direct binaries.
Build a nextest archive before the codesign (`nextest run --archive-file`
is offline, so the signature survives) and select harnesses by the `_e2e`
convention, so a new crates/minvmd/tests/*_e2e.rs runs on mac with no
workflow edit. This newly runs boot_e2e and volume_quiesce_e2e on mac.

The full-boot harnesses spawn the codesigned minvmd as a subprocess
(MINVMD_BIN). krun_smoke_e2e stays a separate kernel-less FFI-smoke step,
excluded from the archive run: it boots in-process through
krun_smoke_child, which cannot take the hypervisor entitlement from
within the archive.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU

* docs: update specs for the e2e test renames and bridge_e2e removal

Reflect the auto-discovery migration in the frozen unit specs: krun_smoke
-> krun_smoke_e2e in R1.5, and note that bridge_e2e was removed
(superseded by minimald_session_e2e).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU

* refactor(minvmd,ci): drop the dead host-side tap relay + its KVM wiring

minvmd's net::relay (open_tap/attach_to_switch/SwitchRelay) has had no
production caller since #581 moved the per-PTask tap+relay into the guest
minimald over the vsock shuttle; it was a frozen byte-for-byte fork of
minimald's net::switch that never received the ingress gate the live copy
grew. Its only caller was vsock_relay_root_e2e, which booted no VM and
re-proved the relay against a hand-made netns — coverage already provided,
with real PTask-to-PTask traffic, by minimald's netns_root_e2e UC6.

Delete net/relay.rs, its module wiring, and the test; correct the net.rs
module doc (the tap + relay run in the guest, not here). The KVM lane's
root step existed only for that test, so it is removed along with the
gvproxy materialize step and the vendor/gvproxy pin filter it needed; the
lane keeps a single auto-discovered harness step (a future minvmd
*_root_e2e would need its own sudo-wrapped step).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU

* refactor(minimald,ci): name and describe the root e2e proofs functionally

Drop the internal use-case numbers (UC1/UC4/UC6/UC7) from the netns test
function names, its module doc, the native-lane comments, and the minimald
naming guard. Describe each proof by what it does — a no-network task
cannot reach the internet, a static ingress mapping exposes then removes a
task's port, two own-IP tasks reach each other over the gvproxy switch —
and state why they need root: they create tap devices and configure
network namespaces, which requires CAP_NET_ADMIN, so the unprivileged CI
runner grants it via sudo.

No behavior change; the harness is selected by binary name, not function
name, so the renames are self-contained.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU

* refactor(minvmd,minimald,ci): rename harness e2e tests to _integration

The harness-driven nextest proofs are component/subsystem INTEGRATION
tests, not full-system e2e: each stands up one subsystem against real
resources (boot a microVM, round-trip the vsock bridge, drive the libkrun
FFI, check a real ext4 volume, exercise the tap/netns stack) rather than
driving the `minimal` CLI through the whole system. The only true
end-to-end proof is the CLI script (scripts/session-e2e.sh).

Rename the auto-discovery suffix _e2e / _root_e2e -> _integration /
_root_integration across the five harnesses, the KVM/native/mac lane
filtersets, and both naming guards; the guard docs now state when to
reach for an integration harness vs. a full-system e2e script. Rename the
minvmd-scoped lifecycle script (lifecycle-e2e.sh -> minvmd-lifecycle.sh)
and the native root job (minimald-root-e2e -> minimald-root-integration).

Also scrub the workflow YAMLs of internal plan jargon (UC/N/R references)
in favour of functional descriptions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HbXL6jxHTY2YuLPaMxc1yU

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants