Skip to content

Do not review: CI test - #585

Closed
norrietaylor wants to merge 40 commits into
mainfrom
fix/vm-bringup-timeouts-ram
Closed

Do not review: CI test#585
norrietaylor wants to merge 40 commits into
mainfrom
fix/vm-bringup-timeouts-ram

Conversation

@norrietaylor

@norrietaylor norrietaylor commented Jun 26, 2026

Copy link
Copy Markdown
Member

Stacked on #581 (base: feat/networking-host-exposure) to exercise CI — primarily the x86_64 Linux/KVM lane — on two VM-bring-up fixes.

What

  1. Boot-readiness timeouts were too short for a cold multi-GiB VM (~22–28s to userspace, longer under host load): the READY-marker wait was a fixed 5s and the autospawn UDS wait 8s, both firing before the kernel reached /init. Now env-configurable with generous defaults:
    • MINVMD_READY_TIMEOUT_SECS (default 60s) — shared by boot/run.
    • MINIMAL_SPAWN_TIMEOUT_SECS (default 75s) — autospawn UDS wait (an upper bound; only costs time on a slow/failed boot).
  2. x86_64 boot panic. The 4096 MiB guest boots on aarch64 but on x86_64 exactly 4 GiB straddles the 32-bit MMIO/PCI hole (~3–4 GiB), mis-placing the initramfs so the kernel finds no /init and panics in prepare_namespace (VFS: cannot open root device) before any minimald code. Guest RAM is now cmd::vm_ram_mib() (env MINVMD_VM_RAM_MIB, default 4096 — aarch64 cache headroom unchanged); ci-linux-kvm pins the x86_64 lane to a hole-safe 2048. status now reports the resolved value.

Why a stacked test PR

The RAM fix was validated on aarch64 only — this dev host can't run x86_64 libkrun guests. The x86_64 KVM CI lane (minvmd-linux-kvm-e2eboot_e2e) is the only thing that can confirm the VFS cannot mount root panic is gone. This PR exists to run that lane on the fix.

If the x86_64 lane goes green, fold 6a13cfa5 into #581.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added support for configurable session networking, including network mode selection and ingress port mappings.
    • Enabled optional “own IP” networking and guest egress setup for VM sessions.
    • Added automatic workspace scaffolding when required files are missing.
    • Introduced configurable RAM and startup timeout settings for VM and daemon startup.
  • Bug Fixes

    • Improved attach behavior for interactive sessions and PTY handling.
    • Made network and proxy startup more resilient, with better fallback behavior when optional components are unavailable.

norrietaylor and others added 30 commits June 24, 2026 15:38
`minimal activate` hardcoded NetworkMode::default() (HostNet) with empty
policy, so NoNet/OwnIp and ingress port mappings had no CLI surface and were
reachable only via the netns proofs constructing a Record directly.

- `--network <no-net|host-net|own-ip>` (default host-net; no-flag behavior
  unchanged) via a local CliNetworkMode ValueEnum, keeping the sessions crate
  free of a clap dependency.
- `--ingress EXT:INT[/PROTO]` (repeatable; PROTO defaults tcp) parsed into
  sessions::PortMapping; non-tcp/udp and malformed specs rejected at parse time.
- Surface the daemon's typed CreateSession validation error (e.g. ingress on a
  non-own-ip session) instead of a generic failure line.

Daemon-side validate_policy is unchanged and remains the enforcement point.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0143kv2BRrRqGxmVwwHskQtS
On DM1/3/4 (a libkrun VM) gvproxy must run on the host, owned by
minvmd, not inside the guest where it has no host uplink (issue #572).
Wire up the previously-dead host-side switch:

- HostGvproxy: an RAII supervisor that spawns the host gvproxy switch
  (reusing GvproxyConfig::spawn) on a dedicated current-thread tokio
  runtime, so minvmd's synchronous boot/run supervisor can own an async
  switch lifecycle and tear it down on drop. Started with an empty
  static-lease table — the guest configures each PTask's switch IP
  statically, so minvmd need not own the per-PTask address book.
- run --foreground spawns + supervises the host gvproxy for an own-IP VM
  before booting the VMM child, so the -listen switch socket exists when
  libkrun dials it; the handle is held for the VM's lifetime.
- VmConfig::apply registers the per-PTask shuttle vsock bridge for an
  own-IP VM: add_vsock_port2(VSOCK_GVPROXY_SHUTTLE_PORT, switch_sock,
  listen = false) — the guest shuttle connects to AF_VSOCK CID 2 and
  libkrun splices it to the host gvproxy -listen socket, carrying raw L2
  frames (no second TCP/IP stack).
- net::shuttle: the shuttle vsock port + host switch-socket path
  resolver (placed beside the minimald bridge socket).
- image::resolve_gvproxy_path reads MINVMD_GVPROXY_BIN (fallback to the
  fixed install path); cmd::own_ip_requested reads MINVMD_VM_OWN_IP so
  the parent's gvproxy-spawn decision and the VMM child's VM network mode
  stay in lock-step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In a libkrun VM (DM1/3/4) minimald-in-guest must NOT spawn gvproxy
locally — gvproxy runs on the host (owned by minvmd) and the guest only
relays raw L2 frames to it (issue #572). Previously the DM2 spawn-local
path ran verbatim in the guest, so an OwnIp PTask got a 100.64 switch IP
but no egress (gvproxy NAT'd into a dead-end netns).

- net::SwitchTransport enum (LocalSpawn vs HostShuttle { cid, port })
  makes "spawn locally" and "relay to the host switch" mutually
  exclusive in the type system. GvproxySwitch::with_transport selects
  it; in HostShuttle mode attach() skips the local gvproxy spawn/config
  and only tracks the attach count.
- switch::attach_to_switch_vsock relays a PTask tap to the host gvproxy
  over AF_VSOCK (CID 2 : shuttle port); it shares a transport-agnostic
  spawn_relay with the DM2 UDS attach_to_switch — same HyperKit-framed
  raw-L2 relay, so exactly one gVisor stack stays in the path.
- session_host::attach_own_ip branches on the switch transport: DM2 uses
  the local control socket, DM1/3/4 the vsock shuttle. Static ingress on
  a VM own-IP PTask is skipped with a warning (gvproxy's port-forward API
  is host-side and unreachable over the frame-only shuttle); egress, the
  #572 focus, needs no such call.
- DM signal: the `vsock` listen-arg is exactly the libkrun-VM boundary
  (DM1/3/4) vs the UDS daemon (DM2); threaded through Config::in_microvm
  into the transport selection. DM2 behaviour is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`just up` materializes the guest kernel + rootfs, builds the
networking-enabled guest initramfs, builds minvmd (codesigned on
macOS), builds the minimal CLI, then runs `minimal ls` to autospawn
the supervised minvmd and reach minimald over the UDS bridge.

Recipes branch on $(uname -s):
- macOS: `minimal materialize`, Homebrew libkrun, codesign.
- Linux: fetch-artifact.sh / fetch-libkrun.sh with LIBKRUN_PREFIX +
  LD_LIBRARY_PATH exported so build.rs links the real implementation;
  no codesign; requires durable kvm group membership.

build-initramfs.sh gains a FEATURES env hook so the guest minimald is
compiled with networking-proxy,networking-wg (the HTTPS mTLS proxy and
WireGuard mesh peer); it previously passed no --features.

artifacts is idempotent (skips a present artifact) and clean removes
only the bring-up files, never the shared .scratch directory.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract `generate_mfile` from `minimal init` (cmd_init) into
mctx::scaffold and add `scaffold_default_mfile`, which resolves the
latest gominimal/pkgs @main commit and writes a default shell-stack
minimal.toml into a directory that has none.

minimald's session `context()` now scaffolds such a file when the
session workspace lacks one, so `attach` no longer fails with
"minimal.toml not found" on an empty workspace. Scaffolding is lazy
(on first attach / context build), not on activate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the issue #572 per-PTask vsock shuttle to the guest's root
network namespace, so minimald itself has egress (default route + DNS)
and can fetch the upstream pkgs repo — e.g. the clone that scaffolding
a session's minimal.toml performs.

minvmd:
- spawn the host gvproxy switch for every VM (best-effort: skip with a
  warning when the gvproxy binary is absent, so the boot/session e2e
  lanes still boot without egress), not just own-IP VMs.
- register the gvproxy shuttle vsock port unconditionally; the root tap
  and any own-IP PTask taps are all L2 clients on the one switch.

minimald:
- bring up a primary `eth0` tap in the root netns at boot, attached to
  the host gvproxy over the vsock shuttle. Configure address/up/default
  route via AF_INET ioctls (the generic rootfs ships no iproute2), and
  install /etc/resolv.conf by bind-mounting from the writable /run tmpfs
  (the rootfs is read-only). Hold the relay for the server's lifetime.
- reserve a daemon switch address (`broadcast - 2`) from the top of the
  subnet so the PTask range still starts at `network + 2`.

justfile: `up` fetches gvproxy and exports MINVMD_GVPROXY_BIN so the
host switch comes up; gvproxy fetch is idempotent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ary copy

The session-workspace scaffolding is a stop-gap for the workspace-upload
gap, so keep it isolated rather than refactoring shared code. Revert the
extraction of `generate_mfile` from `minimal::cmd_init` (cmd_init keeps
its own copy) and instead duplicate the generator inside mctx::scaffold,
marked TEMPORARY throughout. Deleting one module then removes the
workaround cleanly, with no cmd_init unpicking.

mctx now exports only `scaffold_default_mfile`; `generate_mfile` and its
constants are private to the module.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… on DM1

Unify the forwarder API behind a `ControlChannel { Unix | Vsock }`:
- DM2 keeps the local unix control socket.
- DM1/3/4 reach the host gvproxy's `-listen` socket over the same vsock
  shuttle port the L2 `/connect` relay uses (it serves
  `/services/forwarder/*` too), so `post_json` rides vsock there.

Replaces the prior "skip ingress with a warning on the VM path" branch in
`attach_own_ip`: own-IP PTasks in a libkrun VM now get their static
ingress forwards published on the host loopback (gvproxy binds
127.0.0.1:ext -> ptask-switch-ip:int), torn down on session end.

`post_json` is refactored into a transport-generic `exchange` over any
AsyncRead+AsyncWrite stream, shared by the unix and vsock channels.

Verification of the live data path is pending a scriptable interactive
attach: `attach -c` runs the exec path (daemon host netns) and never
mints the own-IP session host, so it cannot exercise this path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…HG1)

Interactive `minimal2 attach` panicked the daemon and never delivered a
working shell. Root causes, fixed here:

- rustls had no process-default CryptoProvider, so the first rustls user
  on the attach path (the remote-cache HTTPS client during a session
  build) panicked ("no process-level CryptoProvider"), killing the
  session actor; the caller then panicked at session.rs:398 on the lost
  reply. Install ring as the default provider at minimald startup, and
  make `rustls` a normal dep (it is always linked via reqwest anyway).
- The session actor dropping its reply now surfaces as an AttachError
  instead of `expect`-panicking the worker.
- `minimal2 attach` (interactive) requested no PTY, so the daemon's
  shell_request rejected it whenever stdin was not a tty (automation).
  Force one with `ssh -tt` for the interactive path.
- The read-only guest rootfs left `/tmp` unwritable, so remote-cache
  staging (tempfile) failed with EROFS; mount a tmpfs at `/tmp`.

With these, attach reaches the in-sandbox session build. Remaining
blocker is cache capacity (the session package set overflows the
tmpfs cache) — the seeded-cache-disk work, tracked separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two in-sandbox-shell bring-up fixes uncovered while exercising the
interactive attach path:

- Bump the microVM from 1024 to 4096 MiB. The in-VM session build stages
  packages into a RAM-backed tmpfs cache (/run/minimal/cache); 1024 MiB
  overflowed (StorageFull) unpacking large packages. Stop-gap until the
  persistent seeded cache disk lands.
- Exec the session shell at /usr/bin/bash, not /bin/bash: the bash
  package installs to --prefix=/usr and the generic rootfs has no
  /bin/bash, so the old path failed with ENOENT.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The interactive session shell's server-side `Pty::open` does posix_openpt
then opens `/dev/pts/N`, which ENOENT'd because the guest mounted devtmpfs
but never mounted devpts at `/dev/pts`. Mount it in `enter_rootfs`
(best-effort, so a pty failure degrades to a warning rather than turning
boot into READY-only). Matches the "devpts+/tmp" guest-compose fix.

Also label the previously-bare error sites in the session launch
(container build / pty open / build command) so a future failure names
the failing step instead of surfacing as an opaque
"session spawn: No such file or directory".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Park point for the HG1 in-sandbox-shell investigation.

- devpts: mount with ptmxmode=0666 and repoint /dev/ptmx -> pts/ptmx so
  posix_openpt and /dev/pts/N share one instance (verified correct in the
  guest). This removed the pty-creation failure.
- log the session process exit code in the host mainloop.

Finding (parked): the in-sandbox shell on minvmd fails with the sandbox
process exiting 125 — hakoniwa cannot construct the container inside the
libkrun guest (works natively on DM2). Not the pty (devpts fixed), not
bash, not the package closure (125 != 127). Next: capture hakoniwa's
exact setup error (debug logging / sandbox stderr); suspect unprivileged
userns or pivot_root for pid-1 in the guest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uest

Two bugs blocked the interactive sandbox shell (HG1) on minvmd while it
worked natively on DM2:

1. exit 125 — guest pid-1 entered the rootfs with a bare chroot(/newroot),
   leaving its root dir != the mount-namespace root. The kernel then
   refused hakoniwa's unshare(CLONE_NEWUSER) with EPERM (a "chroot
   environment" per user_namespaces(7)), so every sandbox build failed.
   Enter the rootfs the switch_root(8) way: MS_MOVE /newroot onto / then
   chroot("."), so the new root is the namespace root and the unprivileged
   userns unshare succeeds. (pivot_root can't move the initramfs source.)

2. vt100 panic — the attach client requests a 0x0 PTY, so WinSize::from
   built a 0-cell grid and vt100-ctt panicked on bash's first write,
   killing the session worker. Clamp 0 rows/cols to 24x80 (corrected by
   the first window_change).

Also log hakoniwa's exit reason on a non-zero sandbox exit (the reason
field was discarded, which made the 125 opaque).

Verified via the full interactive session path (no attach -c): a host-net
session runs an in-sandbox command and curls example.com -> 200 with
eth0+route; a no-net session runs in-sandbox but has no egress (curl 000)
and only lo — namespace isolation holds. No exit 125 / EPERM / panic in
the guest log.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Formatting-only; the ingress-over-vsock commit (ControlChannel) shipped
unformatted closures. No behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… host loopback

Slice 2 of the networking host-exposure unit (unblocks TC3/TC7 on DM1).

Start minimald's two host-side proxies for the daemon's lifetime in
Server::run (both DM1 and DM2), consolidating the startup that previously
only bound (never served) the egress proxy on DM2:
- B5 egress/DNS proxy on :7654 (always);
- B8 mTLS reverse proxy on :7655 (networking-proxy feature), using the
  CertAuthority's server config.

Both route by Host header through the sessions manager's PTask hostname
registry, now shared as Arc<RwLock<HostnameRegistry>> (a HostRoute impl on
the RwLock) and reachable via a new ManagerHandle::hostnames() accessor.

Bind address: in a microVM (DM1) the proxies bind the daemon switch IP
(100.64.255.253) so the host gvproxy forward can reach them, then minimald
publishes 127.0.0.1:{7654,7655} -> daemon-ip:{7654,7655} on the macOS host
via the gvproxy forwarder over the vsock control channel (reusing
policy::post_json / ControlChannel::Vsock). On native Linux (DM2) they bind
host loopback directly and need no expose. All startup/expose is
best-effort: a bind or expose failure warns and the daemon keeps serving.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Executable test plan (test-plan.sh) + companion doc (test-plan.md) for the
networking epic (#478). Every per-session networking assertion runs INSIDE
the sandbox via the full interactive session path (expect-driven
`minimal2 attach`, never `attach -c`, which bypasses the sandbox). One
reusable in-session driver; host->PTask TCs keep an in-session socat
backend alive across the host check; switch-IP via /proc/net/fib_trie (no
`ip` in-session); idempotent cleanup. Cert dir is OS-aware (macOS vs XDG).

Last run on this branch: TC1/TC1b/TC5/TC6/TC9 PASS, TC7 no-cert->401 PASS;
TC2/3/4/7(200)/8 blocked on own-ip PTask attach, which shells out to
ip/nsenter (absent from the guest rootfs) in move_tap_into_netns.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A panic in one session task could poison the hostname registry RwLock,
after which every proxy route resolution silently returned 502. Recover
the guard via into_inner() so a poisoned lock still resolves routes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the microVM the host-side proxies bound the guest switch IP, coupling
listener startup to eth0 readiness. Bind 0.0.0.0 instead so the listeners
always come up, and only publish a port on the host loopback once its
listener has actually bound.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scaffold_default_mfile does a blocking git clone inside the async
session-actor task; wrap it in block_in_place so it moves other futures
off the worker thread instead of stalling them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A clean session exit (code 0) is routine; log it at debug and reserve the
warning for non-zero exits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An own-IP VM has no network without the gvproxy switch. Rather than warn
and boot a session that silently has no egress, hard-error when the
binary is absent and own-ip was requested; point at MINVMD_GVPROXY_BIN.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
apply_ingress/remove_ingress now take a &ControlChannel; wrap the switch
control socket in ControlChannel::Unix at the call sites.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
That work is folded into this integration; remove the now-stale issue
references from code comments. Comment-only, no behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pre-existing formatting drift flagged by CI `cargo fmt --check`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the hand-written Default impl with #[derive(Default)] +
#[default] on LocalSpawn; fixes clippy::derivable_impls under -D warnings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the sibling SwitchTransport enum and the coding standard for
public enums that may grow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A present-but-unlaunchable gvproxy binary (wrong arch, not executable)
hard-failed every VM boot, including plain HostNet boots. Now a spawn
failure only bails for own-IP VMs and otherwise warns and boots without
guest egress, matching the missing-binary path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
own_ip_requested only matched "1"/"true"/"TRUE", so "True", "yes", "on",
or padded values silently fell through to a HostNet VM. Trim and
lowercase before matching the accepted truthy set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
norrietaylor and others added 5 commits June 26, 2026 01:20
The spawned serve/serve_https tasks dropped their io::Result, so a fatal
accept-loop error left the daemon running with a dead proxy and no log.
Wrap each task to log the error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
try_wait already warns on a non-zero exit with the richer hakoniwa
diagnostics; the mainloop warning double-logged the same event. Keep a
single routine debug line in mainloop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…anic

Making rustls non-optional pulled the `ring` provider into the
default-feature build, which already carries `aws-lc-rs` via
google-cloud-auth. With two providers compiled, rustls can no longer
auto-select one and panics ("no process-level CryptoProvider") the first
time a config is built without an explicit install — breaking the mctx
`env_channel_add_session` test under `cargo nextest run --workspace`.

Restore rustls to optional (only `networking-proxy` pulls it) and gate
the startup provider install under that feature — the one configuration
where both providers are present and an explicit install is required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce a `Network` trait (with `HostNet`/`NoNet` built-ins and a
`NetGuard` for explicit teardown) so network setup is configured via the
Config builder and applied by the Sandbox, rather than living only in the
minimald session host. `Config::with_network` injects a custom impl;
`new_container` prefers it for the netns-isolation decision and
`Sandbox::attach_network` invokes its post-spawn wiring.

Additive and behaviour-preserving: with no custom Network set, the
existing `network_mode` path is unchanged. Consumer migration (minimald
own-IP switch attach → an injected Network; tasks via mctx) follows.

Refs #581 review (twitchyliquid64): networking belongs in sandbox2 so it
works for tasks and any sandbox, not just minimald sessions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two crates each had a type named `GvproxySwitch` with different roles,
which obscured the gvproxy ownership model. Rename for clarity:
- minvmd: `GvproxySwitch` -> `GvproxySupervisor` (owns + supervises the
  host gvproxy process via pidfd on DM1/3/4).
- minimald: `GvproxySwitch` -> `SwitchClient` (an IP allocator + attach
  refcounter; on DM2 it spawns gvproxy locally, on DM1/3/4 it spawns no
  process and relays taps to the host-owned switch over vsock).

No behaviour change. Refs #581 review (twitchyliquid64): clarifies the
gvproxy object lifecycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor
norrietaylor changed the base branch from feat/networking-host-exposure to main June 26, 2026 17:59
@norrietaylor norrietaylor changed the title fix(minvmd,minimal2): make VM bring-up timeouts and guest RAM configurable Do not review: CI test Jun 26, 2026
@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@norrietaylor, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 1 minute and 33 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8bd14a4b-49f0-4b74-bff8-f70183ddf057

📥 Commits

Reviewing files that changed from the base of the PR and between 0952088 and a261d89.

📒 Files selected for processing (22)
  • crates/mctx/src/error.rs
  • crates/minimal2/src/autospawn.rs
  • crates/minimald-rpc/src/lib.rs
  • crates/minimald-rpc/src/switch.rs
  • crates/minimald/src/net/gvproxy_network.rs
  • crates/minimald/src/net/mod.rs
  • crates/minimald/src/server.rs
  • crates/minimald/src/session.rs
  • crates/minimald/src/session_host.rs
  • crates/minimald/src/sessions.rs
  • crates/minimald/tests/netns.rs
  • crates/minvmd/src/cmd/boot.rs
  • crates/minvmd/src/cmd/mod.rs
  • crates/minvmd/src/cmd/run.rs
  • crates/minvmd/src/cmd/status.rs
  • crates/minvmd/src/cmd/vmm_child.rs
  • crates/minvmd/src/net.rs
  • crates/minvmd/src/net/shuttle.rs
  • crates/sandbox2/src/config.rs
  • crates/sandbox2/src/error.rs
  • crates/sandbox2/src/lib.rs
  • crates/sandbox2/src/network.rs
📝 Walkthrough

Walkthrough

The PR adds own-IP networking, gvproxy shuttle plumbing, and ingress forwarding across minvmd and minimald. It also scaffolds missing workspace mfiles, updates session attach handling, changes the minimal2 CLI and timeout behavior, and expands build, CI, and networking test support.

Changes

Networking bring-up and support tooling

Layer / File(s) Summary
Shared runtime knobs and boot flow
crates/minvmd/src/cmd/*, crates/minvmd/src/image.rs, crates/minvmd/src/vmm_child.rs
minvmd now reads env-driven RAM, READY timeout, own-IP, and gvproxy path settings for boot, run, status, and VMM config.
gvproxy shuttle and host supervisor
crates/minvmd/src/net.rs, crates/minvmd/src/net/shuttle.rs, crates/minvmd/src/vm.rs
minvmd now resolves a host gvproxy switch socket, supervises gvproxy on a dedicated thread, and registers the extra vsock bridge for the VM.
Switch transport and ingress control
crates/minimald/src/net/*, crates/minimald/src/server.rs
The minimald networking core now distinguishes local-spawn vs host-shuttle switch transport, routes ingress control over unix or vsock, and updates proxy routing and switch tests.
Guest rootfs and daemon startup
crates/minimald/src/guest.rs, crates/minimald/src/main.rs
The daemon entrypoint now enables rustls ring, switches on in_microvm, and the guest rootfs setup adds writable tmpfs, devpts, and egress bring-up.
Session scaffolding and own-IP attach
crates/mctx/src/*, crates/minimald/src/session.rs, crates/minimald/src/session_host.rs, crates/minimald/src/sessions.rs, crates/minimald/src/test_harness.rs, crates/minimald/tests/netns.rs
Session context scaffolding now fills missing mfiles, the manager shares hostname state through Arc, and own-IP attachments use typed control channels with updated teardown.
minimal2 activation and attach
crates/minimal2/src/autospawn.rs, crates/minimal2/src/main.rs
The CLI now parses network mode and ingress rules, surfaces daemon CreateSession errors, forces a PTY only for interactive attach, and raises the spawn timeout.
Build and validation support
.github/workflows/ci-linux-kvm.yml, justfile, scripts/build-initramfs.sh, docs/specs/03-spec-networking/*
The workflow, justfile, initramfs script, and networking test-plan artifacts now match the new RAM, features, and validation flow.

Sequence Diagram(s)

sequenceDiagram
  participant m2 as "minimal2 cmd_activate"
  participant mh as "sessions::ManagerHandle"
  participant sha as "session_host::attach_own_ip"
  participant ats as "net::switch::attach_to_switch"
  participant atsv as "net::switch::attach_to_switch_vsock"
  participant pol as "net::policy::apply_ingress"
  participant gp as "gvproxy"

  m2->>mh: CreateSession with network and ingress policy
  mh->>sha: start session attachment
  alt SwitchTransport::LocalSpawn
    sha->>ats: attach tap
  else SwitchTransport::HostShuttle
    sha->>atsv: attach tap
  end
  sha->>pol: apply_ingress
  pol->>gp: POST ExposeRequest / UnexposeRequest
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • gominimal/minimal#522: Adds the gvproxy switch supervisor and network-mode plumbing that this PR extends with shuttle and control-channel paths.
  • gominimal/minimal#547: Modifies the OwnIp attach flow and session_host.rs/net/switch.rs path that this PR refactors around transport-aware attachment.
  • gominimal/minimal#556: Adds the static ingress implementation that this PR reworks through ControlChannel and updated teardown handling.

Suggested reviewers

  • twitchyliquid64

Poem

🐰 I hopped through sockets, bright and new,
through gvproxy tunnels, day and through.
New mfiles bloom, and sessions start to sing,
with tiny paws on networking spring. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title is too generic and does not describe the main timeout and RAM changes in the branch. Use a concise, specific title that names the primary change, such as the VM timeout and RAM configuration update.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

Extract the per-PTask switch attach (lease, tap, move-into-netns, frame
relay, static ingress) out of the session host into `GvproxyNetwork`, an
`impl sandbox2::Network`. The session launcher now constructs it and calls
`Network::attach` rather than the inline `attach_own_ip`, and the returned
`NetGuard` is torn down explicitly in `mainloop` at session end instead of
via a runtime-fragile `Drop` schedule.

This puts the networking logic behind the shared sandbox2 abstraction so
the same path can serve tasks, not only minimald sessions, and keeps the
gvproxy process owner (the daemon-scoped SwitchClient / minvmd host
supervisor) outside sandbox2 (spec R1.4).

Refs #581 review (twitchyliquid64): networking belongs in sandbox2 and
should not be session-only; teardown made explicit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/ci-linux-kvm.yml (1)

203-219: 🩺 Stability & Availability | 🟡 Minor

Extend the status poll window. run --detach --timeout 30 only bounds the initial host UDS connect wait; the READY wait still uses the 60s default. The 75 × 0.2s status loop caps out at ~15s, which is shorter than that READY budget and can fail before Starting -> Running on slower KVM boots. Consider widening the poll to match the READY timeout.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci-linux-kvm.yml around lines 203 - 219, The status
polling in the CI KVM workflow is too short and can exit before the VM reaches
Running on slower boots. Update the poll loop in the status check section to
wait long enough to match the READY timeout used by the detach flow, and keep
the same jq-based running-state check around the existing "$BIN" status --json
probe.
🧹 Nitpick comments (2)
crates/minimal2/src/main.rs (1)

489-498: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Surfacing the daemon's typed validation error is the right call. Matching Errorable::Err { error } propagates the daemon's policy/network-mode message (e.g. ingress on a non-OwnIp session, privileged host port) instead of a generic line. Note the --ingress doc says it "Requires --network own-ip" but that is only enforced server-side via validate_policy; relying on the surfaced daemon error is acceptable, though a local pre-check would give faster feedback.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/minimal2/src/main.rs` around lines 489 - 498, Add a client-side
pre-check in main’s session creation flow so `--ingress` is rejected earlier
when the selected network mode is not `own-ip`, rather than waiting for the
daemon response. Keep the existing `minimald_rpc::Errorable::Err { error }`
handling in the session creation match so `CreateSession failed: {error}` still
surfaces the daemon’s typed `validate_policy` message for other validation
failures.
crates/minimald/src/session_host.rs (1)

683-779: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Transport-aware attach and teardown are consistent. The match transport for the relay attach (Lines 723-728) and the ControlChannel construction (Lines 742-747) line up with SwitchTransport/ControlChannel, and the ingress-apply failure path correctly drops the relay before detaching so no half-configured PTask survives.

Minor: the transport is matched twice (tap attach, then control-channel build). If these ever diverge it would be a silent mismatch. Optional consolidation — derive the ControlChannel once up front and reuse it for both — but the current form is readable.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/minimald/src/session_host.rs` around lines 683 - 779, The transport
handling in attach_own_ip is duplicated, which can silently drift between the
relay attach and ControlChannel setup. Consolidate the
SwitchTransport-to-ControlChannel mapping once in attach_own_ip, then reuse that
single derived value for both attach_to_switch/attach_to_switch_vsock and the
control construction so the logic stays consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/minimal2/src/autospawn.rs`:
- Around line 38-43: The spawn timeout parsing in spawn_timeout_secs currently
accepts 0 as a valid override, which can propagate a zero timeout into the
autospawn flow. Update the parsing logic in spawn_timeout_secs so that
MINIMAL_SPAWN_TIMEOUT_SECS values of 0 (and any non-positive input) are rejected
and the code falls back to DEFAULT_SPAWN_TIMEOUT_SECS or a minimum allowed
value, keeping the existing trim/parse behavior for valid positive integers.

In `@crates/minimald/src/net/switch.rs`:
- Around line 314-321: Wrap the vsock handshake in `switch.rs` so
`tokio_vsock::VsockStream::connect` and the initial `AsyncWriteExt::write_all`
of `CONNECT_REQUEST` are both covered by a `tokio::time::timeout`. Update the
relay setup path around `spawn_relay` to return a best-effort error on timeout
instead of waiting indefinitely, and keep the async trait disambiguation for
`VsockStream` unchanged. Use the existing connect-and-split flow in this
function as the place to apply the deadline.

In `@crates/minimald/src/session.rs`:
- Around line 312-317: The `context()` path currently uses
`tokio::task::block_in_place`, which can panic on current-thread runtimes such
as default `#[tokio::test]` callers. Replace the blocking section in
`session::context()` around `mctx::scaffold_default_mfile` with `spawn_blocking`
or otherwise move the scaffolding outside the async task, and keep the existing
`map_err`/`String` error handling intact so empty-workspace cases return an
error instead of panicking.

In `@crates/minvmd/src/cmd/mod.rs`:
- Around line 60-66: The ready_timeout() helper currently accepts a parsed value
of 0, which can make boot time out immediately; update the parsing in
ready_timeout() to reject zero and fall back to DEFAULT_READY_TIMEOUT_SECS,
matching the validation style used by vm_ram_mib(). Keep the fix localized to
the READY_TIMEOUT_ENV handling so that boot.rs continues to receive a non-zero
Duration from ready_timeout().

In `@crates/minvmd/src/net.rs`:
- Around line 922-950: The readiness signal in spawn is sent too early after
obtaining the child PID, so gvproxy can be marked ready before the switch socket
is actually listening. Update the spawn flow in GvproxyConfig::spawn to wait for
switch_sock to become connectable, or otherwise race that wait against
exit.recv(), before sending Ok(pid) through ready_tx so run only proceeds once
the switch is usable.
- Around line 1405-1440: The HostGvproxy tests are using a real binary
placeholder that does not accept the argv passed by HostGvproxy::spawn, so the
process may exit immediately instead of behaving like a long-lived supervised
gvproxy. Replace the direct use of sleep in
host_gvproxy_spawns_supervises_and_stops and host_gvproxy_drop_stops_the_switch
with a fake gvproxy helper that accepts and ignores the gvproxy arguments while
staying alive long enough for supervision and stop/drop behavior to be verified.
Keep the assertions focused on HostGvproxy::spawn, pid(), stop(), and drop()
semantics.

In `@crates/minvmd/src/net/shuttle.rs`:
- Around line 72-78: The test in switch_sock_sits_beside_the_bridge_socket
mutates XDG_RUNTIME_DIR via std::env, which is process-global and can race with
parallel tests. Refactor the path logic used by resolve_switch_sock into a pure
helper that derives the socket path from an explicit runtime dir argument, and
have the test assert against that helper directly; if env mutation must remain,
guard the set_var/remove_var pair with the existing test mutex.

In `@justfile`:
- Around line 126-140: The up recipe is missing a pinned MINVMD_VM_RAM_MIB, so
x86_64 Linux hosts can fall back to the unsafe default RAM size and hit the same
MMIO-hole boot failure CI avoids. Update the justfile’s up recipe to set
MINVMD_VM_RAM_MIB for x86_64 Linux, mirroring the existing CI safeguard, and
keep the change scoped alongside the other MINVMD_* environment exports in up.

---

Outside diff comments:
In @.github/workflows/ci-linux-kvm.yml:
- Around line 203-219: The status polling in the CI KVM workflow is too short
and can exit before the VM reaches Running on slower boots. Update the poll loop
in the status check section to wait long enough to match the READY timeout used
by the detach flow, and keep the same jq-based running-state check around the
existing "$BIN" status --json probe.

---

Nitpick comments:
In `@crates/minimal2/src/main.rs`:
- Around line 489-498: Add a client-side pre-check in main’s session creation
flow so `--ingress` is rejected earlier when the selected network mode is not
`own-ip`, rather than waiting for the daemon response. Keep the existing
`minimald_rpc::Errorable::Err { error }` handling in the session creation match
so `CreateSession failed: {error}` still surfaces the daemon’s typed
`validate_policy` message for other validation failures.

In `@crates/minimald/src/session_host.rs`:
- Around line 683-779: The transport handling in attach_own_ip is duplicated,
which can silently drift between the relay attach and ControlChannel setup.
Consolidate the SwitchTransport-to-ControlChannel mapping once in attach_own_ip,
then reuse that single derived value for both
attach_to_switch/attach_to_switch_vsock and the control construction so the
logic stays consistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 435cea8e-037a-4034-b48e-2148be48282e

📥 Commits

Reviewing files that changed from the base of the PR and between bf05ff4 and 0952088.

📒 Files selected for processing (30)
  • .github/workflows/ci-linux-kvm.yml
  • crates/mctx/src/lib.rs
  • crates/mctx/src/scaffold.rs
  • crates/minimal2/src/autospawn.rs
  • crates/minimal2/src/main.rs
  • crates/minimald/src/guest.rs
  • crates/minimald/src/main.rs
  • crates/minimald/src/net/mod.rs
  • crates/minimald/src/net/policy.rs
  • crates/minimald/src/net/proxy.rs
  • crates/minimald/src/net/switch.rs
  • crates/minimald/src/server.rs
  • crates/minimald/src/session.rs
  • crates/minimald/src/session_host.rs
  • crates/minimald/src/sessions.rs
  • crates/minimald/src/test_harness.rs
  • crates/minimald/tests/netns.rs
  • crates/minvmd/src/cmd/boot.rs
  • crates/minvmd/src/cmd/mod.rs
  • crates/minvmd/src/cmd/run.rs
  • crates/minvmd/src/cmd/status.rs
  • crates/minvmd/src/cmd/vmm_child.rs
  • crates/minvmd/src/image.rs
  • crates/minvmd/src/net.rs
  • crates/minvmd/src/net/shuttle.rs
  • crates/minvmd/src/vm.rs
  • docs/specs/03-spec-networking/test-plan.md
  • docs/specs/03-spec-networking/test-plan.sh
  • justfile
  • scripts/build-initramfs.sh

Comment thread crates/minimal2/src/autospawn.rs
Comment thread crates/minimald/src/net/switch.rs
Comment thread crates/minimald/src/session.rs
Comment thread crates/minvmd/src/cmd/mod.rs
Comment thread crates/minvmd/src/net.rs
Comment thread crates/minvmd/src/net.rs
Comment thread crates/minvmd/src/net/shuttle.rs
Comment thread justfile
norrietaylor and others added 2 commits June 26, 2026 11:14
`run_with_cancel` now attaches the configured `Network` to each
invocation's freshly-unshared netns after spawn and tears it down
explicitly once the invocation completes (both the cancel and completion
arms). No-op for `HostNet`/`NoNet` and when no custom `Network` is set, so
existing build/task consumers are unaffected.

This is the structural completion of the #581 review (twitchyliquid64):
networking is now applied by the shared run path, so any sandbox consumer
— tasks and builds, not only minimald sessions — gets it through the
abstraction. Add `Error::Network` for attach failures; the borrow is
scoped to `&dyn Network` (Send+Sync) so it imposes no `C: Sync` on the
run future.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host gvproxy switch spawns with an empty static-lease table (the guest
assigns PTask IPs statically over the vsock shuttle), so the in-process
lease/attach machinery was never exercised in production — only by its own
tests. Remove `PtaskAttachment`, `attach_ptask`, `allocate_ptask`,
`detach_ptask`, `allocate_ip`, and the now-dead `next_index`/`attached_count`
fields (and the `subnet` field, read only by `allocate_ip`), plus their
tests. The supervisor keeps its real job: spawn, pidfd-supervise, and
stop the one host gvproxy process.

Refs #581 review (twitchyliquid64).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor
norrietaylor force-pushed the fix/vm-bringup-timeouts-ram branch from 0952088 to cfb03c2 Compare June 26, 2026 18:41
norrietaylor and others added 2 commits June 26, 2026 11:51
…itives

The gvproxy switch subnet, MAC derivation, wire constants, and `-config`
rendering had drifted into parallel copies in minimald and minvmd (the
SwitchSubnet types had even diverged: /8..=/29 vs /1..=/30 validation,
Option vs non-Option accessors, separate error types). Collapse them into
one definition in `minimald-rpc` (the crate both already depend on),
unified on minimald's design (the richer one the netns path uses):
SwitchSubnet (/8..=/29, non-Option accessors, daemon_ip), MacAddr,
GATEWAY_MAC, DEFAULT_MTU, DEFAULT_SUBNET, the VSOCK constants, and
render_gvproxy_config (tuple leases). Both crates re-export them, so
`minimald::net::*` / `minvmd::net::*` keep working.

minvmd adapts: its render drops the now-unnecessary Option handling and
its SwitchSubnet API tests (which asserted the old /1,/30,Option
behaviour) are dropped — the shared type is tested in minimald-rpc.

No behaviour change for the deployed /16 subnet. Refs #581 review
(twitchyliquid64).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rable

`just up` / boot e2e could not bring up the VM for two distinct,
environment-dependent reasons; both are now configurable with safe,
arch-aware defaults.

1. Timeouts too short. A cold multi-GiB VM on the generic guest kernel
   takes ~22-28s to reach userspace (longer under load), but the
   READY-marker wait (boot/run) was a fixed 5s and the autospawn UDS wait
   8s -- both firing before the kernel reached /init.
   - minvmd: cmd::ready_timeout() (env MINVMD_READY_TIMEOUT_SECS, default
     60s), shared by boot and run; the timeout error names the override.
   - minimal2: autospawn UDS wait via MINIMAL_SPAWN_TIMEOUT_SECS (default
     75s). All three helpers reject 0 -- a zero timeout would give the
     boot no readiness window / make recv_timeout fail instantly.

2. x86_64 boot panic. A 4096 MiB guest boots on aarch64 but on x86_64
   exactly 4 GiB straddles the 32-bit MMIO/PCI hole (~3-4 GiB),
   mis-placing the initramfs so the kernel finds no /init and panics in
   prepare_namespace ("VFS: cannot open root device"). Guest RAM is now
   cmd::vm_ram_mib() (env MINVMD_VM_RAM_MIB; status reports the resolved
   value), and DEFAULT_VM_RAM_MIB is arch-conditional -- 2048 on x86_64
   (CI-proven, hole-safe), 4096 on aarch64 -- so every boot path (CI,
   `just up`, any host) is safe without per-lane pinning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor
norrietaylor force-pushed the fix/vm-bringup-timeouts-ram branch from cfb03c2 to a261d89 Compare June 26, 2026 18:58
norrietaylor added a commit that referenced this pull request Jun 26, 2026
- Bound the host-shuttle vsock connect + `/connect` upgrade with a 5s
  timeout so an unresponsive/absent host gvproxy fails the OwnIp attach fast
  instead of stalling bring-up.
- Guard the scaffold `block_in_place` by runtime flavor: it panics on a
  current-thread runtime, so fall back to a plain blocking call there.
- RenameSession now re-points the HostNet hostname route to the new name
  (was stranded under the launch name).

Refs #581/#585 review (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
norrietaylor added a commit that referenced this pull request Jun 26, 2026
HostGvproxy::spawn now polls gvproxy's `-listen` socket (5s) before sending
Ok(pid), so a switch that died during startup is surfaced as an error
rather than reported network-ready. Also factor switch-socket derivation
into a pure helper so its test no longer mutates process-global
XDG_RUNTIME_DIR. Refs #585 review (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@norrietaylor

Copy link
Copy Markdown
Member Author

Superseded — this fix was merged into #581 as 3290a6e (configurable READY/autospawn timeouts with 0-guards + arch-conditional guest RAM default; x86_64 boot_e2e validated green). Closing the stacked test PR.

@norrietaylor
norrietaylor deleted the fix/vm-bringup-timeouts-ram branch June 26, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant