feat(minimald,sandbox2): rootless host-native (DM2) own-ip networking - #589
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds own-IP TAP/DNS plumbing across sandbox2 and minimald, splits gvproxy attach handling for HostShuttle and LocalSpawn, introduces deployment recipes, updates dependency pins and initramfs build selection, and adjusts the networking test plan. ChangesOwn-IP networking and launch flow
Dependency and deployment tooling
Networking test plan
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant SandboxLauncher
participant SwitchClient
participant sandbox2
participant GvproxyNetwork
participant SessionHost
SandboxLauncher->>SwitchClient: phase 1 own-IP attach and lease
SwitchClient-->>SandboxLauncher: lease_ip + control socket
SandboxLauncher->>sandbox2: build Env with own_ip_tap / own_ip_dns
sandbox2-->>SandboxLauncher: spawn process
alt spawn or build fails
SandboxLauncher->>SwitchClient: rollback detach via guard
else spawn succeeds
SandboxLauncher->>sandbox2: read rustslirp_tapfd()
SandboxLauncher->>GvproxyNetwork: complete_local_own_ip_attach(tap_fd, lease_ip)
GvproxyNetwork->>SwitchClient: finish_own_ip_attach(lease_ip, ingress)
SandboxLauncher->>SessionHost: disarm rollback guard
end
Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
You dont need to do this yourself, this was implemented upstream in hakoniwa (which already does CLONE_NEWNET). Call See here for some rough code setting up the tap + getting it out of hakoniwa. |
|
Thanks — this is a much cleaner direction! There's one blocker for the gvproxy switch topology, though. Claude says:
We may need a small RustSlirp addition — a gateway/next-hop option so the in-netns default route can be 0.0.0.0/0 via . |
|
Sounds good, we can get that into hakoniwa. Sounds like the addition is the ability to configure the default gateway? Do you still need next-hop if we have default gateway? |
|
Done, added |
Yup if we can configure the default gateway we are good. |
d592056 to
894fcbe
Compare
|
Pinned our
The break is at
So Would you be open to pinning the I can't |
Replace the native (DM2) own-IP tap setup with hakoniwa's RustSlirp, which builds and configures the tap *inside* the sandbox's own user+net namespace (rootless — it acts as container-root and needs no host CAP_NET_ADMIN). This removes the privileged path entirely: no `setcap`, no `PR_SET_DUMPABLE`, and no in-process `setns` + tap ioctls. Per review on #589 (thanks @twitchyliquid64): the own-IP tap creation now lives behind `container.network(RustSlirp)` and the tap fd comes back via `Child.rustslirp_tapfd`, which minimald relays to the gvproxy switch. - sandbox2: a `Config::own_ip_tap` (`OwnIpTap { address, netmask, gateway, mtu }`) drives `container.network(RustSlirp::TAP … gateway(Address(gw)))` in `new_container`, after the netns unshare. - minimald: `EnvArgs::with_own_ip_tap` threads the lease into the sandbox. The session launch splits into two phases because RustSlirp needs the address before spawn: phase 1 (pre-spawn) allocates the lease + ensures gvproxy for the native `LocalSpawn` transport; phase 2 (post-spawn) wraps `Child.rustslirp_tapfd` and relays it via `complete_local_own_ip_attach`. Build/spawn failures roll the phase-1 attach back so gvproxy's refcount stays accurate. - The in-VM DM1/3/4 (`HostShuttle`) path is unchanged — `attach_own_ip` keeps the proven open-tap + move-into-netns + vsock relay (minimald is root there); the two paths share `finish_own_ip_attach` (ingress + guard). - Delete the now-dead in-process tap code (`open_tap_in_netns*`, `open_netns_fd`, `configure_switch_interface`) and `restore_sandbox_dumpable`. - `dm2` recipe: drop the `setcap` step — own-IP is rootless now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Pushed the RustSlirp swap (rootless own-IP) per @twitchyliquid64's suggestion — 2 commits on top of the interim privileged path:
Verification caveat: I could not verify the rootless path end-to-end in my dev environment — it's a nested lima/user-namespace setup where hakoniwa's own |
Make own-IP work for an unprivileged, `setcap`'d host-native minimald (the
DM2 deployment model: native Linux, no VM), end to end. Three coupled
changes are needed; none alone is sufficient:
1. In-process tap setup (`net/switch.rs`, `net/gvproxy_network.rs`). The
per-PTask tap is created and configured (MAC, IP, netmask, default
route, link-up) directly via `setns(CLONE_NEWNET)` + AF_INET ioctls,
with the PTask's netns fd pinned before the gvproxy-spawning switch
attach (a short-lived PTask can exit during that window, and a dead
process's `/proc/<pid>/ns/net` vanishes). This removes the privileged
`ip`/`nsenter` child processes, whose `setcap +ep` is effective-only
and not inherited by children. DM1/3/4 (root-in-VM, HostShuttle) keep
the existing move-into-netns path.
2. Dumpable reset (`main.rs`). Gaining file capabilities at `execve` sets
the process dumpable flag to SUID_DUMP_ROOT, which makes
`/proc/<pid>/{uid_map,gid_map,setgroups}` root-owned. A forked hakoniwa
sandbox inherits that and can no longer write its own `/proc/self/
uid_map` as the unprivileged user — EPERM, breaking every session
(own-IP or not). `prctl(PR_SET_DUMPABLE, 1)` at startup restores it.
3. Gateway DNS (`sandbox2` + `env.rs`). An own-IP sandbox runs in a fresh
netns where the synthesized `/etc/resolv.conf` (the host's 127.0.0.53
systemd-resolved stub) is unreachable. A new `Network::nameserver()`
and a `Config::dns_nameserver` (set from the network mode) let the
sandbox builder write `nameserver <switch-gateway>` into the rootfs
before spawn — gvproxy serves DNS at the gateway, already the PTask's
default route. Written unconditionally to overwrite the synth default.
Also adds a `--gvproxy-bin` flag to `minimald run` so the per-host own-IP
switch can use a local gvproxy build without a system install.
Verified on native Linux: own-IP session resolv.conf -> 100.64.0.1,
`curl http://example.com` -> HTTP 200.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`build-initramfs.sh` always shelled out to `cross` (Docker), so hosts without Docker could not build the guest initramfs (`cross: not found`). Auto-detect a same-arch native static-musl toolchain (the `*-linux-musl` rustup target plus a matching `*-linux-musl-gcc` linker) and build natively in that case, deriving the cargo linker-override var from the target triple; otherwise fall back to `cross`. `FORCE_CROSS=1` keeps the container path. This is what lets the `dm3`/`up` recipes build the guest on a plain Linux host. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Explicit per-deployment-model bring-up recipes: - `dm1` — macOS + Linux VM over Hypervisor.framework (the `up` path on macOS); a clean SKIP on Linux. - `dm2` — native Linux host-native minimald (no VM) over UDS. Builds the daemon, applies `setcap cap_net_admin,cap_sys_admin=ep` (re-applied each bring-up; `cargo build` strips file caps), and starts it with `--gvproxy-bin` so own-IP needs no system gvproxy install. - `dm3` — native Linux + one Linux VM, bridging the CLI socket to minvmd's guest bridge (the F1 path mismatch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enable hakoniwa's `rustslirp` feature for rootless own-IP, pinned to the gominimal fork of souk4711/hakoniwa (rev 44969f2) which carries the RustSlirp `gateway()` next-hop-route support (`RustSlirpGateway:: IfaceWithAddr`). Repoint to upstream once that lands. The rustslirp feature pulls tun-rs 2.8.3 -> c2rust-bitfields 0.22, whose derive macro exact-pins `proc-macro2 = "=1.0.103"`. clap_derive 4.6 requires `proc-macro2 >= 1.0.106`, which the resolver cannot unify with `=1.0.103`. Holding clap/clap_complete at 4.5 (which needs only `proc-macro2 ^1.0`) lets the resolver settle on 1.0.103. Bump back to 4.6 once tun-rs/c2rust relax that pin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the native (DM2) own-IP tap setup with hakoniwa's RustSlirp, which builds and configures the tap *inside* the sandbox's own user+net namespace (rootless — it acts as container-root and needs no host CAP_NET_ADMIN). This removes the privileged path entirely: no `setcap`, no `PR_SET_DUMPABLE`, and no in-process `setns` + tap ioctls. Per review on #589 (thanks @twitchyliquid64): the own-IP tap creation now lives behind `container.network(RustSlirp)` and the tap fd comes back via `Child.rustslirp_tapfd`, which minimald relays to the gvproxy switch. - sandbox2: a `Config::own_ip_tap` (`OwnIpTap { address, netmask, gateway, mtu }`) drives `container.network(RustSlirp::TAP … gateway(IfaceWithAddr(gw)))` in `new_container`, after the netns unshare. The next-hop route (`0.0.0.0/0 via gateway`) is required — gvproxy is a real gateway and does not proxy-ARP, so RustSlirp's default on-link route breaks egress. - minimald: `EnvArgs::with_own_ip_tap` threads the lease into the sandbox. The session launch splits into two phases because RustSlirp needs the address before spawn: phase 1 (pre-spawn) allocates the lease + ensures gvproxy for the native `LocalSpawn` transport; phase 2 (post-spawn) wraps `Child.rustslirp_tapfd` and relays it via `complete_local_own_ip_attach`. Build/spawn failures roll the phase-1 attach back so gvproxy's refcount stays accurate. - The in-VM DM1/3/4 (`HostShuttle`) path is unchanged — `attach_own_ip` keeps the proven open-tap + move-into-netns + vsock relay (minimald is root there); the two paths share `finish_own_ip_attach` (ingress + guard). - Delete the now-dead in-process tap code (`open_tap_in_netns*`, `open_netns_fd`, `configure_switch_interface`) and `restore_sandbox_dumpable`. - `dm2` recipe: drop the `setcap` step — own-IP is rootless now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
0140f81 to
5bd31c2
Compare
`host_gvproxy_spawns_supervises_and_stops` and `host_gvproxy_drop_stops_the_switch` spawned a bare `sleep` as the gvproxy stand-in, but `HostGvproxy::spawn` hands the launched binary gvproxy's argv (`-config … -listen … -ssh-port -1`). `sleep` rejects those flags and exits within ~1 ms, so the supervisor's background reaper races the `pid_is_alive` assertions — passing locally but flaking on slow/contended CI runners (the assert saw the process already reaped). The test's "sleep ignores argv" comment was simply wrong. Use a stand-in that stays alive regardless of the argv it is handed — a tiny script that `exec`s a long sleep — so the liveness and teardown assertions are deterministic, matching real gvproxy's run-until-signalled behaviour. Test-only; no production change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RustSlirp own-IP verified working ✅ + the
|
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/minimald/src/env.rs`:
- Around line 264-271: The DNS nameserver for the OwnIp network mode is using
the default subnet gateway instead of the actual gateway from the DM2 TAP setup.
Update the env-building logic in the `with_dns_nameserver` call near
`args.network_mode` to source the gateway from the same DM2 subnet/TAP
parameters used by `net_switch.subnet()` so `/etc/resolv.conf` matches the TAP
route. Keep the existing non-OwnIp behavior unchanged and ensure the
`with_own_ip_tap` path uses the same gateway value consistently.
In `@crates/minimald/src/session_host.rs`:
- Around line 680-707: The phase-1 Own-IP lease in session_host’s launch flow is
not cancellation-safe, because the switch attach/refcount is acquired before
several await points and the existing rollback only runs on an Err return. Add
an armed cleanup guard around the attach/lease work in the launch path so it
releases the switch if the future is dropped or cancelled, and only disarm it
once ownership has been transferred into OwnIpGuard/local_own_ip. Keep the
cleanup tied to the launch/build_and_spawn flow and the OwnIpGuard handoff so
the gvproxy attach count cannot leak.
In `@crates/sandbox2/src/lib.rs`:
- Around line 526-538: Reject own_ip_tap unless the sandbox is isolated by
checking self.config.own_ip_tap before the container.network setup in the
networking path. In the logic around RustSlirp::TAP, add an early return with
ExecutionError::NetworkIsolationUnavailable when own_ip_tap is present but
isolate is false (including HostNet or any custom Network that does not unshare
the netns), and only build the RustSlirp configuration after that guard passes.
In `@scripts/build-initramfs.sh`:
- Around line 37-45: The native musl build path in build-initramfs.sh only sets
the cargo linker override via LINKER_VAR, but ring’s build script also needs a
target-scoped CC to use $MUSL_CC. Update the same conditional branch in
build-initramfs.sh to export the appropriate CC_<target> environment variable
alongside CARGO_TARGET_*_LINKER before invoking cargo, using the same
TARGET/MUSL_CC values already computed there.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 58b3b125-b4a6-453b-a11b-b3fb1403b3d6
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (11)
Cargo.tomlcrates/minimald/src/env.rscrates/minimald/src/main.rscrates/minimald/src/net/gvproxy_network.rscrates/minimald/src/session_host.rscrates/minvmd/src/net.rscrates/sandbox2/src/config.rscrates/sandbox2/src/lib.rscrates/sandbox2/src/network.rsjustfilescripts/build-initramfs.sh
|
|
||
| fn nameserver(&self) -> Option<std::net::Ipv4Addr> { | ||
| // An own-IP PTask lives in a fresh netns where the host's stub resolver | ||
| // (`127.0.0.53`, baked into the synth rootfs) is dead, so DNS fails even |
There was a problem hiding this comment.
Whats 127.0.0.53? Wouldnt that resolve to loopback in the sandboxed net ns?
There was a problem hiding this comment.
Good catch, this is actually dead code that didn't get removed after a previous bug fix/refactor.
I am going to clean this up a consolidate the DNS code into one place.
| # 0.22 -> proc-macro2 =1.0.103 pin is accommodated by holding clap at 4.5; see | ||
| # the clap note above.) Repoint to upstream hakoniwa once the gateway support | ||
| # lands there. | ||
| hakoniwa = { git = "https://github.com/gominimal/hakoniwa.git", rev = "44969f29201c47bfad41f8942d5afe89c3ea398c", features = ["cgroups", "rustslirp"] } |
There was a problem hiding this comment.
This just got merged upstream, so we can switch back to the souk4711 one.
Three review-driven fixes to the own-IP path (#589): - DNS consolidation (Tom, CodeRabbit #1). The "own-IP DNS lives at the switch gateway" fact was re-derived in three places and split across a dead `GvproxyNetwork::nameserver()` (never reached on the live path, which sets `config.network = None`) and `env.rs`'s independent `DEFAULT_SUBNET.gateway()`. Add `switch::SwitchSubnet::dns_server()` as the single source of truth; have the sandbox write `/etc/resolv.conf` from the live `own_ip_tap.gateway` (so the resolver and the tap route come from one value, not two derivations); point `guest.rs` at `dns_server()`; and delete the now-redundant `Config::dns_nameserver`, `Network::nameserver()`, and `GvproxyNetwork::nameserver()`. - Reject own-IP tap without netns isolation (CodeRabbit #3). Configuring `container.network(RustSlirp)` against a shared netns silently no-ops (hakoniwa skips the setup, leaving no tap fd), so `sandbox2` now returns `NetworkIsolationUnavailable` when `own_ip_tap` is set but the sandbox is not isolated. - Cancel-safe phase-1 rollback (CodeRabbit #2). The pre-spawn own-IP attach bumps gvproxy's count before the slow env build; an `Err` was handled but a dropped/cancelled launch future leaked it. A `PhaseOneAttachGuard` now rolls the attach back on drop (spawned detach, since `Drop` can't await), disarmed once ownership passes to `OwnIpGuard`. `complete_local_own_ip_attach` and `finish_own_ip_attach` no longer self-detach on the LocalSpawn path (the guard owns it); `attach_own_ip` (HostShuttle) keeps its inline rollback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The native same-arch musl branch set `CARGO_TARGET_*_LINKER` but not `CC_<triple>`, which cc-rs (used by ring's build script to compile its C/asm) reads to pick the compiler. Export it alongside the linker so the musl toolchain is used for the C objects too, not just the link (CodeRabbit #4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The RustSlirp `gateway()` next-hop support (`RustSlirpGateway::IfaceWithAddr`) merged into souk4711/hakoniwa upstream, so drop the interim gominimal fork pin and track upstream directly (rev bc0abab). API-identical — no code change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/minimald/src/net/gvproxy_network.rs (1)
130-183: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy liftAdd a cancellation guard after
SwitchClient::attach().After the attach count is bumped, several
.awaits run beforeOwnIpGuardexists. If the launch future is dropped instead of returningErr, the detach branches at Lines 167 and 180 never run, leaking the gvproxy attach count. Mirror the LocalSpawn phase-one guard or add an armed rollback guard that disarms only afterOwnIpGuardis constructed.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/minimald/src/net/gvproxy_network.rs` around lines 130 - 183, `attach_own_ip` leaks the switch attach count if the future is cancelled after `SwitchClient::attach()` succeeds but before `OwnIpGuard` is created. Add a cancellation-safe rollback guard immediately after the attach in `attach_own_ip` (before the later awaits) that detaches the switch on drop, and disarm it only after `finish_own_ip_attach` returns the guard successfully. This should mirror the existing LocalSpawn phase-one guard behavior and keep the rollback logic tied to `SwitchClient::attach()`, `finish_own_ip_attach`, and the final `OwnIpGuard` construction.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/minimald/src/env.rs`:
- Around line 259-264: The OwnIp path in env setup drops DNS rewriting when
`.with_own_ip_tap(args.own_ip_tap)` is `None`, which leaves HostShuttle DM1/3/4
sessions using the wrong resolver. Update the `EnvArgs`/`sandbox2` flow so
`OwnIp` carries an explicit DNS server for the active switch subnet even without
DM2 tap params, and ensure the `/etc/resolv.conf` rewrite in `sandbox2` uses
that shared DNS value instead of relying only on `own_ip_tap`.
---
Outside diff comments:
In `@crates/minimald/src/net/gvproxy_network.rs`:
- Around line 130-183: `attach_own_ip` leaks the switch attach count if the
future is cancelled after `SwitchClient::attach()` succeeds but before
`OwnIpGuard` is created. Add a cancellation-safe rollback guard immediately
after the attach in `attach_own_ip` (before the later awaits) that detaches the
switch on drop, and disarm it only after `finish_own_ip_attach` returns the
guard successfully. This should mirror the existing LocalSpawn phase-one guard
behavior and keep the rollback logic tied to `SwitchClient::attach()`,
`finish_own_ip_attach`, and the final `OwnIpGuard` construction.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7a2daaf1-ddc6-4c67-b01f-eea889d9cfe2
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (9)
Cargo.tomlcrates/minimald/src/env.rscrates/minimald/src/guest.rscrates/minimald/src/net/gvproxy_network.rscrates/minimald/src/session_host.rscrates/sandbox2/src/config.rscrates/sandbox2/src/lib.rscrates/switch/src/lib.rsscripts/build-initramfs.sh
💤 Files with no reviewable changes (1)
- crates/sandbox2/src/config.rs
🚧 Files skipped from review as they are similar to previous changes (3)
- scripts/build-initramfs.sh
- crates/minimald/src/session_host.rs
- Cargo.toml
The DNS consolidation sourced `/etc/resolv.conf` from `own_ip_tap.gateway`, but `own_ip_tap` is only set on the native DM2 (`LocalSpawn`) path — so DM1/3/4 (`HostShuttle`, in-VM) own-IP sessions lost their resolver override and kept the synth rootfs's dead host stub (`127.0.0.53`) in an isolated netns (CodeRabbit). Decouple DNS from the tap: a new `Config::own_ip_dns` is set for *every* own-IP sandbox from the live `net_switch.subnet().dns_server()` (both transports), and the sandbox writes resolv.conf from it. This keeps the single-live-source property (no drift from the tap route, which comes from the same subnet) while covering the shuttle path. Also collapses the phase-1 double switch-lock into one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TC2's same-host-peer step drove two concurrent `minimal attach` sessions from
a single `expect` via two spawn ids read one-at-a-time. `attach` exec's
`ssh -tt`, whose pty has a fixed ~16KB kernel buffer; while the one process
drained session A, session B's ssh kept writing shell-setup into B's unread
pty until it filled and ssh blocked on write, so B's RequestShell never
reached the daemon. That looked like a server-side own-ip attach wedge ("first
attach works, the second hangs") but was the client starving its own pty —
and it cascaded 360s step-timeout SIGKILLs onto TC3/4/7/8.
Fixes (harness only; no product change):
- One pty per process: TC2 now runs the peer listener as a backgrounded
`run_in_session` and dials it from a separate `session_out`, each draining
its own pty. Documented as a RULE in the header.
- Line-buffer expect stdout (`fconfigure stdout -buffering line`) in
`run_in_session` and the TC3/4/7/8 blocks, so a gtimeout SIGKILL can't
discard a verdict stuck in expect's block buffer.
- Warm peer's sandbox (via the switch-IP read) before the timed trial, and
retry demo's connect to absorb listener-bind latency.
Verified locally on DM2 (rootless own-ip): TC1/TC1b/TC2 egress pass and the
same-host peer connect returns RC=0 with no wedge and no step-timeout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/specs/03-spec-networking/test-plan.sh`:
- Line 157: The TC2 verdict in test-plan.sh is using the shell exit status from
the loop tail, so a failed reachability check can still print success; update
the session_out demo around the curl retry loop to track whether curl ever
succeeded instead of relying on the final $?. Use the existing TC2_RC reporting
in the same demo block, and set it based on an explicit success/failure flag or
the loop outcome so the printed result reflects whether the peer was actually
reached.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 92cb95d0-54dd-450a-aff1-215848c41d0a
📒 Files selected for processing (1)
docs/specs/03-spec-networking/test-plan.sh
The TC2 peer retry loop ended each failing iteration with `sleep 1`, so `echo TC2_RC=$?` read the sleep's exit (always 0) and reported success even when curl never reached the peer — masking a real failure. Track reachability in an explicit flag instead of the trailing `$?`, and assert on the response body: socat's `printf PEER_REACHED` is a raw (HTTP/0.9) reply, so curl's exit status is unreliable — `--http0.9` surfaces the body, which is grepped for the actual proof. Emits `TC2_PEER=YES|NO`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#542) Land the deferred OwnIp DNS registration and align host->PTask routing with the DM2 topology in networking-with-diagrams.md (the daemon is not on the gvproxy switch). An OwnIp PTask now registers its hostname on launch, resolving to 127.0.0.1 (like HostNet) rather than its switch IP: the PTask is reached through a gvproxy-published loopback port (its forwarder binds 127.0.0.1:<external> -> lease:<internal>, the same mechanism static ingress uses), so the host-side proxies never need switch access and no CAP_NET_ADMIN is required. The client selects the published external port; the registry gates only on the host. - dns: add register_own_ip (-> 127.0.0.1); rewrite the module doc from the switch-IP model to the published-loopback model. - sessions: register OwnIp hostnames on launch and follow renames, in addition to HostNet (NoNet still registers nothing). - proxy: replace the own_ip_routes_to_its_switch_ip test with own_ip_routes_to_its_published_loopback_port. Verified end-to-end on DM2 native: with an own-IP PTask publishing :8080 via ingress 18080:8080, a host request to web.local.min.internal:18080 through the :7654 proxy reaches the in-sandbox listener (host -> proxy -> gvproxy forward -> lease:8080). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A session is tmux-like: detach (ctrl-w) or an abrupt client disconnect holds the shell and its network open; only the shell exiting (or an explicit kill/destroy) tears the network down. Add two session-host tests over a recording NetGuard to lock this in: - exit_releases_the_network: the shell process exiting drives net_guard.teardown() in the host mainloop. - detach_keystroke_holds_the_session_and_network: a ctrl-w keystroke is swallowed as a detach signal (not forwarded, not fatal), the shell keeps running, and the network is only released on a later kill. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ished port The e2e harness now treats a session like tmux: it DETACHES (ctrl-w) between assertions instead of sending `exit`, so a session's shell and its own-ip lease stay alive and stable across the multiple attaches a test makes (killing the per-attach lease churn and letting a backgrounded in-session server survive for host-side curls). Also: - unprivileged ports only (:8080): the sandbox has no CAP_NET_BIND_SERVICE, so `socat TCP-LISTEN:80` fails with EACCES; - per-call nonce markers, since reattach replays screen state that may still show a prior call's markers; - drop the flaky inline `proc ready` blocks for the reliable run_in_session path. TC3 (UC2a) now reflects the published-loopback model: activate with --ingress 18080:8080 and reach the service BY HOSTNAME at web.local.min.internal:18080 through the :7654 proxy. Status table and per-TC notes updated to the verified DM2 results. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…/R3.3/R4.4) Resolve the contradiction between the normative requirements (which said an OwnIp hostname resolves to its gvproxy switch IP, reached via the switch) and the DM2 topology in networking-with-diagrams.md (daemon not on the switch; host->PTask via gvproxy-published 127.0.0.1 ports) in favour of the published-loopback model: - R3.1/R3.3: an OwnIp PTask resolves to 127.0.0.1 and is reached through its gvproxy-published loopback port (R2.3), not the switch IP. - R4.4: the mTLS reverse proxy forwards to the published loopback port. - Proof Artifact 2 (UC2a) now curls the published port by hostname. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fill the previously-empty DM2/DM3/DM4 result columns with real 2026-07-03 end-to-end runs on a native Linux/aarch64 host (DM=dm2, DM=dm3, plus TC15 for DM4). AppArmor's unprivileged-userns restriction was disabled and the binaries rebuilt first (the prior build predated the #589 DM2 own-ip feature under test). Key results: - DM2 (native, no VM): TC1/1b/2/5/6/9/10 PASS; TC3/TC4/TC8 FAIL and TC7 with-cert FAIL, all with the in-session backend confirmed live. - DM3 (KVM VM): same PASS set; TC3/TC7 return 502 matching DM1. - DM4: TC15 PASS; G-N1 proxy-port contention confirmed observably (second daemon loses the bind with EADDRINUSE, one listener remains). Findings added to the gap register: - G-N9: the host->PTask ingress forward reaches the backend in the daemon netns, not the session's sandbox netns. Code-confirmed for TC8 (direct-tcpip validates the session UUID then connects to 127.0.0.1:<int> in the daemon netns). Reproduced with a live backend and no VM on DM2, ruling the VM out. - G-N8: on the KVM VM, attach of an own-ip session carrying --ingress hangs (shell never starts, >200s vs ~8s without ingress), so the backend never comes up. DM3's 502 is thus a different mechanism than DM1/DM2 -- TC11 parity holds on the verdict, not the mechanism. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Wjns5t5SQFRvNWmYqXdQ5
Fill the previously-empty DM2/DM3/DM4 result columns with real 2026-07-03 end-to-end runs on a native Linux/aarch64 host (DM=dm2, DM=dm3, plus TC15 for DM4). AppArmor's unprivileged-userns restriction was disabled and the binaries rebuilt first (the prior build predated the #589 DM2 own-ip feature under test). Key results: - DM2 (native, no VM): TC1/1b/2/5/6/9/10 PASS; TC3/TC4/TC8 FAIL and TC7 with-cert FAIL, all with the in-session backend confirmed live. - DM3 (KVM VM): same PASS set; TC3/TC7 return 502 matching DM1. - DM4: TC15 PASS; G-N1 proxy-port contention confirmed observably (second daemon loses the bind with EADDRINUSE, one listener remains). Findings added to the gap register: - G-N9: the host->PTask ingress forward reaches the backend in the daemon netns, not the session's sandbox netns. Code-confirmed for TC8 (direct-tcpip validates the session UUID then connects to 127.0.0.1:<int> in the daemon netns). Reproduced with a live backend and no VM on DM2, ruling the VM out. - G-N8: on the KVM VM, attach of an own-ip session carrying --ingress hangs (shell never starts, >200s vs ~8s without ingress), so the backend never comes up. DM3's 502 is thus a different mechanism than DM1/DM2 -- TC11 parity holds on the verdict, not the mechanism. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Wjns5t5SQFRvNWmYqXdQ5
Stacked on #581 (
feat/networking-host-exposure). Makes DM2 (nativeLinux, a host-native
minimald, no VM) own-IP work rootless, end to end —the gap #581 calls out ("DM2 … is not separately exercised here") — and adds
explicit
dm1/dm2/dm3bring-up recipes.The problem
On DM2 the daemon runs as an unprivileged host user, not as root in a VM. #581's
own-IP path relied on root-in-VM mechanics, so on a host-native daemon it failed
in three independent ways, each hidden behind the next:
by shelling out to
ip/nsenter.setcap +epis effective-only and is notinherited by child processes, so those children ran without
CAP_NET_ADMIN→EPERM.setcapthen breaks every sandbox. Granting the daemon file capabilitiessets its
dumpableflag toSUID_DUMP_ROOT, which makes/proc/<pid>/uid_maproot-owned. A forked hakoniwa sandbox inherits that andcan no longer write its own
uid_mapas the unprivileged user →EPERM("early eof"), breaking all sessions, own-IP or not.
the synthesized
/etc/resolv.conf(nameserver 127.0.0.53, the hostsystemd-resolved stub) is unreachable, so name resolution fails even once
egress works.
The fix (three coupled changes)
setns(CLONE_NEWNET)+ AF_INET ioctls; no privileged child processes. netns fd pinned before the gvproxy-spawning attach (a short-lived PTask's/proc/<pid>/ns/netvanishes on exit). DM1/3/4 keep the root-in-VM path.minimald/src/net/{switch,gvproxy_network}.rsprctl(PR_SET_DUMPABLE, 1)at startup so asetcap'd daemon can still fork unprivileged-userns sandboxes.minimald/src/main.rsNetwork::nameserver()+Config::dns_nameserver; the sandbox builder writesnameserver <switch-gateway>(gvproxy serves DNS there) into the rootfs before spawn.sandbox2/src/{config,lib,network}.rs,minimald/src/env.rsPlus a
--gvproxy-binflag onminimald runso the own-IP switch can use a localgvproxy build with no system install, and the
dm1/dm2/dm3recipes.DM2 verification (this PR, native Linux)
Reproduce with
just dm2(builds,setcaps, starts with--gvproxy-bin):ip/nsenterEPERM(no inherited caps)setcap'd daemonuid_mapEPERM(dumpable)curlby IP)curl example.com)127.0.0.53dead in netnscargo fmt/clippy -D warningsclean;sandbox2+minimaldlib tests pass; thefull workspace compiles.
Not in scope / not changed
HostShuttle) path and are unchanged; thegateway-DNS change benefits their PTasks too but their data path is still gated
by the separate own-ip attach intermittently hangs forever on AF_VSOCK missed wakeup #588 attach-reliability bug.
.minimal/minimal.tomllocked_commitbump is intentionally not here —this rootless path needs no
ip/nsenterin the guest rootfs.Response to Tom's review (PR #589, comment #4827641490)
No separate next-hop is needed. The default route
(
ip route add default via <gw>) already encodes the next-hop — the gatewayaddress is the next-hop. There is no
next_hopfield anywhere in this codebase(or in PR #589). The
SwitchSubnetstruct only holdsbaseandprefix;gateway()is derived asnetwork + 1, and that same address is used as theviatarget in the default route. A default route withvia <gw>subsumesnext-hop entirely.
🤖 Generated with Claude Code
Summary by CodeRabbit