Skip to content

fix(sandbox2): let the sandbox leader own its namespace - #1004

Closed
norrietaylor wants to merge 1 commit into
mainfrom
fix/sandbox-leader-owns-namespace
Closed

fix(sandbox2): let the sandbox leader own its namespace#1004
norrietaylor wants to merge 1 commit into
mainfrom
fix/sandbox-leader-owns-namespace

Conversation

@norrietaylor

@norrietaylor norrietaylor commented Jul 28, 2026

Copy link
Copy Markdown
Member

Closes #1001.

A sandbox directory is reclaimable once the process it belongs to is gone, but
nothing recorded which process that is.

sandbox2 names each directory {name}-{timestamp}-{attempt}-{pid} from
std::process::id() — whoever created the sandbox, stamped there for
uniqueness. Creator and owner coincide only when the creating process is also
the one whose lifetime the sandbox tracks:

  • Under the CLI they do. mip creates the sandbox, runs it, and exits, so
    its pid dying is a faithful signal that the directory is finished with.
  • Under a daemon they do not. minimald creates sandboxes and outlives them
    all, and inside the microVM it is pid 1 — alive by definition, and still
    pid 1 after a restart.

How this surfaced

Found while building #1002. On
a booted VM every directory under the build base was stamped -1, and the
maintenance cycle declined every single time with
a task is in flight (build-1784239277-0-1). Nothing could distinguish a
running build from an abandoned one, so maintenance never ran and the
directory reap could never have removed anything either. That PR ships without
either half; this is what makes them possible.

Failed builds are the leak that matters here: op/src/specs.rs sets
keep_dir(true) for the duration of a build and clears it only on success, so
an interrupted or failed build deliberately leaves its tree behind.

The change

Ownership is recorded explicitly, in a leader.pid file beside the sandbox
contents. The leader pid is already available — child.id() immediately after
cmd.spawn(), already used for the netns attach.

The marker is written twice, which is the subtle part:

  1. At directory creation, naming the creating process. The window between
    create_dir and spawn does real work (fs mappings, synth dir), and a
    concurrent mip cache clean must not mistake a half-built sandbox for an
    abandoned one. Today that window is safe only by accident, because the name's
    pid happens to be the live creator's.
  2. After spawn, naming the leader — the process whose death actually means the
    directory is finished with. Rewritten per invocation, so a sandbox running
    several execs always names the current one.

The contract lives in common::sandbox_owner rather than sandbox2, so readers
that should not depend on the sandbox implementation can still answer the
question. mip cache clean switches to it, and one rule now works for the CLI
and the daemon alike.

Unknown stays distinct from unowned. A directory with no marker predates it,
or was abandoned before its leader spawned; both must be left alone. That is
what keeps this safe to land on a tree full of pre-existing sandbox directories.

Residual

If the daemon crashes during the setup window, the directory is left naming
pid 1, and after a restart pid 1 is alive again — so it is never reclaimed.
Narrow, and strictly better than today (where every directory has that property
permanently), but not zero. Closing it properly needs something a restart
invalidates, such as a boot id alongside the pid. Left out rather than guessed
at.

Also left out: the directory name keeps its -<pid> suffix, now purely for
uniqueness. Removing it is a bigger behavioural change and nothing reads it for
liveness any more.

Not included

Re-adding the maintenance reap and a build-aware deferral to
#1002. That PR is already open
with both removed; they come back on this signal once both land. Note also that
ListSessions already exposes session liveness and activity timestamps, which
is a better basis for the deferral half than a pid scan — discussed on
#1001.

Verification

  • cross (aarch64 musl): common 52, sandbox2 12, mip 14 — all pass.
    Includes the two /proc-gated tests that cannot run on macOS, so the
    live-owner vs dead-owner distinction is genuinely exercised rather than just
    the parsing.
  • cross clippy -p common -p sandbox2 -p mip --all-targets -- -D warnings: clean.
  • just ci on macOS: green.

Not exercised: a real build writing and handing over its marker on a booted VM.
The write sites are two lines each on paths covered by the existing sandbox
tests, but the handover itself has not been observed end-to-end.

🤖 Generated with Claude Code

Note

Let sandbox2 leader own its namespace via a leader.pid marker file

  • Adds a new sandbox_owner module in crates/common with set_owning_pid, owning_pid, and owner_is_gone helpers that read/write a leader.pid file inside sandbox directories.
  • When a sandbox directory is created, the creating process writes its PID to leader.pid; when a child process is spawned, ownership is handed off to the child's PID.
  • Replaces the previous -<pid> suffix heuristic in mip's cleanup_stale with a call to owner_is_gone, so only directories whose recorded owner has exited are reclaimed.
  • Behavioral Change: sandbox directories without a leader.pid marker or with a live owner are no longer considered stale and will not be cleaned up.

Macroscope summarized 2bb13ad.

A sandbox directory is reclaimable once the process it belongs to is gone,
but nothing recorded which process that is. The directory name carries a
trailing `-<pid>` from `std::process::id()` — whoever *created* the
sandbox, stamped there for uniqueness.

Creator and owner coincide only under the CLI, where `mip` creates the
sandbox, runs it, and exits. Under a daemon they do not: `minimald`
creates sandboxes and outlives them all, and inside the microVM it is
pid 1 — alive by definition, and still pid 1 after a restart. Every
directory it creates looks permanently owned, so nothing can be
reclaimed and nothing can distinguish a running build from an abandoned
one.

Record ownership explicitly instead, in a `leader.pid` file beside the
sandbox contents. It names the creating process while the sandbox is
being set up — a sandbox mid-construction must not read as abandoned,
and the window between `create_dir` and `spawn` does real work — then is
rewritten with the leader's pid as soon as one exists.

The contract lives in `common::sandbox_owner` rather than `sandbox2`, so
readers that should not depend on the sandbox implementation can still
answer the question. `mip cache clean` switches to it; one rule now works
for the CLI and the daemon alike.

Unknown stays distinct from unowned throughout: a directory with no
marker predates it or was abandoned before spawn, and a reaper must
leave both alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: dac31329-b658-4b54-a8ad-16226f6a5122

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@norrietaylor
norrietaylor deleted the fix/sandbox-leader-owns-namespace branch July 29, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sandbox2: the sandbox leader should own its namespace, not the creating process

1 participant