fix(sandbox2): let the sandbox leader own its namespace - #1004
Closed
norrietaylor wants to merge 1 commit into
Closed
fix(sandbox2): let the sandbox leader own its namespace#1004norrietaylor wants to merge 1 commit into
norrietaylor wants to merge 1 commit into
Conversation
A sandbox directory is reclaimable once the process it belongs to is gone, but nothing recorded which process that is. The directory name carries a trailing `-<pid>` from `std::process::id()` — whoever *created* the sandbox, stamped there for uniqueness. Creator and owner coincide only under the CLI, where `mip` creates the sandbox, runs it, and exits. Under a daemon they do not: `minimald` creates sandboxes and outlives them all, and inside the microVM it is pid 1 — alive by definition, and still pid 1 after a restart. Every directory it creates looks permanently owned, so nothing can be reclaimed and nothing can distinguish a running build from an abandoned one. Record ownership explicitly instead, in a `leader.pid` file beside the sandbox contents. It names the creating process while the sandbox is being set up — a sandbox mid-construction must not read as abandoned, and the window between `create_dir` and `spawn` does real work — then is rewritten with the leader's pid as soon as one exists. The contract lives in `common::sandbox_owner` rather than `sandbox2`, so readers that should not depend on the sandbox implementation can still answer the question. `mip cache clean` switches to it; one rule now works for the CLI and the daemon alike. Unknown stays distinct from unowned throughout: a directory with no marker predates it or was abandoned before spawn, and a reaper must leave both alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1001.
A sandbox directory is reclaimable once the process it belongs to is gone, but
nothing recorded which process that is.
sandbox2names each directory{name}-{timestamp}-{attempt}-{pid}fromstd::process::id()— whoever created the sandbox, stamped there foruniqueness. Creator and owner coincide only when the creating process is also
the one whose lifetime the sandbox tracks:
mipcreates the sandbox, runs it, and exits, soits pid dying is a faithful signal that the directory is finished with.
minimaldcreates sandboxes and outlives themall, and inside the microVM it is pid 1 — alive by definition, and still
pid 1 after a restart.
How this surfaced
Found while building #1002. On
a booted VM every directory under the build base was stamped
-1, and themaintenance cycle declined every single time with
a task is in flight (build-1784239277-0-1). Nothing could distinguish arunning build from an abandoned one, so maintenance never ran and the
directory reap could never have removed anything either. That PR ships without
either half; this is what makes them possible.
Failed builds are the leak that matters here:
op/src/specs.rssetskeep_dir(true)for the duration of a build and clears it only on success, soan interrupted or failed build deliberately leaves its tree behind.
The change
Ownership is recorded explicitly, in a
leader.pidfile beside the sandboxcontents. The leader pid is already available —
child.id()immediately aftercmd.spawn(), already used for the netns attach.The marker is written twice, which is the subtle part:
create_dirandspawndoes real work (fs mappings, synth dir), and aconcurrent
mip cache cleanmust not mistake a half-built sandbox for anabandoned one. Today that window is safe only by accident, because the name's
pid happens to be the live creator's.
spawn, naming the leader — the process whose death actually means thedirectory is finished with. Rewritten per invocation, so a sandbox running
several execs always names the current one.
The contract lives in
common::sandbox_ownerrather thansandbox2, so readersthat should not depend on the sandbox implementation can still answer the
question.
mip cache cleanswitches to it, and one rule now works for the CLIand the daemon alike.
Unknown stays distinct from unowned. A directory with no marker predates it,
or was abandoned before its leader spawned; both must be left alone. That is
what keeps this safe to land on a tree full of pre-existing sandbox directories.
Residual
If the daemon crashes during the setup window, the directory is left naming
pid 1, and after a restart pid 1 is alive again — so it is never reclaimed.
Narrow, and strictly better than today (where every directory has that property
permanently), but not zero. Closing it properly needs something a restart
invalidates, such as a boot id alongside the pid. Left out rather than guessed
at.
Also left out: the directory name keeps its
-<pid>suffix, now purely foruniqueness. Removing it is a bigger behavioural change and nothing reads it for
liveness any more.
Not included
Re-adding the maintenance reap and a build-aware deferral to
#1002. That PR is already open
with both removed; they come back on this signal once both land. Note also that
ListSessionsalready exposes session liveness and activity timestamps, whichis a better basis for the deferral half than a pid scan — discussed on
#1001.
Verification
cross(aarch64 musl):common52,sandbox212,mip14 — all pass.Includes the two
/proc-gated tests that cannot run on macOS, so thelive-owner vs dead-owner distinction is genuinely exercised rather than just
the parsing.
cross clippy -p common -p sandbox2 -p mip --all-targets -- -D warnings: clean.just cion macOS: green.Not exercised: a real build writing and handing over its marker on a booted VM.
The write sites are two lines each on paths covered by the existing sandbox
tests, but the handover itself has not been observed end-to-end.
🤖 Generated with Claude Code
Note
Let sandbox2 leader own its namespace via a
leader.pidmarker filesandbox_ownermodule incrates/commonwithset_owning_pid,owning_pid, andowner_is_gonehelpers that read/write aleader.pidfile inside sandbox directories.leader.pid; when a child process is spawned, ownership is handed off to the child's PID.-<pid>suffix heuristic inmip'scleanup_stalewith a call toowner_is_gone, so only directories whose recorded owner has exited are reclaimed.leader.pidmarker or with a live owner are no longer considered stale and will not be cleaned up.Macroscope summarized 2bb13ad.