v0.5: diff snapshot chains (M2.1 tracking) #216

Tracking issue for v0.5 — diff snapshot chains. Goal: stop duplicating base bytes per pip install delta. python-numpy → +pandas → +sklearn stores deltas on disk; controller walks the chain at restore time.

Design: DESIGN-v0.5-diff-snapshot-chains.md

Done

• Phase 1 — chain library (crates/forkd-vmm/src/chain.rs): resolve_chain, verify_parent_hashes, assemble_chain_memory. Schema gains parent_tag + parent_content_hash. Content-hash pinning ships in v0.5 (not deferred). 7 unit tests + 1 bonus + 2 round-trip. PR feat(vmm): v0.5 Phase 1 — diff snapshot chain resolver + schema #213.
• Phase 2a — chain-aware spawn. POST /v1/sandboxes resolves chain → verifies parent hashes → assembles memory via cp(base) + apply_diff(each link) into per-spawn scratch → swaps snapshot.memory to the assembled path. Non-chained snapshots take the historical fast path bit-for-bit. live_fork: true on chained source returns HTTP 400 (v0.5 carve-out). PR feat(controller): v0.5 Phase 2a — chain-aware spawn #214.
• Phase 2b — forkd snapshot-diff --from <base> --tag <new> --exec \"<cmd>\" build verb. BranchSandboxRequest.parent_tag opt-in on the existing BRANCH endpoint. CLI orchestrates spawn → wait → exec → BRANCH(diff, parent_tag) → kill. PR feat(cli,controller): v0.5 Phase 2b — forkd snapshot-diff build verb #215.

Remaining

• Phase 3 — Hub pack / pull walk chains. registry.json schema gains parent_tag per recipe entry. Each link's hash verified independently. ~4 days, independent of Phase 4.
• Phase 4 — forkd snapshot info / forkd snapshot compact / forkd rmi-with-dependents (409 + dependent list). ~2 days, independent of Phase 3.
• Phase 5 — bench on a real pip install pandas chain. Closes the vmstate-drift risk empirically. bench/diff-snapshot-chains/RESULTS-v0.5.md mirroring v0.4's bench shape. ~3 days. This is the most important next phase — it proves the design.
• Phase 6 — docs sweep: README + docs/HUB.md + docs/API.md + CHANGELOG. ~1 day, last.

v0.5 done-criteria (from ROADMAP M2.1)

1. forkd snapshot-diff --from <base> --tag <new> --exec <cmd> produces a diff < 100 MB for an apt install / pip install delta.
2. Restore time on a 3-snapshot chain is within 10% of base restore on reflink-capable FS (btrfs / xfs / ext4-with-reflink) — within 2× on non-reflink FS with a forkd doctor warning.
3. Snapshot Hub MVP understands chains: pulling a diff also pulls its parents.

Risks

• vmstate-drift on install-time states: largest unknown. v0.3.1's BRANCH chain restores vmstate against assembled memory in production (SandboxInfo.last_branch_memory_path pattern); v0.5 reuses the same shape at build time. Phase 5 empirical test on pip install pandas closes this — if it passes, the risk is gone. Budget +1 week to Phase 2 if any installer (pip / apt / npm) fails the smoke test.
• Disk usage on Hub-pull of deep chain: a 5-level chain forces consumers to download base + 4 diffs. Documented honestly — solved properly in v0.6 CAS-layered Hub, not here.

Help wanted

If you want to contribute to any of phases 3-6 — drop a comment, I'll point you at the right entry point.

Related: #101 closed (v0.4 fully shipped). v0.5 builds on v0.4's apply_diff + chain semantics primitives.