Tags: sricola/offshoot
Tags
v0.2.9 — supply-chain hardening, hygiene pass 3, Go 1.25 Third hygiene/security pass. The engine surfaces stayed clean (0 critical/ high across the delta); everything shipped here hardens the process layer: - Every GitHub Action SHA-pinned (13 actions, independently verified against their tags), the PyPI publisher moved off a mutable branch ref, per-job least-privilege permissions in the release workflow, Dependabot for actions + Go modules. - Ref-derived values no longer interpolate into shell (env-passed), promtool checksummed, MinIO images digest-pinned, persist-credentials off on all checkouts, docker :latest moves only on real tag pushes, the Sunday nightly double-fire fixed. - LangGraph adapter: a delegation tripwire test (which immediately caught and fixed two undelegated SqliteSaver methods), a direct langgraph-checkpoint pin, and a public TTL alias. - Docs: CI-cadence claims corrected (testing.md is now canonical), site version chip current + in the release checklist, README front door links the trust docs, a four-layer timeout-model index on type S3. - golang.org/x/sys v0.44.0 (clears GO-2026-5024 scanner noise; Windows-only, never called) — this moves the Go floor to 1.25, reflected in the Dockerfile and docs. See CHANGELOG.md [0.2.9].
v0.2.8 — the pre-launch stack Everything from the product gap analysis that could ship before launch: - Measured copy-on-write numbers, published: 377 bytes per shared fork of a 100 MB database (~280,000x less than a copy), ~9 ms checkpoint-fork latency flat from 12 MB to 1 GB, divergence ~O(changed pages). make bench-cow reproduces the table; caveats stated, including the at-head fork's O(size) fingerprint check. - langgraph-checkpoint-offshoot: a LangGraph checkpointer over offshoot checkouts — stock SqliteSaver serialization plus fork_thread/rollback/ promote/destroy with TTLs; branch protection respected by default. - Packaging: in-repo Homebrew formula, Nix flake, sqldiff in the Docker image, install matrix + WSL2 note, release-checklist packaging bumps. - Trust docs: the pre-1.0 stability contract (same-release migration-or- export promise), 'how offshoot is tested' with fresh torture numbers, a copy-pasteable CI cookbook, the merge FAQ, a ready-to-import Grafana dashboard, and a verified CLI/daemon/SDK parity table. See CHANGELOG.md [0.2.8].
v0.2.7 — every S3 call bounded; streaming reads get a progress watchdog Closes the last tracked availability finding: single-shot Get/Put/PutIf/ List/Delete/DeleteObjects/CopyObject now carry per-call deadlines, sized by payload where the size is known (base 15 min + bytes at the same 1 MiB/s pessimistic floor the multipart deadlines use), so a stalled backend can no longer wedge the flush path through any S3 call — and a legitimate large transfer on a slow link is never killed. GetReader's streaming body gets a progress watchdog: a Read that blocks longer than 60s with zero bytes cancels the request; a slow CONSUMER is structurally immune (the timer arms only while a Read is blocked). Also in this release: the real-provider suite (conformance + multipart) verified green against MinIO, and the landing site redesigned — minimal, engineer-first, command table verified against the CLI. See CHANGELOG.md [0.2.7].
v0.2.6 — audit pass 2: bounded S3 waits, create-only preflight, multi… …part cleanup Second hygiene/security audit pass, covering the v0.2.3-v0.2.5 delta. Security result: 0 critical/high/medium findings; the one Low is fixed here. - Every S3 call now bounds its wait for a response to BEGIN (60s ResponseHeaderTimeout; never bounds body transfer), and every multipart RPC carries a per-call deadline — a stalled backend response can no longer wedge the flush path via multipart, and abort/cleanup is bounded too. - A create-only multipart upload preflights with HeadObject and fails fast with ErrCAS when the key already exists, instead of transferring >5 GiB before Complete rejects it. Complete's condition remains authoritative. - Multipart machinery split into s3_multipart.go; duplicated abort/checksum /clamp fragments factored into helpers; checksum adaptors field-keyed. - The out-of-order-completion test is now deterministic (gate machinery, no sleeps) — it previously could pass without exercising reconstruction. - Doc fixes: a stale 'multipart copy unimplemented' claim, CHANGELOG 0.2.4 corrections, test-hook headers. See CHANGELOG.md [0.2.6].
v0.2.5 — fix silent data loss when a fenced writer's snapshot shadowe…
…d a live segment
A correctness fix in chain resolution. Read it if you run offshoot against a
store where a flush has ever failed ambiguously.
THE BUG: members were deduplicated on their {MinTXID, MaxTXID} key, and
snapshots and segments were deduplicated in separate lists. A fenced writer's
snapshot at txid T keys as {Min=0, Max=T}; a live writer's segment at T keys
as {Min=T, Max=T}. Different keys in different lists, so they never collided
and the superseded snapshot survived. Resolution anchors on the newest
snapshot at or below the target, so it picked the fenced writer's object and
returned it alone.
Two ways that lost data:
- Reads silently served the fenced writer's state, dropping the live
writer's committed transaction. No error, no checksum failure — the
snapshot is internally valid, it is simply the wrong object.
- GC's mark shares the same resolver, so it also resolved to the fenced
snapshot, left the real snapshot and the live segment unmarked, and
swept them after the grace period — destroying the live branch.
THE FIX: deduplicate snapshots and single-txid segments together, keyed on
MaxTXID, highest epoch wins. Multi-txid segments (which no writer in this
codebase emits) keep the original range key, so an object at {Min=0, Max=T}
can never evict a real snapshot at T. This is the minimal widening of the
old rule that closes the hole: for every store without a same-txid epoch
conflict, resolution is byte-identical to before.
Verified by three tests that each fail against the pre-fix code, including
an end-to-end one that drives the real shape — an ambiguous ref-write
failure leaving a snapshot orphan, then a second session writing a segment
at the same txid — and asserts the second session's row is readable.
See CHANGELOG.md [0.2.5].
v0.2.4 — concurrent multipart uploads, server-side copy over 5 GiB, e… …poch-aware GC - S3 multipart parts now upload concurrently on the io.ReaderAt path (the production snapshot path). The non-ReaderAt fallback stays sequential by design: it shares one buffer and reads its source in order. - CopyObject handles sources over 5 GiB via UploadPartCopy instead of returning ErrCopyUnsupported, so a large fork keeps the server-side fast path rather than falling back to materialize-and-re-encode. - GC's compensating rule is epoch-aware: an above-head orphan written by a writer that has since been fenced (epoch strictly older than the lineage's current writer generation) can never win a ref CAS, so it is now reclaimed instead of protected for the life of the branch. Objects at or above the current epoch stay protected, and a stale read errs toward protecting. - Docs: the fork-floor bound is a per-process knob; an at-rest CLI fork uses the default (16) even against a daemon running -snapshot-every N. See CHANGELOG.md [0.2.4].
v0.2.3 — S3 multipart uploads; capability-wrapper doc note Lifts the 5 GiB single-PUT ceiling: a snapshot of a database larger than 5 GiB previously failed to upload to S3. PutReader/PutReaderIf now switch to a real multipart upload above a threshold, with part sizing that respects S3's >=5 MiB / <=10,000-part limits up to the 5 TiB object limit. - CAS conditions ride on CompleteMultipartUpload; a precondition rejection maps to ErrCAS exactly as the single-PUT path does. - AbortMultipartUpload runs on every error exit after Create (an abandoned upload bills for its parts indefinitely); only a successful Complete skips it. - Parts stream from the caller's file via SectionReader (zero buffering), honoring the reader's current position. - CRC32 is declared on both CreateMultipartUpload and every UploadPart, and each part's checksum is carried into CompletedPart, so declared == supplied under any RequestChecksumCalculation setting. - Below the threshold the original single-PutObject path is byte-unchanged. NOTE: the multipart path is verified by SDK-semantics review and an in-process fake; the real-provider test (TestS3RealProvider/Multipart) ships UNRUN — it requires OFFSHOOT_S3_TEST_BUCKET. Also: the optional Backend capability interfaces now document that a wrapper embedding store.Backend silently hides them (correct but slower fallback). See CHANGELOG.md [0.2.3].
v0.2.2 — audit pass: hygiene, performance, security Performance: - Streaming materialize + snapshot-flush upload: peak RAM O(one object), not O(whole chain). - Snapshot flush releases replicaMu before the encode (no capture stalls). - Capture resume-offset fsync per-burst, not per-transaction. - GC batch-delete via S3 DeleteObjects (1000 keys/request). - store.Chain Lists a shared-fork prefix once; Fork reuses its resolved chain; GC + layout-check RPCs trimmed. Security: - export rejected over HTTP /rpc (its arbitrary-path trust model is unix-socket-only). - Daemon socket created 0600 from the first instant (umask). Hygiene: - CI now hard-fails on a missing sqlite3/sqldiff (was a silent skip); gofmt/vet lint gates. - Refreshed stale docs, logged best-effort deletes, removed dead code, named magic numbers, split two large files. Three new optional Backend capabilities: BatchDeleter, ReaderGetter, ReaderPutter. No storage-format change. See CHANGELOG.md [0.2.2].
v0.2.1 — copy-on-write polish
Hardening and observability for the 0.2.0 copy-on-write feature.
- store.Chain: base-pointer cycle guard (crash-hardening for a corrupt store).
- Daemon compact refuses while a session is open (matches rollback/promote).
- Fork-time floor tracks the configured -snapshot-every instead of hardcoded 16.
- New metrics: offshoot_gc_errors_total, offshoot_fork_mode_total{mode}.
- Doc fixes (compact cost class, release line, spec reconciliation).
See CHANGELOG.md [0.2.1].
v0.2.0 — copy-on-write object-sharing forks Forks stop being full N×G copies: a fork records a base pointer into the parent's durable chain and writes new objects only as it diverges. Storage-format change: LayoutVersion 1 -> 2. Once any base-pointer fork exists, pre-CoW (v1) binaries refuse the store via CheckManifest. Highlights: - Base-following chain resolution (strict never-merge-across-lineages). - Two automatic snapshot floors keep reads bounded across deep fork chains. - GC rewritten to object-granular transitive-base-closure reachability. - New: offshoot compact <db>[@Branch] cuts the cord (materialize + drop base). - status/branches report shared vs materialized storage class. See CHANGELOG.md [0.2.0].
PreviousNext