Skip to content

Tags: sricola/offshoot

Tags

v0.2.9

Toggle v0.2.9's commit message
v0.2.9 — supply-chain hardening, hygiene pass 3, Go 1.25

Third hygiene/security pass. The engine surfaces stayed clean (0 critical/
high across the delta); everything shipped here hardens the process layer:

- Every GitHub Action SHA-pinned (13 actions, independently verified against
  their tags), the PyPI publisher moved off a mutable branch ref, per-job
  least-privilege permissions in the release workflow, Dependabot for
  actions + Go modules.
- Ref-derived values no longer interpolate into shell (env-passed), promtool
  checksummed, MinIO images digest-pinned, persist-credentials off on all
  checkouts, docker :latest moves only on real tag pushes, the Sunday
  nightly double-fire fixed.
- LangGraph adapter: a delegation tripwire test (which immediately caught
  and fixed two undelegated SqliteSaver methods), a direct
  langgraph-checkpoint pin, and a public TTL alias.
- Docs: CI-cadence claims corrected (testing.md is now canonical), site
  version chip current + in the release checklist, README front door links
  the trust docs, a four-layer timeout-model index on type S3.
- golang.org/x/sys v0.44.0 (clears GO-2026-5024 scanner noise; Windows-only,
  never called) — this moves the Go floor to 1.25, reflected in the
  Dockerfile and docs.

See CHANGELOG.md [0.2.9].

v0.2.8

Toggle v0.2.8's commit message
v0.2.8 — the pre-launch stack

Everything from the product gap analysis that could ship before launch:

- Measured copy-on-write numbers, published: 377 bytes per shared fork of a
  100 MB database (~280,000x less than a copy), ~9 ms checkpoint-fork latency
  flat from 12 MB to 1 GB, divergence ~O(changed pages). make bench-cow
  reproduces the table; caveats stated, including the at-head fork's O(size)
  fingerprint check.
- langgraph-checkpoint-offshoot: a LangGraph checkpointer over offshoot
  checkouts — stock SqliteSaver serialization plus fork_thread/rollback/
  promote/destroy with TTLs; branch protection respected by default.
- Packaging: in-repo Homebrew formula, Nix flake, sqldiff in the Docker
  image, install matrix + WSL2 note, release-checklist packaging bumps.
- Trust docs: the pre-1.0 stability contract (same-release migration-or-
  export promise), 'how offshoot is tested' with fresh torture numbers,
  a copy-pasteable CI cookbook, the merge FAQ, a ready-to-import Grafana
  dashboard, and a verified CLI/daemon/SDK parity table.

See CHANGELOG.md [0.2.8].

v0.2.7

Toggle v0.2.7's commit message
v0.2.7 — every S3 call bounded; streaming reads get a progress watchdog

Closes the last tracked availability finding: single-shot Get/Put/PutIf/
List/Delete/DeleteObjects/CopyObject now carry per-call deadlines, sized
by payload where the size is known (base 15 min + bytes at the same
1 MiB/s pessimistic floor the multipart deadlines use), so a stalled
backend can no longer wedge the flush path through any S3 call — and a
legitimate large transfer on a slow link is never killed.

GetReader's streaming body gets a progress watchdog: a Read that blocks
longer than 60s with zero bytes cancels the request; a slow CONSUMER is
structurally immune (the timer arms only while a Read is blocked).

Also in this release: the real-provider suite (conformance + multipart)
verified green against MinIO, and the landing site redesigned — minimal,
engineer-first, command table verified against the CLI.

See CHANGELOG.md [0.2.7].

v0.2.6

Toggle v0.2.6's commit message
v0.2.6 — audit pass 2: bounded S3 waits, create-only preflight, multi…

…part cleanup

Second hygiene/security audit pass, covering the v0.2.3-v0.2.5 delta.
Security result: 0 critical/high/medium findings; the one Low is fixed here.

- Every S3 call now bounds its wait for a response to BEGIN (60s
  ResponseHeaderTimeout; never bounds body transfer), and every multipart
  RPC carries a per-call deadline — a stalled backend response can no
  longer wedge the flush path via multipart, and abort/cleanup is bounded
  too.
- A create-only multipart upload preflights with HeadObject and fails fast
  with ErrCAS when the key already exists, instead of transferring >5 GiB
  before Complete rejects it. Complete's condition remains authoritative.
- Multipart machinery split into s3_multipart.go; duplicated abort/checksum
  /clamp fragments factored into helpers; checksum adaptors field-keyed.
- The out-of-order-completion test is now deterministic (gate machinery,
  no sleeps) — it previously could pass without exercising reconstruction.
- Doc fixes: a stale 'multipart copy unimplemented' claim, CHANGELOG 0.2.4
  corrections, test-hook headers.

See CHANGELOG.md [0.2.6].

v0.2.5

Toggle v0.2.5's commit message
v0.2.5 — fix silent data loss when a fenced writer's snapshot shadowe…

…d a live segment

A correctness fix in chain resolution. Read it if you run offshoot against a
store where a flush has ever failed ambiguously.

THE BUG: members were deduplicated on their {MinTXID, MaxTXID} key, and
snapshots and segments were deduplicated in separate lists. A fenced writer's
snapshot at txid T keys as {Min=0, Max=T}; a live writer's segment at T keys
as {Min=T, Max=T}. Different keys in different lists, so they never collided
and the superseded snapshot survived. Resolution anchors on the newest
snapshot at or below the target, so it picked the fenced writer's object and
returned it alone.

Two ways that lost data:
- Reads silently served the fenced writer's state, dropping the live
  writer's committed transaction. No error, no checksum failure — the
  snapshot is internally valid, it is simply the wrong object.
- GC's mark shares the same resolver, so it also resolved to the fenced
  snapshot, left the real snapshot and the live segment unmarked, and
  swept them after the grace period — destroying the live branch.

THE FIX: deduplicate snapshots and single-txid segments together, keyed on
MaxTXID, highest epoch wins. Multi-txid segments (which no writer in this
codebase emits) keep the original range key, so an object at {Min=0, Max=T}
can never evict a real snapshot at T. This is the minimal widening of the
old rule that closes the hole: for every store without a same-txid epoch
conflict, resolution is byte-identical to before.

Verified by three tests that each fail against the pre-fix code, including
an end-to-end one that drives the real shape — an ambiguous ref-write
failure leaving a snapshot orphan, then a second session writing a segment
at the same txid — and asserts the second session's row is readable.

See CHANGELOG.md [0.2.5].

v0.2.4

Toggle v0.2.4's commit message
v0.2.4 — concurrent multipart uploads, server-side copy over 5 GiB, e…

…poch-aware GC

- S3 multipart parts now upload concurrently on the io.ReaderAt path (the
  production snapshot path). The non-ReaderAt fallback stays sequential by
  design: it shares one buffer and reads its source in order.
- CopyObject handles sources over 5 GiB via UploadPartCopy instead of
  returning ErrCopyUnsupported, so a large fork keeps the server-side fast
  path rather than falling back to materialize-and-re-encode.
- GC's compensating rule is epoch-aware: an above-head orphan written by a
  writer that has since been fenced (epoch strictly older than the lineage's
  current writer generation) can never win a ref CAS, so it is now reclaimed
  instead of protected for the life of the branch. Objects at or above the
  current epoch stay protected, and a stale read errs toward protecting.
- Docs: the fork-floor bound is a per-process knob; an at-rest CLI fork uses
  the default (16) even against a daemon running -snapshot-every N.

See CHANGELOG.md [0.2.4].

v0.2.3

Toggle v0.2.3's commit message
v0.2.3 — S3 multipart uploads; capability-wrapper doc note

Lifts the 5 GiB single-PUT ceiling: a snapshot of a database larger than
5 GiB previously failed to upload to S3. PutReader/PutReaderIf now switch to
a real multipart upload above a threshold, with part sizing that respects
S3's >=5 MiB / <=10,000-part limits up to the 5 TiB object limit.

- CAS conditions ride on CompleteMultipartUpload; a precondition rejection
  maps to ErrCAS exactly as the single-PUT path does.
- AbortMultipartUpload runs on every error exit after Create (an abandoned
  upload bills for its parts indefinitely); only a successful Complete skips it.
- Parts stream from the caller's file via SectionReader (zero buffering),
  honoring the reader's current position.
- CRC32 is declared on both CreateMultipartUpload and every UploadPart, and
  each part's checksum is carried into CompletedPart, so declared == supplied
  under any RequestChecksumCalculation setting.
- Below the threshold the original single-PutObject path is byte-unchanged.

NOTE: the multipart path is verified by SDK-semantics review and an in-process
fake; the real-provider test (TestS3RealProvider/Multipart) ships UNRUN — it
requires OFFSHOOT_S3_TEST_BUCKET.

Also: the optional Backend capability interfaces now document that a wrapper
embedding store.Backend silently hides them (correct but slower fallback).

See CHANGELOG.md [0.2.3].

v0.2.2

Toggle v0.2.2's commit message
v0.2.2 — audit pass: hygiene, performance, security

Performance:
- Streaming materialize + snapshot-flush upload: peak RAM O(one object), not O(whole chain).
- Snapshot flush releases replicaMu before the encode (no capture stalls).
- Capture resume-offset fsync per-burst, not per-transaction.
- GC batch-delete via S3 DeleteObjects (1000 keys/request).
- store.Chain Lists a shared-fork prefix once; Fork reuses its resolved chain; GC + layout-check RPCs trimmed.

Security:
- export rejected over HTTP /rpc (its arbitrary-path trust model is unix-socket-only).
- Daemon socket created 0600 from the first instant (umask).

Hygiene:
- CI now hard-fails on a missing sqlite3/sqldiff (was a silent skip); gofmt/vet lint gates.
- Refreshed stale docs, logged best-effort deletes, removed dead code, named magic numbers, split two large files.

Three new optional Backend capabilities: BatchDeleter, ReaderGetter, ReaderPutter. No storage-format change.
See CHANGELOG.md [0.2.2].

v0.2.1

Toggle v0.2.1's commit message
v0.2.1 — copy-on-write polish

Hardening and observability for the 0.2.0 copy-on-write feature.

- store.Chain: base-pointer cycle guard (crash-hardening for a corrupt store).
- Daemon compact refuses while a session is open (matches rollback/promote).
- Fork-time floor tracks the configured -snapshot-every instead of hardcoded 16.
- New metrics: offshoot_gc_errors_total, offshoot_fork_mode_total{mode}.
- Doc fixes (compact cost class, release line, spec reconciliation).

See CHANGELOG.md [0.2.1].

v0.2.0

Toggle v0.2.0's commit message
v0.2.0 — copy-on-write object-sharing forks

Forks stop being full N×G copies: a fork records a base pointer into the
parent's durable chain and writes new objects only as it diverges.

Storage-format change: LayoutVersion 1 -> 2. Once any base-pointer fork
exists, pre-CoW (v1) binaries refuse the store via CheckManifest.

Highlights:
- Base-following chain resolution (strict never-merge-across-lineages).
- Two automatic snapshot floors keep reads bounded across deep fork chains.
- GC rewritten to object-granular transitive-base-closure reachability.
- New: offshoot compact <db>[@Branch] cuts the cord (materialize + drop base).
- status/branches report shared vs materialized storage class.

See CHANGELOG.md [0.2.0].