feat(storage) Add bounded selected-row and field projection for stored V2 sections - #58

Exact kNN projection and winner hydration previously exposed only synchronous
whole-operation APIs. A request deadline could be checked around a segment,
but not while RAW JSON rows, dictionary IDs, or CROSS_DEP exceptions were
being consumed. One large stored section could therefore overrun cancellation
by a complete multi-column decode.

Add controlled projection and hydration entry points with a distinct
`Cancelled` control outcome. Callers receive deterministic before-column,
128-row, and after-column checkpoints; cancellation is neither corruption nor
a compatibility fallback. Preserve the existing APIs as non-cancelling
compatibility wrappers.

Stream RAW typed vector and chunk rows directly from zstd through serde
visitors. Thread checkpoints through hydration JSON visitors, all-ID
dictionary validation, and CROSS_DEP exception scans. Identity columns reuse
the controlled row-codec path. Cancellation before a later corrupt column
drops partial state without observing unvisited corruption.

Adversarial tests place an unsupported or corrupt value at row 129 for pooled
vectors, chunk vectors, and hydration. Cancellation at the real row-128
visitor callback stops before row 129; a non-cancelling run reaches the
expected unsupported outcome or hard error.

Known limits are explicit. LZ decompression remains one indivisible buffer
before cancellable row parsing. Controlled identity decoding currently uses an
8*N-byte ordinal selector to reuse the proven selective codec. This change
therefore improves cancellation granularity but does not claim fully bounded
projection memory.

Validation:

- `cargo test -p xerj-storage`: 108 passed
- doc tests: 1 passed, 1 ignored
- `cargo clippy -p xerj-storage --all-targets -- -D warnings`
- `cargo fmt -p xerj-storage -- --check`
- `git diff --check`

ZBS2 previously exposed only a whole-section compatibility decoder. A caller selecting one row still had to materialize every column as Vec<Value>, rebuild every source object, and serialize the complete canonical JSON array.

Add cancellable selected-row and top-level field projection APIs, preserve identity/sequence metadata and legacy fallback, and harden malformed dictionary, packed-id, cross-dependency, framing, and varint handling. New zstd raw/dictionary/id frames carry content sizes so cache callers can conservatively preflight canonical bytes plus row offsets without decoding. Historical frames without sizes remain readable and fail closed only for admission estimates.

The on-disk version does not change: bulk and streaming compression both emit standard zstd frames consumed by the same decoder. Decoders accept historical unknown-size single frames, reject concatenated/trailing data, bound exact-size packed IDs, and bound cross-dependency expansion from format invariants.

A deterministic 20,000-row fixture encoded to 139,062 bytes and decoded to 86,832,275 bytes. The selected full-row bytes matched the compatibility decode (hash 7945261355329323226), as did the body-only projection (hash 17421003361823546720). Across three fresh-process runs, full decode median peak RSS was 334,942,208 bytes and sampled allocated memory 240,674,768 bytes; one-row hydration used 5,906,432 bytes RSS and 2,728,048 allocated; body-only projection used 5,718,016 bytes RSS and 453,488 allocated.

This is a memory primitive with an explicit CPU trade-off. Median wall time was 0.709 s full versus 2.016 s one-row and 2.050 s projected because RAW_JSON still streams O(rows). It is not yet wired into semantic-search late hydration or the shared CacheResident budget.

Tests cover every codec, sparse/nested sources, ordinals and field projections, full-decode parity, cancellation, malformed lengths/varints/dependencies/cardinality, unknown and concatenated zstd frames, decompression bounds, and conservative V1/V2/plain retained estimates.

Make cancellation checkpoints cover every consumed typed-vector row, including unsupported and non-finite values. Decode constant vector and chunk columns through controlled row iteration rather than allocating repeated rows eagerly.

Reject impossible V2 directory cardinalities before allocation and use fallible reserves for directory metadata and identity ordinals. Add adversarial cancellation and malformed-header coverage plus a reproducible SHA-256-friendly projection probe.

Keep the production zstd encoder unchanged: measured bulk compression was shape-dependent, slightly larger, and materially slower for repeated text. Existing frames without content sizes continue to fail closed for retained-size admission.

Record the exact source commit, environment, commands, immutable fixture and output SHA-256 digests, and all three raw runs for full and selective decode.

Include a three-shape encode_all-versus-bulk zstd comparison. The results support deferring production frame changes until an end-to-end cache-admission consumer exists, while preserving the RAW O(N), LZ4 whole-decompression, and giant-cell caveats.

Complete the num_docs allocation audit by replacing infallible full-row, constant-column, cross-dependency row, and exception allocations with checked reserves. This keeps malformed or resource-exhausting stored headers on the error path instead of an allocator panic path.

Point the reproducibility report at the final allocation-audited implementation commit. The probe and encoded output are unchanged by the fallible-reserve hardening.

The bounded stored-projection history was rewritten onto upstream main 16d6df0, but its reproducibility report still named the pre-rebase implementation and base commits. That made the measurements look detached from the reviewed source even though the probe file, fixture hash, output hash, and behavior were unchanged.

Point the report at rewritten implementation commit 0d28f07453e206516c7390c019c1dec45c7d52be and upstream base 16d6df0. A fresh 20,000-row run reproduced fixture SHA-256 71aef08c71077cec61a2482a3d8089571007f5167e7fa6a0b1d89b6b6f25e76d and output SHA-256 d505b2317c26eec30cf7ff40604d37a5b54986b652bb7348730c5d7ce5d2b35d for all nine full, row, and projected outputs.

Selective V2 row hydration cloned an entire selected constant column with vec![value; len]. Unlike RAW, LZ4, DICT, CROSS_DEP, and typed-vector paths, that eager clone had no 128-row control point and used an infallible capacity allocation. A large constant selection could therefore continue allocating and cloning after its caller requested cancellation.

Build constant outputs incrementally after try_reserve_exact, report allocation failure with the requested count, and run the common Rows checkpoint every 128 emitted values. Cancellation now returns the same distinct StoredDecodeRun::Cancelled outcome as the other selective codecs.

Add a 256-row constant fixture proving cancellation at row 128, non-cancelling parity and existing ordinal normalization, exact constant payloads, and deterministic capacity-overflow handling. Full xerj-storage tests pass 122/122.

Update the bounded stored-projection report to name the final implementation commit after constant-column hydration gained fallible allocation and cooperative cancellation. The committed probe itself and its deterministic fixture/output hashes are unchanged.

Record the expanded storage gate at 122 passed, zero failed, zero ignored.