perf(engine): full-corpus agg path re-decoded segments on every query - #43

Summary

Companion to the memtable-side fix (#42) — same search_inner need_full_corpus block, this time for the on-disk segment side. An unconditional open_segment_arc + decode_stored + full serde_json parse of a segment's entire stored section happened on every search request needing the full corpus for an aggregation, once an index's data moved from memtable to a flushed/merged segment (i.e. any real, sustained ingest — not just the freshly-bulk-imported case the memtable fix covers).

Root cause

Segments are immutable post-flush, so re-decoding the same segment's full stored section per concurrent query is pure waste. Measured 15–26 seconds per request under a concurrent Kibana dashboard burst against a real ~4.7k-doc eCommerce index once its data had flushed to a segment (segment_loop=1300–3600ms, hydrate+corpus=15000–22000ms dominant).

There's already a purpose-built cache for exactly this: stored_value_cache / stored_values_for_async — a single-flight, spawn_blocking-safe cache KNN search already relies on for the identical reason (per its own doc comment: "for a 100-segment index with 100MB stored per segment, every vector query re-paid ~10GB of decompress + parse work"). This code path just never used it, doing its own raw decode instead.

Fix

Rewired the segment loop to call stored_values_for_async — first caller decodes and publishes, every other concurrent caller (single-flight) and every later query (cache hit) just clone the Arc. Also tightened error handling to match the loop's own stated intent ("open failures are errors, not skips") — any segment-read failure during full-corpus assembly is now a hard error instead of a silent skip that would undercount every aggregation bucket.

Test plan

• cargo build --release -p xerj-engine -p xerj-api -p xerj-server
• cargo fmt --check / cargo clippy --no-deps — clean
• cargo test --release -p xerj-engine --lib — 156/156 passed
• Replayed the exact captured slow-query shape (20 concurrent requests, query_string wildcard sub-filter + match_phrase on a .keyword field + date range) against the real flushed segment: went from 15–26s to 1–4ms, including the cold-cache first hit (single-flight coalesces the concurrent misses into one decode)

🤖 Generated with Claude Code