Skip to content

fix(autoindex): --no-graph durable path indexed zero documents for every code file - #316

Merged
xerj-org merged 1 commit into
mainfrom
fix/issue-294-nograph-code-extract
Aug 12, 2026
Merged

fix(autoindex): --no-graph durable path indexed zero documents for every code file#316
xerj-org merged 1 commit into
mainfrom
fix/issue-294-nograph-code-extract

Conversation

@xerj-org

Copy link
Copy Markdown
Owner

Summary

On the durable-generation path (--no-graph), every source-code file was silently dropped: walked, sniffed correctly as family: code, counted, and then prepared as zero documents. The generation still committed and the run reported success (exit 3), so the loss was invisible — the #204 class of accepted-and-ignored input.

Root cause

prepare_artifact sniffs with the logical path (sniff_with_name(snapshot_blob, &source.path)) but extracts with only the blob path. code::extract was the one family extractor that recovered a parameter — the language, via path.extension() — from its content path instead of from Sniffed. Snapshot blobs are content-addressed and extensionless (sync-snapshots/<tx>/blobs/00000000), so the registry lookup never matched and every code file, in every language, became junk.

Fix

Sniffed now carries logical_name, set by sniff_with_name, and code::extract resolves both the grammar and the document title from it, falling back to the content path (which keeps the legacy path byte-identical: there logical == content). The title matters too — without it the fixed durable path would have indexed every code doc titled by blob ordinal 00000000.

Reproduction (run at HEAD, 803c85b)

The new regression test preparation_extracts_code_from_extensionless_snapshot_blobs fails on the parent commit exactly as the issue describes:

assertion `left == right` failed: a code file must prepare one document, not silent junk
  left: (0, 1)      # (records, junk)
 right: (1, 0)

End-to-end with this fix (release binary, throwaway server on :9483), the issue's own repro corpus (app.py + notes.txt) under --no-graph:

generation 1 committed — 1 datasets, 2 records live
{'ax_path': 'app.py',    'ax_format': 'py',  'title': 'app.py', 'language': 'python', 'defs': 'function alpha_helper'}
{'ax_path': 'notes.txt', 'ax_format': 'txt', ...}

and the issue's real-tree case (crates/xerj-autoindex/src, 53 .rs files) goes from 0 records live / count: 0 to 53 records live / count: 53.

Tests

  • sync_executor::tests::preparation_extracts_code_from_extensionless_snapshot_blobs — the durable-path regression the issue asked for (fails before the fix).
  • extract::code::tests::extracts_from_extensionless_blob_via_sniffed_logical_name — the same contract at the extractor's own boundary.

Local gate

  • cargo test -p xerj-autoindex --lib: 530 passed / 0 failed
  • cargo fmt --all --check: clean
  • cargo clippy -p xerj-autoindex --all-targets -- -D warnings: clean
  • ES-YAML conformance suite: 1366 passed · 0 failed · 3 skipped

Noted, out of scope

Other extractors (txt, html, pdf, docx) use their content path only as a fallback title when the document has no intrinsic one, so on the durable path a titleless file can still get a blob-ordinal title. Cosmetic, no document loss — worth a small follow-up that threads logical_name into those fallbacks.

Fixes #294

🤖 Generated with Claude Code

…napshot blob path (#294)

On the durable-generation path (--no-graph) every source-code file was
silently dropped: walked, sniffed correctly as family code, counted, and
then prepared as ZERO documents. The generation still committed and the
run reported success (exit 3), so the loss was invisible - the #204
class of accepted-and-ignored input.

Root cause: prepare_artifact sniffs with the logical path
(sniff_with_name(snapshot_blob, &source.path)) but extracts with only
the blob path. code::extract was the one family extractor that recovered
a parameter - the language, via path.extension() - from its content path
instead of from Sniffed. Snapshot blobs are content-addressed and
extensionless (sync-snapshots/<tx>/blobs/00000000), so the registry
lookup never matched and every code file, in every language, became
junk. Prose/CSV/JSON files were unaffected, which is what made the
symptom look format-specific.

Before: preparing a dir with app.py -> prepared.records == 0,
prepared.junk == 1, empty prepared/*.ndjson. After: 1 record with
language, title app.py, defs 'function alpha_helper'.

Fix: Sniffed now carries logical_name, set by sniff_with_name, and
code::extract resolves both the grammar and the document title from it
(falling back to the content path, which keeps the legacy path
byte-identical: there logical == content). Title matters too - without
it the fixed durable path would have indexed code docs titled by blob
ordinal '00000000'.

Files: crates/xerj-autoindex/src/sniff.rs (logical_name),
extract/code.rs + extract/mod.rs (signature + lookup),
sync_executor.rs + extract/code.rs (regression tests that fail on the
parent commit with (records, junk) == (0, 1)).

Fixes #294
@cla-bot cla-bot Bot added the cla-signed label Aug 11, 2026
@xerj-org
xerj-org merged commit be7bc4e into main Aug 12, 2026
14 checks passed
@xerj-org
xerj-org deleted the fix/issue-294-nograph-code-extract branch August 12, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

autoindex: --no-graph durable path silently indexes ZERO documents for every code file (extensionless snapshot blob defeats grammar lookup)

1 participant