Why
A cold pixel build-index --history . on a worktree of this repo took 253 s. Reproduced on a fresh clone of the same repo: 140.7 s, with 275 of 287 work ticks spent in phase B (blob measuring). Phase A (metadata) and phase C (diff text) are cheap; phase B is the bottleneck.
Evidence (measured 2026-10-06)
| Phase |
Evidence |
Cost |
| Text index (1,028 files) |
printed early, base_files=1028 |
negligible |
| Phase A — commit metadata |
git log batches of 200 |
~24 ms/batch |
| Phase B — blob measuring |
275/287 ticks |
~70–80 s of 140 s |
| Phase C — diff text (365-day window) |
10 ticks |
~15–20 s |
Root cause: measure_commit_blobs (crates/pixel-facts/src/ingest.rs:803) measures one git ls-tree -l process per changed path (blob_size, ingest.rs:847), plus a second spawn per rename against the parent tree. The repo has ~9,170 changed-path rows across 1,024 commits → 10k+ git spawns at ~7 ms each. The comment above the function says cat-file --batch-check, but the implementation is per-path ls-tree.
Amplifier: ingest scope is all refs + stash + every reflog-only commit (rev-list --branches --remotes --tags --reflog, ingest.rs:487,512). A worktree's reflog enumeration picked up 2,382 commits vs 853 on HEAD (commits=2224 in the log), ~1.9× the work.
Ask
- Batch phase B: replace per-path
ls-tree spawns with one batched cat-file --batch-check (or one ls-tree per commit listing all paths). Expected: phase B drops from ~70 s to seconds; cold --history build under a minute on this repo.
- Per-phase timing logs: report/log phases A, B and C separately (tick reports already carry
phase; surface a per-phase duration breakdown in the build-index summary so regressions are visible without PIXEL_FACTS_DEBUG_TICKS=1).
- Scope control: don't index much more than needed — reflog-only commits inflate the work ~1.9× on worktrees and churn-heavy repos. Study what actually pays off in
excavate/rescue/history queries and consider bounding reach (e.g. skip reflog-only by default, or an opt-in/out flag), keeping the age window + size budget limits.
Notes
PIXEL_FACTS_DEBUG_TICKS=1 exists but only prints raw tick reports.
- Reproduction: wipe
history.db, run pixel build-index --history ., watch ticks 2–277 sit in phase_b.
Why
A cold
pixel build-index --history .on a worktree of this repo took 253 s. Reproduced on a fresh clone of the same repo: 140.7 s, with 275 of 287 work ticks spent in phase B (blob measuring). Phase A (metadata) and phase C (diff text) are cheap; phase B is the bottleneck.Evidence (measured 2026-10-06)
base_files=1028git logbatches of 200Root cause:
measure_commit_blobs(crates/pixel-facts/src/ingest.rs:803) measures onegit ls-tree -lprocess per changed path (blob_size,ingest.rs:847), plus a second spawn per rename against the parent tree. The repo has ~9,170 changed-path rows across 1,024 commits → 10k+ git spawns at ~7 ms each. The comment above the function sayscat-file --batch-check, but the implementation is per-pathls-tree.Amplifier: ingest scope is all refs + stash + every reflog-only commit (
rev-list --branches --remotes --tags --reflog,ingest.rs:487,512). A worktree's reflog enumeration picked up 2,382 commits vs 853 on HEAD (commits=2224in the log), ~1.9× the work.Ask
ls-treespawns with one batchedcat-file --batch-check(or onels-treeper commit listing all paths). Expected: phase B drops from ~70 s to seconds; cold--historybuild under a minute on this repo.phase; surface a per-phase duration breakdown in thebuild-indexsummary so regressions are visible withoutPIXEL_FACTS_DEBUG_TICKS=1).excavate/rescue/historyqueries and consider bounding reach (e.g. skip reflog-only by default, or an opt-in/out flag), keeping the age window + size budget limits.Notes
PIXEL_FACTS_DEBUG_TICKS=1exists but only prints raw tick reports.history.db, runpixel build-index --history ., watch ticks 2–277 sit inphase_b.