perf(cache): probe-based cache admission and an off-request board warmer - #83

Problem

Opening aeman after any break longer than ten minutes hung on a 20-30s progress bar. Measured on the production-size board (~1400 cards): a full load is ~15 sequential GraphQL pages at ~2s each — sequential because GitHub's cursor pagination cannot be parallelized — so ~28s wall-clock. Two designs made that load far more frequent than it had to be:

• The shared board cache is (correctly) gated per login, but the only way a login could prove access was a full token-scoped board load. Every engineer coming back from a break personally re-paid ~28s to be admitted into a cache their teammates kept warm.
• Overnight, with every laptop asleep, no watch connection survived to keep the cache alive — the first open of the morning always started cold. During the day, the opposite: every open tab kicked a stale revalidation from its 30s ping ticker, a near-continuous full reload.

Change

Probe-based admission. Proving that a token can read the board costs one project-id GraphQL query (~0.5s), not a full load. A login reaching a warm cache is admitted by that probe; an aging proof refreshes the same way in the background while the read is served fresh. The full load remains the authorization path wherever the probe is unavailable or fails, and a probe that positively reports access gone drops the proof. A parity test runs identical GitHub fixtures (not-found, partial errors, rate-limit, 401/403, insufficient scopes, 502) through both the probe and the full load and pins their verdicts together — the probe is exactly as strict as the load it stands in for.

Off-request warmer. One server-side loop per board refreshes the cache on the cadence of the fresh window while the board has watchers, and on a slower idle cadence for 16h past the last read — carrying the cache through the night, so the morning's first open is served instantly. It replaces the per-tab ping revalidation entirely. The warmer's identity rotates to the most recent successful loader and is bound to that user's web session: it stops on logout or session expiry, never runs on MCP-request tokens (nothing to bind their lifetime to), tolerates transient upstream failures by skipping a tick, and stops with a log line on persistent failure or access loss. Warm refreshes never vouch for anyone's access.

Mutations benefit directly: with the warmer keeping the cache inside the fresh window, reads and mutations land on the cached board instead of blocking on a reload.

Review

The change went through three independent adversarial reviews (concurrency/lifecycle, cache-authorization security, perf/operations). The security review verified empirically that the probe admits no one the full load would reject, and found no bypass of the per-login cache gate. All blockers and majors they raised are addressed in the second commit; the notable ones: mutations re-paying full loads (fresh window now matches the warmer cadence), the warmer dying overnight on a single 502 (bounded retries), a captured OAuth token outliving its owner's session (session-bound, rotating source), and an unauthorized caller being able to extend the warm window (reads stamp it only when actually served).

Numbers

On the ~1400-card board: first open after a break drops from ~28s to milliseconds (probe ~0.5s when a re-proof is needed); the only remaining cold load is the very first request after a server restart. Steady-state GitHub traffic drops from roughly one full reload per active tab per minute to one per board per 3 minutes (8 minutes when nobody is watching).

Known trade-offs

• External GitHub edits (made outside aeman) surface within one warmer tick (~3 min worst case, was ~30-60s via per-tab revalidations) for passive viewers; active users' own reads still trigger revalidation as before.
• A user whose token can read the project but not some linked private issues gets the shared cache's fuller view (titles instead of "(untitled)" placeholders). Not new — the pre-probe flow had the same property one full load later — but worth stating: the gate proves "can read this board", not "sees exactly what their own token would fetch".
• The post-deploy cold start is not addressed here; a follow-up could warm the pinned board at boot from a persisted session token.

Testing

• TestBoardCacheProbeAdmitsSecondUser, TestBoardCacheProbeDeniedFallsThrough — probe admits exactly one cheap check, denial falls through to the caller's own load.
• TestCachedAuthz — the full state matrix including the new re-proof flag.
• TestBoardCacheWarmerFollowsWatchers — refreshes on its own, outlives watchers within the idle window, stops when both conditions lapse.
• TestCheckBoardAccessMatchesFullLoad — 10-fixture probe/load parity table.
• Shared test fakes made goroutine-safe (the async title resolve and the warmer hit them from background goroutines); everything runs and passes under -race.
• Verified live against the production-size board: cold prime once, then instant reads across warmer ticks.