/leaderboard has no result cache and no request coalescing. Every concurrent viewer pays a full recompute of the entire owner population to see the same 25 rows.

Measured on the dev dataset (17.3M events), one board query, varying concurrency:

Concurrent viewers	avg latency	peak mem/query	summed mem
1	40 ms	143 MiB	143 MiB
8	124 ms	160 MiB	1.09 GiB
32	484 ms	171 MiB	5.12 GiB

Latency and memory scale linearly with concurrency. This is the Discord tier-up announce pattern: everyone arrives at once, and the cluster does N identical aggregations.

Two independent reasons nothing is cached today:

ChReadClient.query calls chReadSettings(SCORE_READ_CEILING_S) with no cacheTtlS. Only /metrics opts into ClickHouse's server-side query cache; every score read skips it.
initialFill mints asOf as parseAsOf(url) ?? new Date() — a fresh millisecond per request. Even with the cache enabled, the key would never repeat. This is precisely the unquantized-bound trap already documented in read-protocol.ts.

Proposal

Three pieces:

Quantize asOf to a bucket instead of new Date(), so the cache key repeats across viewers.
Add a monotonic {epoch:UInt64} bound param to the board reads, carried as an inert predicate. This is the bust lever.
Pass cacheTtlS through chReadSettings on the board read path.

The freshness contract

Caching is only acceptable if significant events bust it. Verified on ClickHouse 24.10 — a bound param does change the query-cache key:

sequence	result
`epoch=1`	miss, 3.73M rows
`epoch=1`	hit, 5 rows
`epoch=2`	miss, 3.73M rows
`epoch=2`	hit, 5 rows
`epoch=1` again	hit (old entry still live)

Note the last row: epochs must be monotonic. Reusing an old value re-hits its stale entry, so a counter or timestamp, never a toggle.

Wiring: TierCrossingWatchDeps.onTierReached already exists as an injected hook, and the tier watcher already performs an atomic Mongo compare-and-advance in claimCrossing. Bumping an epoch doc in that same write is nearly free. The board API reads it behind a ~1s per-pod cache — the same shape as the existing enterpriseHiddenUsernames cache in repos.ts, so no new pattern.

Result: cached hard between crossings, provably fresh at them. Eager-tick-plus-announcer-poke, not a lazy TTL.

Expected effect

N concurrent viewers collapse to 1 query per replica (3 across the cluster). Measured on the 10B synthetic with a snapshot table behind it: 64-way concurrency served in 27 ms total wall, 0.6 ms average.

Even without phase 3's snapshot, this alone removes the concurrency multiplier at current scale.

Constraints and traps

The asOf echo-back contract must survive quantization. The fill returns the pin and every later page fetch echoes it rather than re-pinning, or rows and the pagination that follows them come from different instants (#1055).
The query cache is per-node on a 3-replica cluster, so expect 3 misses per epoch, not 1. That is fine and worth stating so it isn't read as a bug.
Confirm query_cache_min_query_runs and query_cache_min_query_duration defaults on the prod nodes are permissive enough to admit these queries, and that the cache size ceiling accommodates the entry.
Errors and ceiling hits are not cached by ClickHouse, so a degraded read cannot get stuck in the cache.

Verification

A call-counting assertion in the spirit of tests/routes/leaderboard_initial_fill_test.ts — N page loads must produce 1 upstream read, not N.
Drive a real tier-up through the local stack and assert the board reflects the new standing within one tick, not one TTL.
Assert epoch monotonicity explicitly; a non-monotonic epoch is a silent staleness bug.

Risk / rollback

Low. Rollback is deleting the cacheTtlS argument — the epoch param is inert on its own.

02Bog Flow

Shipped

8/10/2026, 10:30:22 PM

Click a lifecycle step above to view its details.

03Sludge Pulse

keeb assigned keeb8/10/2026, 7:37:15 PM

Leaderboard phase 1: cache the board read and bust it on tier crossings (no schema change)

Catalog backfill deletes rows for any model whose data names are numeric

migrate-index silently drops entries with shallow paths — all of workflows-evaluated/ and any root-level file

Docs: document catalog completeness check and --repair in doctor reference

reconcile_from_disk_bench.ts fails deno check (stale DenoRuntime stub)

Catalog backfill reconciles against the local cache, not the datastore

Leaderboard board windows re-anchor on wall-clock while the rest of the view is frozen at asOf

Decision: move per-event grant MVs to day-grain — ledger is on track for ~1.5B score_grants / ~4B quest_grants

Leaderboard phase 5: score-rollup-projector write cost at 10B — ~430M rows scanned per 60s tick

Profile candidateIds prune costs a full ledger scan to build itself — 42.88M rows for one keyed read

Leaderboard phase 3: owner-grain board_totals so a cache miss is survivable at 10B

Leaderboard phase 2: take score_grants off the board read path — read today from score_daily

Epic: /leaderboard and /u/{username} do not survive 10B events — measured audit + phased plan

Add native Bedrock Knowledge Base Retrieve support

Datastore-managed configuration for multi-instance serve deployments

Ghosts on the leaderboard 500

Improve multi-word search query handling and guidance

swamp command doesn't check cwd

swamp auth login process fails when run remotely

serve boot OOM: findAllGlobalSince loads all workflow runs as full aggregates during reconciliation

extension search: channel-filtered results carry null version fields, and multiple --channel values collapse instead of unioning

s3-datastore: localHasAllRemoteEntries uses bare rel instead of localRelPath, always returns false in namespaced mode

datastoreSetupExtension migrates .swamp/ to cache root instead of namespace directory

swamp serve writes outputs/ outside the bound namespace — silently never persisted to the remote datastore

s3-datastore: bulk pull writes to cache root instead of the namespace, wedging push and arming a full-datastore delete

git: add pull, fetch --tags, and cherry-pick methods

Workflow runs with wide dataArtifact fan-out (~400/run) OOM both workflow history get and swamp serve boot, crash-looping the scheduler; rollback doesn't help (state-triggered, not release-triggered)

Show user:<name> instead of user:<id> in grant list output

Reusable long-lived invite link: one URL to bring people into the swamp

Exclude swamp internal model types from model list output

Skill docs use invalid two-argument 'vault get' form

access token mint prints an invalid retrieval command (vault get, not vault read-secret)

serve reload and model edit timeout over --server in Kubernetes

swamp auth whoami should report paid tier and trial status

Use whoami entitlement to explain private-extension push refusals

Document --restricted-commands flag for swamp serve

vault put: pass tags through to extension provider CreateSecret/setSecret calls

Docs: update manual for 23 new --server commands

Update VaultProvider interface in @systeminit/swamp-testing to support optional tags in put()

Docs: document --grants-file flag for swamp serve

Docs: document --server support for serve reload

Docs: add --tag flag to vault put reference

Docs: update manual for --detach-runs deprecation and HA auto-detection

Document the Codex plugin in swamp docs

Add --server support to CLI commands missing it

Document the swamp Claude Code plugin in the swamp documentation

Make --server a global CLI option

Add --server support to model cancel (server-side run cancellation)

Load grants from external source on swamp serve startup

swamp vault put should support tags for secret governance

Improve hot reload of the deamon, include extension source/bundle changes

Authenticated users must re-login after swamp serve restart

Add --server support to serve reload

Harden queue claim semantics: atomic claims + status-guarded markDelivered/markFailed across app queues

Auto-detect HA mode and deprecate --detach-runs flag

version-drift check (from #236) is a raw string-inequality, false-positives on multi-model manifests and metadata-only bumps

Resend newsletter audience sync: contacts, segments, backfill, unsubscribe sync-back

Move per-login token secrets from vault to encrypted control-plane store

Docs: document macOS TLS trust limitation and SSL_CERT_FILE workaround

Validate swamp serve deployment stack on startup

Docs: serve config file (.swamp/serve.yaml) and --config flag

Serve config file — read swamp serve configuration from YAML

Deno runtime does not trust ISRG Root YR — extensions fail with valid Let's Encrypt certs

Docs: update hot-reload reference to include trust list refresh

Cross-instance client re-attach for multi-instance swamp serve

vercel codegen: inline rawSyncResult in non-unwrap sync path

vercel codegen: unused listAll import blocks extension publish for 4 models

vercel codegen: add round-trip idempotency test for the pipeline

Operatives button on Collectives page does nothing

Track active runs in the control-plane store for cross-instance visibility

validateGlobalArgInputRefs checks step-level inputs but step schema rejects 'inputs' key

Docs: document continuous reconciliation and multi-instance serve flags

Docs: update workflow guard skip output to include guardExpression and guardResult

Make it aware hot reload on the deamon does not take into account trust list changes, or do take into account the trust list for the hot reload.

workflow validate: new 'GlobalArgument input references' check resolves model-definition ${{ inputs.* }} against workflow inputs, false-failing steps that supply the input

Continuous reconciliation for multi-instance swamp serve

Split the score-read circuit breaker so leaderboard pressure can't blank profile scores

oauth_sign_in telemetry records provider as the literal ":id" for every event

Ghost profile bypasses the score-read seam and 500s on a slow ledger

Profile score section blocks the render on a ClickHouse read