A long-lived swamp serve is telemetry-dark for its entire uptime. Workflow runs it executes — scheduled, webhook, and API-triggered — record no cli_invocation telemetry at all, and the daemon has no flush path of its own. swamp-club therefore receives zero events for work that demonstrably happened, which is what reset kneel's 32-day streak in #1590.

This is not a scoring or streak bug. swamp-club cannot credit a day it received no event for. The gap is entirely CLI-side.

Impact

Scheduled runs contribute nothing to score, streak, or the combat log.
An operative whose automation is the point of their setup looks idle.
The more correctly you run swamp (persistent serve, cron triggers), the less of your work is visible. #1590 is the first report; it will not be the last.

Root cause — three independent layers, all in serve

1. Serve never wires the workflow telemetry sink.

WorkflowRunDeps.telemetrySink is optional (src/libswamp/workflows/run.ts:287) and is bound in exactly one place: src/cli/commands/workflow_run.ts:323, the interactive swamp workflow run command. Serve's createWorkflowRunDeps (src/serve/deps.ts, returns at :281) never sets it, and telemetrySink appears nowhere in src/serve/. Every serve-side run goes through executeWorkflowWithLocks (src/serve/deps.ts:293) with the sink undefined, so the run produces no parent entry and no child method-invocation entries. There is nothing to flush, authed or not.

2. Recording and flushing exist only in the CLI wrapper.

TelemetryService is constructed at src/cli/mod.ts:1233, recordSuccess is called at :1602, and flushTelemetry at :1613 — all after cli.parse(args) returns. For swamp serve that instant is process exit. A daemon that never restarts never flushes, even for whatever it did spool.

3. The only record of a scheduled fire is an OTel span.

ScheduledExecutionService.executeWorkflow wraps each fire in withSpan("swamp.scheduled.fire", …) (src/libswamp/workflows/scheduled_execution.ts:441). That goes to the operator's own OTLP endpoint, not to swamp-club — the two pipelines are entirely separate (HttpTelemetrySender POSTs to <endpoint>/ingest, src/infrastructure/telemetry/http_telemetry_sender.ts:73). So the operator watches their workflow fire every 4h in their own Grafana while swamp-club sees silence, which makes the bug read as "the site lost my streak."

Evidence (from #1590, verified against the prod events mirror)

11 consecutive successful scheduled runs, 2026-08-09 04:00 → 2026-08-11 00:00 UTC, each on the 4-hour boundary. Zero events reached swamp-club.
The reporter's own serve events carry whole-process durationMs (81s, 166s, 330s, 806s) with startedAt→completedAt — one event per process, emitted at exit. His last one is 2026-08-09 05:06, before the systemd unit went persistent.
Across 230M events, cli_invocation.source ∈ {user, agent_skill} only — there is no scheduler or serve source, and no scheduled-run/workflow-run event type in the catalog.
Result: one missing UTC day (last event 2026-08-09 22:41:47Z, next 2026-08-11 01:49:10Z), streak 32 → 1.

Not the cause

Not OTEL env vars. OTEL_EXPORTER_OTLP_ENDPOINT and friends only affect withSpan/OTLP export under src/infrastructure/tracing/. They cannot divert or suppress the /ingest sender.
Not auth. The flush attaches x-api-key only when ~/.config/swamp/auth.json has credentials (src/cli/mod.ts:1200-1211), and the token is optional — a missing one changes attribution, not whether the send happens. --auth-mode none governs serve's inbound HTTP auth, not outbound telemetry credentials.
Not the cron cadence. 0 */4 * * * fires at 00:00 UTC every day; it could never skip a UTC day. Evenness isn't the problem — silence is.

Suggested fix

Two parts, and the second is the one that's easy to forget:

Wire a telemetrySink into serve's createWorkflowRunDeps so scheduled, webhook, and API-triggered runs record parent + child invocations, with a source that distinguishes them from interactive use (the current cli_invocation.source enum has no value for this).
Give the daemon a periodic flush loop. Serve cannot rely on process exit, so a sink alone just spools forever — and the 7-day hard retention in TelemetryService.cleanupOldTelemetry would eventually drop it.

Identity for the flush should come from the same place the CLI gets it: the user-level identity plus ~/.config/swamp/auth.json credentials the serve process already runs under.

Environment

swamp 20260804.201713.0-sha.c582ab91
Self-hosted swamp serve as a persistent systemd user service, --auth-mode none on loopback
Workflow with trigger.schedule: "0 */4 * * *"; /health reports scheduling.enabled: true

#1590 — the originating report (scheduled runs don't count toward the streak).

02Bog Flow

In Progress

8/11/2026, 4:20:58 AM

Click a lifecycle step above to view its details.

03Sludge Pulse

keeb assigned keeb8/11/2026, 3:08:47 AM

score-rollup projector runs N× per cadence — the run gate is per-replica memory, not shared state

Docs: manual never mentions telemetry, and the serve/scheduling pages imply automated runs are visible when they are not

Explore: CLI binary update integrity lacks independent verification

Leaderboard phase 2: take score_grants off the board read path — read today from score_daily

Explore: reconcile quality rubric's dormant provenance factor

Explore: pull-time signature verification for extensions

Explore: transparency-log integration (Rekor) for extension signatures

Explore: publish-time artifact signing for extensions (cosign keyless)

swamp serve is telemetry-dark: scheduled workflow runs never reach /ingest, and the daemon never flushes

Profile day drill-down returns 0 entries at the 5s ClickHouse ceiling for every operative

A ledger read failure still renders as 'no activity': loadProfileScore swallows to an empty standing behind HTTP 200

swamp quest: drop the heartbeat field the server no longer sends

reconcile_from_disk_bench.ts fails deno check (stale DenoRuntime stub)

Improve multi-word search query handling and guidance

Retire the genesis_public flag and delete the dead cutover transcription scaffolding

Leaderboard phase 1: cache the board read and bust it on tier crossings (no schema change)

Catalog backfill deletes rows for any model whose data names are numeric

migrate-index silently drops entries with shallow paths — all of workflows-evaluated/ and any root-level file

Docs: document catalog completeness check and --repair in doctor reference

Catalog backfill reconciles against the local cache, not the datastore

Leaderboard board windows re-anchor on wall-clock while the rest of the view is frozen at asOf

Decision: move per-event grant MVs to day-grain — ledger is on track for ~1.5B score_grants / ~4B quest_grants

Leaderboard phase 5: score-rollup-projector write cost at 10B — ~430M rows scanned per 60s tick

Profile candidateIds prune costs a full ledger scan to build itself — 42.88M rows for one keyed read

Leaderboard phase 3: owner-grain board_totals so a cache miss is survivable at 10B

Epic: /leaderboard and /u/{username} do not survive 10B events — measured audit + phased plan

Add native Bedrock Knowledge Base Retrieve support

Datastore-managed configuration for multi-instance serve deployments

Ghosts on the leaderboard 500

swamp command doesn't check cwd

swamp auth login process fails when run remotely

serve boot OOM: findAllGlobalSince loads all workflow runs as full aggregates during reconciliation

extension search: channel-filtered results carry null version fields, and multiple --channel values collapse instead of unioning

s3-datastore: localHasAllRemoteEntries uses bare rel instead of localRelPath, always returns false in namespaced mode

datastoreSetupExtension migrates .swamp/ to cache root instead of namespace directory

swamp serve writes outputs/ outside the bound namespace — silently never persisted to the remote datastore

s3-datastore: bulk pull writes to cache root instead of the namespace, wedging push and arming a full-datastore delete

git: add pull, fetch --tags, and cherry-pick methods

Workflow runs with wide dataArtifact fan-out (~400/run) OOM both workflow history get and swamp serve boot, crash-looping the scheduler; rollback doesn't help (state-triggered, not release-triggered)

Show user:<name> instead of user:<id> in grant list output

Reusable long-lived invite link: one URL to bring people into the swamp

Exclude swamp internal model types from model list output

Skill docs use invalid two-argument 'vault get' form

access token mint prints an invalid retrieval command (vault get, not vault read-secret)

serve reload and model edit timeout over --server in Kubernetes

swamp auth whoami should report paid tier and trial status

Use whoami entitlement to explain private-extension push refusals

Document --restricted-commands flag for swamp serve

vault put: pass tags through to extension provider CreateSecret/setSecret calls

Docs: update manual for 23 new --server commands

Update VaultProvider interface in @systeminit/swamp-testing to support optional tags in put()

Docs: document --grants-file flag for swamp serve

Docs: document --server support for serve reload

Docs: add --tag flag to vault put reference

Docs: update manual for --detach-runs deprecation and HA auto-detection

Document the Codex plugin in swamp docs

Add --server support to CLI commands missing it

Document the swamp Claude Code plugin in the swamp documentation

Make --server a global CLI option

Add --server support to model cancel (server-side run cancellation)

Load grants from external source on swamp serve startup

swamp vault put should support tags for secret governance

Improve hot reload of the deamon, include extension source/bundle changes

Authenticated users must re-login after swamp serve restart

Add --server support to serve reload

Harden queue claim semantics: atomic claims + status-guarded markDelivered/markFailed across app queues

Auto-detect HA mode and deprecate --detach-runs flag

version-drift check (from #236) is a raw string-inequality, false-positives on multi-model manifests and metadata-only bumps

Resend newsletter audience sync: contacts, segments, backfill, unsubscribe sync-back

Move per-login token secrets from vault to encrypted control-plane store

Docs: document macOS TLS trust limitation and SSL_CERT_FILE workaround

Validate swamp serve deployment stack on startup

Docs: serve config file (.swamp/serve.yaml) and --config flag

Serve config file — read swamp serve configuration from YAML

Deno runtime does not trust ISRG Root YR — extensions fail with valid Let's Encrypt certs

Docs: update hot-reload reference to include trust list refresh

Cross-instance client re-attach for multi-instance swamp serve

vercel codegen: inline rawSyncResult in non-unwrap sync path

vercel codegen: unused listAll import blocks extension publish for 4 models

vercel codegen: add round-trip idempotency test for the pipeline