[53699:0x7a13694000] 15302 ms: Scavenge (during sweeping) 4085.1 (4092.9) -> 4082.5 (4093.4) MB, pooled: 0.0 MB, 2.48 / 0.00 ms (average mu = 0.222, current mu = 0.168) allocation failure; [53699:0x7a13694000] 15696 ms: Incremental Mark-Compact (reduce) 4088.2 (4096.2) -> 4083.0 (4088.4) MB, pooled: 0.0 MB, 371.37 / 0.00 ms (+ 0.1 ms in 90 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 381 ms) (average

Fatal JavaScriptx out of memory: Ineffective mark-compacts near heap limit

==== C stack trace ===============================

0   swamp                               0x0000000103347e14 uv_unref + 13258976
1   swamp                               0x000000010334b7d8 uv_unref + 13273764
2   swamp                               0x0000000103343f28 uv_unref + 13242868
3   swamp                               0x00000001033adce4 uv_unref + 13676464
4   swamp                               0x000000010358fa78 uv_unref + 15650116
5   swamp                               0x000000010358f254 uv_unref + 15648032
6   swamp                               0x000000010358de48 uv_unref + 15642900
7   swamp                               0x000000010350a7f8 uv_unref + 15104708
8   swamp                               0x00000001039767bc uv_unref + 19741320
9   swamp                               0x000000010481f26c uv_unref + 35112248
10  ???                                 0x000000011cb30840 0x0 + 4776462400
11  ???                                 0x000000011cb22d4c 0x0 + 4776406348
12  ???                                 0x000000011cb2fe18 0x0 + 4776459800
13  ???                                 0x000000011cb35490 0x0 + 4776481936
14  ???                                 0x000000011cb2ff40 0x0 + 4776460096
15  ???                                 0x000000011cb22d4c 0x0 + 4776406348
16  ???                                 0x000000011cb2fe18 0x0 + 4776459800
17  ???                                 0x000000011cb35490 0x0 + 4776481936
18  ???                                 0x000000011cb2ff40 0x0 + 4776460096
19  ???                                 0x000000011cb35490 0x0 + 4776481936
20  ???                                 0x000000011cb2ff40 0x0 + 4776460096
21  ???                                 0x000000011cb35490 0x0 + 4776481936
22  ???                                 0x000000011cb2ff40 0x0 + 4776460096
23  ???                                 0x000000011cb35490 0x0 + 4776481936
24  ???                                 0x000000011cb2ff40 0x0 + 4776460096
25  ???                                 0x000000011cb35490 0x0 + 4776481936
26  ???                                 0x000000011cb2ff40 0x0 + 4776460096
27  ???                                 0x000000011cb35490 0x0 + 4776481936
28  ???                                 0x000000011cb2ff40 0x0 + 4776460096
29  ???                                 0x000000011cb22d4c 0x0 + 4776406348
30  ???                                 0x000000011cb2fe18 0x0 + 4776459800
31  ???                                 0x000000011cb35490 0x0 + 4776481936
32  ???                                 0x000000011cb2ff40 0x0 + 4776460096
33  ???                                 0x000000011cb22d4c 0x0 + 4776406348
34  ???                                 0x000000011cb2fe18 0x0 + 4776459800
35  ???                                 0x000000011cb35490 0x0 + 4776481936
36  ???                                 0x000000011cb2ff40 0x0 + 4776460096
37  ???                                 0x000000011ca98748 0x0 + 4775839560
38  ???                                 0x000000011ca6e230 0x0 + 4775666224
39  swamp                               0x00000001047bb40c uv_unref + 34703064
40  ???                                 0x000000011ca975c0 0x0 + 4775835072
41  ???                                 0x000000011cae2828 0x0 + 4776142888
42  swamp                               0x00000001047b8884 uv_unref + 34691920
43  swamp                               0x00000001048a3218 uv_unref + 35652836
44  swamp                               0x00000001047a766c uv_unref + 34621752
45  swamp                               0x0000000104776778 uv_unref + 34421316
46  swamp                               0x00000001034bc340 uv_unref + 14784012
47  swamp                               0x00000001034bcc08 uv_unref + 14786260
48  swamp                               0x00000001034bcd40 uv_unref + 14786572
49  swamp                               0x0000000103508c90 uv_unref + 15097692
50  swamp                               0x0000000103508bc8 uv_unref + 15097492
51  swamp                               0x0000000102301ac0 uv_default_loop + 24428
52  swamp                               0x000000010230054c uv_default_loop + 18936
53  swamp                               0x000000010211b41c uv_free_cpu_info + 4624
54  swamp                               0x00000001021302ec uv_free_cpu_info + 90336
55  swamp                               0x00000001021182d0 swamp + 17104
56  swamp                               0x0000000102123058 uv_free_cpu_info + 36428
57  swamp                               0x0000000102131660 uv_free_cpu_info + 95316
58  swamp                               0x000000010211f138 uv_free_cpu_info + 20268
59  swamp                               0x0000000102133a8c uv_free_cpu_info + 104576
60  dyld                                0x0000000194c63e88 start + 6688

Steps to Reproduce

Run a data processing workflow

Environment

swamp version: swamp 20260815.023608.0-sha.a648c130
OS: macos 27
Shell: zsh

Additional Context

02Bog Flow

Triaged

8/16/2026, 8:53:16 PM

Click a lifecycle step above to view its details.

03Sludge Pulse

stack72 assigned stack728/16/2026, 8:50:45 PM

stack72 commented 8/17/2026, 8:15:52 PM

Hi @jesserobbins - was it a workflow or a model you were running here? I am trying to track down the critical path to help

Paul

Dispatched steps over 15 minutes cannot write results: dispatch credentials expire and never refresh

data prune still misclassifies live records as orphans on 20260820 (after #1396) — serve-minted token/grant records, and report data of every live workflow

wss:// direct to serve's own TLS listener still fails on 20260817 — client ALPN lands on h2, WS-over-h2 unimplemented (follow-up to #1363)

Concurrent pushes to one namespace fail with HTTP 409 ConditionalRequestConflict — the conditional PUT is never retried

serve: triggerSource not persisted to workflow run summary

dataOutputOverrides not applied during remote execution — ephemeral resources GC'd before override takes effect

extension rm also deletes the lockfile entry, and install on an empty lockfile is a silent success — rm/install is not a round trip

vault: audit trail should record put, delete, and annotate operations

ExtensionWorkflowRepository scans manifest.yaml as a workflow, emitting 'Unknown key manifestVersion' warning

--max-run-duration only applies to detached runs, not attached runs

Cancel endpoint reports status: cancelled for runs it did not actually cancel

@swamp/git: no way to amend a commit, but push can --force

Installation with new account creation times out

pre-flight checks run on orchestrator, not on worker — breaks remote execution with filesystem-dependent checks

Docs: update cancel endpoint response statuses in REST API reference and workflows page

docs: document --remote-only flag for swamp serve

Swamp serve daemon doesn't run with --hot-reload

serve: include workflowName in HealthSnapshotSchedule

Retyping a model instance strands catalog rows no command can reclaim — prune matches on model_id, delete resolves the current type (follow-up to #1009)

knowledge-base retrieve persists query output into the state resource, with lifetime infinite and the query text in the instance name

Method-name collision between a base model type and an extending extension silently drops the whole extension file — no error, doctor reports pass

Skip 50MB monolithic index rewrite on v2 shard-first path

Share single AwsS3Client instance across lock, sync, and verifier

Double shard assembly per command lifecycle (pull + push both read all shards)

pullChanged never records commitSeq for shard-first (v2) index

localHasAllRemoteEntries() permanently blocks sidecar update on partial caches

serve: workflow.history.get cannot find CLI-created runs or look up by run ID

"TRIAL ENDED" modal shown to team plan user when toggling extension visibility

Concurrent suspended workflow runs can't be disambiguated from 'approvals'; fresh runs should supersede matching-input suspended runs

I _think_ I might like a guard in the trigger yaml

swamp cmds don't realise the user is sshed into the system

serve dashboard: add cluster.instances and serve.config endpoints

CEL expressions have timestamp/duration but no current-time value, so no guard can express "older than N"

Support direct type execution (@type) with --server

access token mint writes to user vault, not serve's _token-secrets control plane in HA mode

Generic OTLP endpoint appends signal path after query or fragment

Issue lifecycle start drops top-level issue comments

serve dashboard: add instanceId, triggerSource, and step progress to WorkflowRunSummary

Document worker affinity (affinity: true) for workflows and jobs

Datastore MongoDB driver leaks connections — 5960 accumulated over 8 days

workflow approvals displays the latest run's prompt for every suspended run

@swamp/git: add a read-only working-tree diff for pre-commit review

docs: document --oauth-client-name flag for swamp serve

Docs: update workflow reference to include trigger in JSON output shapes

@swamp/git branch and push methods fail data naming when branch contains '/'

Docs: update workflow-placement.md for job/workflow-level placement inheritance

feat(serve): use identifiable OAuth client_name instead of random UUID

@swamp/git does not pass AbortSignal to Git subprocesses

Global Secret Manager methods build an invalid regional parent

Document model.method() support in assert expressions

No way to store code-review results — verdicts, findings, and cost are not persisted for any repo

Redaction regex boundary bug: label_base64 secrets with an unrecognized label pass through unredacted (e.g. swamp_ API keys)

User token page (/u/<user>) shows only "swamp_..." for every API key, making leaked-key identification impossible

upstream_state treats sparse-fetch branches as having no upstream

Add native support for the Pi coding harness

Cloud Build create supplies project instead of required projectId path parameter

workflows: support labels/target/platform at job and workflow level

swamp access grant create never registers the grant — a --deny created this way silently fails open

Support credentials in datastore config (vault-resolvable)

swamp serve OOMs on repos with 2000+ model definitions

Serve telemetry attribution for collectives

Extension workflows: no way to override trigger configuration

Add a list/discover method to the generated @swamp/aws types — no way to enumerate resources in a region

Document @vault= prefix for webhook secrets in serve.yaml

swamp access grant create installs an unrelated extension from the registry

docs: document SWAMP_API_KEY headless OAuth bootstrap for swamp serve

workflow-succeeded selects the globally-latest run, so siblings block each other under parallel use

swamp serve: webhook secrets should support @vault= resolution

Misleading collective_not_trusted auto-resolve error when a local source extension transiently fails to index during workflow validate

datastore setup: symlinks not counted in bytesCopied, causing misleading '(0B)' output

vault.get() CEL error embeds raw newline when wrapped by model resolver, producing broken output

Feature: user-configurable redaction rules (.swamp/redaction.yaml)

Fatal JavaScript out of memory: Ineffective mark-compacts near heap limit

Sync every signup to Attio, and backfill the existing population (#809 follow-up)

@swamp/issue-lifecycle probes /api/health, which swamp-club stopped serving when the Fresh scaffold stubs were removed

Lab lifecycle POST rejects any payload string that starts with a dollar sign, not just operator keys

Feed: detect and gracefully handle dead (404) post links (link rot)

/feed: per-post unread indicators and mark-as-read to track reading progress

/feed: filter and search by author, title/text, and topic

Add a :micdrop: emoji / reaction shortcode (swamp-native art included)