Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublicTeam
AssigneesNone

Relationships

#1684 swamp serve OOMs on repos with 2000+ model definitions

Opened by webframp · 8/17/2026

Summary

swamp serve hits the V8 heap limit (4GB) and crashes during boot or shortly after the first workflow run on repos with 2000+ model definitions. The eager extension loading path at startup imports all extension bundles into memory simultaneously, exhausting the heap before serve becomes usable.

Reproduction

# Repo with ~2300 models across 30+ extension types
# (@webframp/aws/networking, inventory, alarms, logs, etc.)
swamp serve --auth-mode token --admins user:webframp --no-schedule

Crashes with:

Fatal JavaScript out of memory: Ineffective mark-compacts near heap limit

Memory climbs from ~500MB to 4GB within 20 seconds. If it survives boot, the first workflow execution pushes it over.

Repos with ~100 models serve fine indefinitely.

Root Cause

serve.ts:1297 eagerly calls ensureLoaded() on all four registries:

await Promise.all([
  modelRegistry.ensureLoaded(),
  vaultTypeRegistry.ensureLoaded(),
  datastoreTypeRegistry.ensureLoaded(),
  reportRegistry.ensureLoaded(),
]);

This triggers loadUserModels() -> loader.buildIndex() which imports every extension bundle into the V8 isolate simultaneously. For 30+ extension types with Zod schemas, method implementations, and output spec factories, this exceeds the 4GB V8 heap.

The CLI avoids this via lazy loading (setTypeLoader registers types as lazy entries, importing bundles only when a command targets that type). Serve bypasses the lazy path by calling ensureLoaded() which forces all types to load.

Suggested Fixes (ordered by impact/effort)

A. Lazy extension loading in serve (high impact, moderate effort)

The setTypeLoader mechanism already supports per-type lazy loading. At boot, only run buildIndex to populate the catalog (filesystem scan, no bundle imports). On first workflow step targeting a type, call ensureTypeLoaded(type). Remove or gate the eager ensureLoaded() calls behind a flag like --eager-extensions.

B. --serve-models whitelist (low effort, immediate fix)

Allow serve to specify which model types to load:

swamp serve --serve-models "@webframp/system,@webframp/network,command/shell"

Everything else stays unloaded. Acceptable for dedicated serve instances running specific workflows.

C. Subprocess isolation for local steps (architecture alignment)

Run workflow steps in forked child processes (same as remote workers do). The serve process stays lean (orchestration only), each child loads only the extensions it needs, memory is reclaimed on exit.

Workaround

Use a minimal repo for serve containing only the models and extensions the target workflows need. Repos with 100-200 models work fine.

Environment

  • swamp 20260816.214004.0-sha.6915a209
  • Linux (WSL2, NixOS 26.11)
  • Repo: 2317 models, 30+ extension types, filesystem datastore
02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

8/17/2026, 4:56:45 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.