pullFile in @swamp/s3-datastore writes to the bare cache-relative path while reading from the namespaced S3 key. Every bulk pull therefore hydrates the whole datastore into the ROOT of ~/.swamp/repos/<repoId>/, while pushes only ever walk <repoId>/<namespace>/. The cache ends up holding both layouts at once, and the repo can no longer push.

Location

datastores/_lib/s3_cache_sync.ts (~L1518-1548):

const localPath = assertSafePath(this.cachePath, relativePath);          // <cache>/data/...
({ data } = await ... this.s3.getObject(this.dataKey(relativePath), signal)); // <ns>/data/...

Index keys are bare, so pullChanged writes every object to the cache root. Push does the inverse (walks <cache>/<ns>/**, strips the prefix). The local path needs the same dataKey-symmetric namespace prefixing the read side already applies.

Lazy hydration is NOT affected — that hook passes a path already containing the namespace, misses dataKey(), and lands correctly via the bare fallback. Only the bulk path is broken.

Reproduce

On a repo with namespace configured:

swamp datastore sync --pull
ls ~/.swamp/repos/<repoId>/ -> data/, outputs/, workflow-runs/, ... at root, alongside <namespace>/
swamp datastore sync --push -> Cannot push: un-migrated data found at root level
swamp doctor datastores -> FAIL

Three separate commands each reproduce it in one invocation: bare swamp datastore sync, sync --pull, and swamp datastore setup extension (which calls a full pullChanged).

Impact: this can delete the entire remote datastore

Worse than blocked pushes. In the full-walk branch, pushChanged computes toDelete as every index entry not present under <cache>/<namespace>/. With the files stranded at the root, that set is effectively the whole datastore. It is gated only on dirtyPathsOverflowed, which flips true when one command exceeds DIRTY_PATHS_CAP = 200 dirty paths.

My repo sat in this state for 8 days with 7431 live S3 objects one boolean away from deletion, plus 348 files of local run history that could not be pushed the whole time. On a different repo, a related index inconsistency did prune ~1456 live objects for real.

The documented recovery does not work

The error text points at swamp datastore namespace migrate, which is a dead end once both layouts exist: findFileCollisions blocks on the overlap, and with the namespace dir non-empty it prints the preview and exits 0 without migrating. doctor --repair returns {"status":"not_needed"} (it only targets foreign-namespace contamination).

migrate is also actively harmful here: it calls invalidateCatalog(), and the next query triggers bulkReplaceNamespace -> DELETE FROM catalog WHERE namespace = ?, rebuilding from local disk and truncating the catalog. That is the mechanism behind the 3-byte .catalog-export.json I have seen reported elsewhere.

What actually repaired it

Out-of-band, bypassing swamp entirely:

stop all other writers
aws s3 sync s3://<bucket>/<prefix>/<ns>/ <cache>/<ns>/ — download-only, never --delete, excluding .datastore-index.json, _index/*, .namespace.json
rm -rf the six root-level trees (they must be removed, not emptied — both the doctor check and the push guard stat isDirectory)
swamp datastore catalog pull --namespaces <ns> to restore the truncated catalog
push

The excludes in step 2 matter: isInternalCacheFile matches against the cache-relative path, so a downloaded <ns>/_index/... is not filtered and round-trips back as junk index entries.

Suggested fixes

Make pullFile namespace-symmetric with dataKey (the actual bug).
Never compute a delete set that large from a layout mismatch — treat "index entry exists but local file is wholly absent from the namespace tree" as needs-investigation, not delete.
Make namespace migrate handle the both-layouts-present case, or fail with a message that does not recommend itself.
Implement previewPush — sync --push currently has no dry-run and no confirmation, so there is no supported way to inspect a push (including its deletes) before it happens.

Environment

CLI 20260806.001601.0-sha.5bfddaa2 (also present in 20260730.013940.0)
@swamp/s3-datastore with both prefix and namespace configured
macOS arm64

Upstream repository: https://github.com/systeminit/swamp-extensions

Environment

Extension: @swamp/s3-datastore@2026.07.25.1
swamp: 20260807.031228.0-sha.9a36314e
OS: darwin (aarch64)
Deno: 2.8.3
Shell: /bin/zsh

02Bog Flow

Shipped

8/7/2026, 10:23:52 PM

Click a lifecycle step above to view its details.

03Sludge Pulse

stack72 assigned stack728/7/2026, 10:11:10 PM

stack72 commented 8/7/2026, 10:24:05 PM

Hey @sntxrr — this bug was fixed in PR #158 (swamp-club#1483). The fix shipped in @swamp/s3-datastore@2026.07.31.1. You're on @2026.07.25.1 — upgrading to the latest version should resolve the pullFile namespace path issue. A related latent bug in localHasAllRemoteEntries() was also found during triage and will be tracked as a separate issue.

Improve multi-word search query handling and guidance

swamp command doesn't check cwd

swamp auth login process fails when run remotely

serve boot OOM: findAllGlobalSince loads all workflow runs as full aggregates during reconciliation

extension search: channel-filtered results carry null version fields, and multiple --channel values collapse instead of unioning

s3-datastore: localHasAllRemoteEntries uses bare rel instead of localRelPath, always returns false in namespaced mode

datastoreSetupExtension migrates .swamp/ to cache root instead of namespace directory

swamp serve writes outputs/ outside the bound namespace — silently never persisted to the remote datastore

s3-datastore: bulk pull writes to cache root instead of the namespace, wedging push and arming a full-datastore delete

git: add pull, fetch --tags, and cherry-pick methods

Workflow runs with wide dataArtifact fan-out (~400/run) OOM both workflow history get and swamp serve boot, crash-looping the scheduler; rollback doesn't help (state-triggered, not release-triggered)

Show user:<name> instead of user:<id> in grant list output

Reusable long-lived invite link: one URL to bring people into the swamp

Exclude swamp internal model types from model list output

Skill docs use invalid two-argument 'vault get' form

access token mint prints an invalid retrieval command (vault get, not vault read-secret)

serve reload and model edit timeout over --server in Kubernetes

swamp auth whoami should report paid tier and trial status

Use whoami entitlement to explain private-extension push refusals

Document --restricted-commands flag for swamp serve

vault put: pass tags through to extension provider CreateSecret/setSecret calls

Docs: update manual for 23 new --server commands

Update VaultProvider interface in @systeminit/swamp-testing to support optional tags in put()

Docs: document --grants-file flag for swamp serve

Docs: document --server support for serve reload

Docs: add --tag flag to vault put reference

Docs: update manual for --detach-runs deprecation and HA auto-detection

Document the Codex plugin in swamp docs

Add --server support to CLI commands missing it

Document the swamp Claude Code plugin in the swamp documentation

Make --server a global CLI option

Add --server support to model cancel (server-side run cancellation)

Load grants from external source on swamp serve startup

swamp vault put should support tags for secret governance

Improve hot reload of the deamon, include extension source/bundle changes

Authenticated users must re-login after swamp serve restart

Add --server support to serve reload

Harden queue claim semantics: atomic claims + status-guarded markDelivered/markFailed across app queues

Auto-detect HA mode and deprecate --detach-runs flag

version-drift check (from #236) is a raw string-inequality, false-positives on multi-model manifests and metadata-only bumps

Resend newsletter audience sync: contacts, segments, backfill, unsubscribe sync-back

Move per-login token secrets from vault to encrypted control-plane store

Docs: document macOS TLS trust limitation and SSL_CERT_FILE workaround

Validate swamp serve deployment stack on startup

Docs: serve config file (.swamp/serve.yaml) and --config flag

Serve config file — read swamp serve configuration from YAML

Deno runtime does not trust ISRG Root YR — extensions fail with valid Let's Encrypt certs

Docs: update hot-reload reference to include trust list refresh

Cross-instance client re-attach for multi-instance swamp serve

vercel codegen: inline rawSyncResult in non-unwrap sync path

vercel codegen: unused listAll import blocks extension publish for 4 models

vercel codegen: add round-trip idempotency test for the pipeline

Operatives button on Collectives page does nothing

Track active runs in the control-plane store for cross-instance visibility

validateGlobalArgInputRefs checks step-level inputs but step schema rejects 'inputs' key

Docs: document continuous reconciliation and multi-instance serve flags

Docs: update workflow guard skip output to include guardExpression and guardResult

Make it aware hot reload on the deamon does not take into account trust list changes, or do take into account the trust list for the hot reload.

workflow validate: new 'GlobalArgument input references' check resolves model-definition ${{ inputs.* }} against workflow inputs, false-failing steps that supply the input

Continuous reconciliation for multi-instance swamp serve

Split the score-read circuit breaker so leaderboard pressure can't blank profile scores

oauth_sign_in telemetry records provider as the literal ":id" for every event

Ghost profile bypasses the score-read seam and 500s on a slow ledger

Profile score section blocks the render on a ClickHouse read

Council: set the scoring balance — command tiers, farm-decay parameters, cross-category ratio

Scoring coverage gaps: instrument drop-off, score ripples/mentions, reward invite-converts

Push down the two cheap leaderboard scans: ghost-gate dictionary and eventCounts owner filter

Materialize an active-days-by-id table for the streak board

score-rollup-projector: O(all-time ledger) closure + per-pod fan-out saturates prod ClickHouse CPU

Startup cache hydration for swamp serve instance replacement

Output a log line for steps skipped due to guards

extension quality scorer misreads a template-literal import specifier in generated code as an unresolvable bare import, hard-erroring the whole extension as UNSCORABLE

Add @swamp/kubernetes/statefulset model type

Docs: access token commands reference says plaintext is 'shown once'

Document --restricted-model-types flag for swamp serve

swamp auth server-login ignores SWAMP_SERVE_URL

s3-datastore extension: pullChanged writes namespace-stripped paths to local cache

serve oauth: an unrecognized device-token error kills the whole serve process instead of the bootstrap

s3-datastore: bidirectional sync pull writes namespace-stripped copies to cache root, wedging push with no supported recovery

CLI reports 'HTTP/2 not supported by this client' when the server actually returned 401 — masks auth failures as a transport error