Skip to main content
← Back to list
01Issue
BugOpenSwamp CLIPublicTeam
AssigneesNone

Relationships

#1730 pullChanged never records commitSeq for shard-first (v2) index

Opened by magistr · 8/19/2026

Summary

When using shard-first (v2) index, pullChanged assembles the index from shards and sets indexETag = null (s3_cache_sync.ts:1685). The sidecar update at line ~1824 is guarded by if (indexETag), so markSynced is never called on the shard path. No equivalent commitSeq recording happens.

The sidecar can ONLY be updated via the push path (pushChanged), but that path is gated on localHasAllRemoteEntries() (see companion issue). Together these two bugs form a vicious cycle: pull never writes commitSeq, push can't write commitSeq on partial caches, so the fast path is permanently dead.

Impact

The tryCommitSeqFastPath() probe at the top of pullChanged always fails because v2.commitSeq === undefined in the sidecar. Every pull takes the full slow path: read _meta.json + all shard files from S3 + rebuild 129k-entry index + rewrite 50MB local index file.

Suggested fix

After successful shard assembly in pullChanged, record the assembled commitSeq in the sidecar:

const assembled = await this.assembleIndexFromShards(signal);
if (assembled) {
    this.index = { version: 1, lastPulled: ..., entries: assembled.entries };
    this.scrubIndex();
    await atomicWriteTextFile(this.indexPath, ...);
    indexETag = null;
    // NEW: record commitSeq for pull fast-path
    try {
        const sidecar = this.buildV2State();
        sidecar.commitSeq = assembled.commitSeq;
        await this.writeSyncState(sidecar);
    } catch { /* non-fatal */ }
}

References

  • s3_cache_sync.ts:1672-1688 (shard assembly in pullChanged)
  • s3_cache_sync.ts:1824 (if (indexETag) guard skipping markSynced)
  • s3_cache_sync.ts:1207-1220 (tryCommitSeqFastPath — checks sidecar.commitSeq)

Upstream repository: https://github.com/systeminit/swamp-extensions

Environment

  • Extension: @swamp/s3-datastore@2026.08.12.1
  • swamp: 20260819.011806.0-sha.a9c7ee9a
  • OS: darwin (x86_64)
  • Deno: 2.8.3
  • Shell: /bin/zsh
02Bog Flow
◉OPEN○TRIAGED○IN PROGRESS○SHIPPED

Open

8/19/2026, 5:25:04 PM

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.