Skip to content

Fix concurrent ephemeral-blob loss#5659

Merged
StachuDotNet merged 2 commits into
darklang:mainfrom
OceanOak:blob-race
May 26, 2026
Merged

Fix concurrent ephemeral-blob loss#5659
StachuDotNet merged 2 commits into
darklang:mainfrom
OceanOak:blob-race

Conversation

@OceanOak

@OceanOak OceanOak commented May 20, 2026

Copy link
Copy Markdown
Collaborator

Example of the old bug:

  1. Request A starts. It does: Blob.pushScope exeState
    Stack:
    [ scopeA ] <- top

  2. Request B starts before A finishes. It also does: Blob.pushScope exeState
    Stack:
    [ scopeB ] <- top
    [ scopeA ]

  3. Request A creates an ephemeral blob.
    The old code did not remember "the scope that belongs to Request A." Instead, it always attached new blobs to whatever scope was currently on top of the shared stack. But Request B had already started, so B's scope was now on top. That means A's blob was accidentally registered under B’s scope.
    scopeB tracks idA // wrong
    scopeA tracks nothing

  4. Request B finishes first.
    Blob.popScope exeState This pops scopeB and deletes every blob ID tracked by it. That deletes idA, even though Request A is still running. blobStore no longer has idA

  5. Request A later tries to read or promote its blob. But idA was deleted when B finished, so it fails with: ephemeral blob not found

The core issue was that concurrent requests shared one ExecutionState.blobScopes stack. A stack only works if scopes exit in exact last-in-first-out order.
Concurrent HTTP requests don't guarantee that, so one request could accidentally put its blob into another request's top scope.

After:
DBlob(Ephemeral { id; bytes }) carries the bytes directly inside the value. So as long as the Dval exists, the bytes exist. There is no shared ephemeral store and no UUID lookup that can go stale.

@OceanOak OceanOak force-pushed the blob-race branch 2 times, most recently from 1083c95 to 962a78e Compare May 22, 2026 02:45
@OceanOak OceanOak marked this pull request as ready for review May 22, 2026 12:39
@OceanOak OceanOak requested a review from StachuDotNet May 22, 2026 12:39
@StachuDotNet StachuDotNet merged commit 92f41b0 into darklang:main May 26, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants