fix(streaming): add advanceStreamWithBuffer method #1190
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@mikestaub is attempting to deploy a commit to the tambo ai Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
|
@mikestaub the intention behind our api is that we always return fully formed JSON - we're handling the partial JSON response from the llm provider server side, and always (theoretically) emitting correct JSON. But that doesn't mean it is always working though! We did have some code in the sdk to just ignore malformed chunks at one point, I wonder if it broke. @MichaelMilstead do you remember some of the issues we were seeing and where the fix to ignore bad chunks was? |
MichaelMilstead
added a commit
that referenced
this pull request
Nov 17, 2025
Co-authored-by: CharlieHelps <charlie@charlielabs.ai>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#1189
Issue: Streaming Responses Fail with
Failed to parse JSON after multiple chunks.Summary
Interactive chat sessions were intermittently aborting when the UI attempted to
stream assistant messages. The React SDK delegated streaming to
@tambo-ai/typescript-sdk’sadvanceStream, which tries toJSON.parseeachnewline-delimited SSE chunk as it arrives. Under normal network conditions the
server can split a single JSON payload across multiple TCP frames, so the client
occasionally received incomplete JSON and threw
Failed to parse JSON after multiple chunks.—terminating the stream and surfacing an error toast.What Changed
advanceStreamWithBuffer(react-sdk/src/util/advance-stream.ts)that wraps the generated SDK call but buffers
data:lines until a full SSEevent is assembled before attempting to parse JSON.
tambo-thread-providerand related tests to use the buffered helperso streaming consumers no longer experience partial JSON failures.
react-sdk/src/__tests__/advance-stream.test.tstoverify we correctly handle split chunks and still surface the original error
when the payload remains malformed.
Why It Works
By reassembling the event payload inside the React SDK we ensure
JSON.parseonly runs on complete SSE messages. This prevents premature failures caused by
network chunking, without waiting for upstream regeneration of the Stainless
SDK. The change is isolated to the client, so existing API contracts remain
unchanged while the UI regains stable streaming behavior.