Repository navigation
Conversation
…stream_chat astream_chat split each raw iter_any() chunk on the literal string 'data: ', so a delta whose text contains 'data: ' (log excerpts, code snippets) and any SSE frame spanning two network reads failed with JSONDecodeError and killed the stream. The sync stream_chat path already iterates lines and slices off the data: prefix. Buffer the decoded bytes, split on newlines, skip SSE comments and blank separators, and parse only complete data: lines. A trailing unterminated line is still processed so a stream without a final newline keeps its last event. raw now holds the parsed event payload, matching stream_chat and chat. Adds three regressions for the reported failure modes plus one guard for multiple events in a single chunk; existing test_astream_chat still passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
SiliconFlow.astream_chatparsed the streaming response by splitting each rawiter_any()chunk on the literal string"data: ". That breaks in two ways thatstream_chat(the sync path) already handles correctly:data:— for example a log excerpt or a code snippet — is split into two fragments andjson.loadsfails mid-stream, so the whole async stream dies.iter_any()yields arbitrary chunk boundaries, so an event split across reads is decoded as two invalid JSON fragments.This PR reads the stream the same way the sync path does: buffer bytes, split on newlines, and only parse complete
data:lines. SSE comment lines (: keepalive) and blank separators are ignored,[DONE]ends the stream, and a trailing unterminated line is still processed so a stream that ends without a final newline does not lose its last event.rawon the yieldedChatResponseis now the parsed event payload (matchingstream_chatandchat) instead of the raw network chunk.Why this is the right fix
data:prefix; the async path was the only place that re-derived framing from a byte-chunk split.Tests
Three regressions cover the reported failure modes; each is RED on
mainand GREEN with this change, and the existingtest_astream_chatstill passes.maintest_astream_chat_content_containing_data_markerJSONDecodeErrortest_astream_chat_event_split_across_chunksJSONDecodeErrortest_astream_chat_ignores_comments_and_blank_linesJSONDecodeErrortest_astream_chat_multiple_events_in_one_chunkWithout the source change, the same suite reports
3 failed, 7 passed.Package version bumped to
0.6.1.AI assistance disclosure
Per the repository's AI contribution guidelines: this change was prepared with an AI coding agent. I reviewed the diff, reproduced both failure modes locally against
main, confirmed the three regressions fail without the fix and pass with it, and ran the package's test and lint commands above.Closes #22223