fix(responses): report truncation and failure in streaming conversion - #2171
Conversation
Two bugs in the Responses streaming conversion:
1. outbound (Responses upstream -> Chat Completions client)
response.completed always mapped to finish_reason stop or tool_calls,
ignoring the response's status field. Truncated responses
(status: incomplete) were reported as a clean stop, and failures
(status: failed) were swallowed entirely. The status is now mapped:
incomplete -> length / content_filter (from incomplete_details),
failed -> error, cancelled -> cancelled. Unknown or missing status
keeps the old stop/tool_calls inference.
2. inbound (Chat Completions upstream -> Responses client)
response.completed was always built with status: completed, so a
stream truncated by the upstream ('length') or rejected by content
filters reached the Responses client as a success. finish_reason is
now mapped onto the response status (length -> incomplete with
max_output_tokens details, content_filter -> incomplete, error ->
failed, cancelled -> cancelled), and the final fallback only sets
completed when no terminal status was mapped.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe stream transformers now preserve terminal statuses from Chat Completions finish reasons and map Responses API statuses back to finish reasons. Tests cover incomplete, failed, cancelled, completed, usage ordering, content filtering, and tool-call cases. ChangesResponses stream terminal status mapping
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Problem
Two gaps in the Responses streaming conversion made abnormal terminations look like success.
Upstream Responses -> Chat Completions client (
outbound_stream.go):response.completedalways producedfinish_reason: stop(ortool_calls), ignoring thestatusfield on the response. A response truncated bymax_output_tokenswas reported to the client as a clean stop; a failed response was reported as a normal completion.Upstream Chat Completions -> Responses client (
inbound_stream.go): the finalresponse.completedwas always built withstatus: completed. A stream that ended withfinish_reason: length(truncation) orcontent_filter(moderation rejection) reached the Responses client as a success.Changes
outbound_stream.go: mapresponse.completedstatus ontofinish_reason:incomplete->length, orcontent_filterwhenincomplete_details.reasonsays sofailed->errorcancelled/canceled->cancelledcompletedstatus keeps the existingstop/tool_callsinferenceinbound_stream.go: mapfinish_reasononto the response status before buildingresponse.completed:length->incompletewithincomplete_details.reason: max_output_tokenscontent_filter->incompletewithreason: content_filtererror->failedcancelled/canceled->cancelledcompletedstatus: completedunconditionally now only do so when no terminal status was mapped, so the finish-reason mapping survives both the usage-chunk path and the stream-end fallback.Tests
TestOutboundTransformer_TransformStream_MapsCompletedStatusToFinishReason: incomplete / content_filter / failed / cancelled / canceled / nil status / completed + tool callsTestInboundTransformer_TransformStream_MapsFinishReasonToCompletedStatus: length / content_filter / error / cancelled / canceled / unknown / stop / tool_callsTestInboundTransformer_TransformStream_UsageBeforeFinishReasonKeepsMappedStatus: usage chunk arriving before the finish_reason chunkgo test ./llm/transformer/openai/responsespasses (the websocket executor test fails locally on Windows with awsarecverror unrelated to these changes; it passes in CI).Summary by CodeRabbit
Bug Fixes
Tests