Skip to content

fix(responses): report truncation and failure in streaming conversion - #2171

Merged
looplj merged 1 commit into
looplj:unstablefrom
Lxiny-zy:fix/responses-stream-completion-status
Aug 10, 2026
Merged

looplj merged 1 commit into
looplj:unstablefrom
Lxiny-zy:fix/responses-stream-completion-status

Conversation

@Lxiny-zy

@Lxiny-zy Lxiny-zy commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Problem

Two gaps in the Responses streaming conversion made abnormal terminations look like success.

  1. Upstream Responses -> Chat Completions client (outbound_stream.go): response.completed always produced finish_reason: stop (or tool_calls), ignoring the status field on the response. A response truncated by max_output_tokens was reported to the client as a clean stop; a failed response was reported as a normal completion.

  2. Upstream Chat Completions -> Responses client (inbound_stream.go): the final response.completed was always built with status: completed. A stream that ended with finish_reason: length (truncation) or content_filter (moderation rejection) reached the Responses client as a success.

Changes

  • outbound_stream.go: map response.completed status onto finish_reason:
    • incomplete -> length, or content_filter when incomplete_details.reason says so
    • failed -> error
    • cancelled/canceled -> cancelled
    • absent / completed status keeps the existing stop/tool_calls inference
  • inbound_stream.go: map finish_reason onto the response status before building response.completed:
    • length -> incomplete with incomplete_details.reason: max_output_tokens
    • content_filter -> incomplete with reason: content_filter
    • error -> failed
    • cancelled/canceled -> cancelled
    • other values leave the default completed
  • Both places that previously set status: completed unconditionally now only do so when no terminal status was mapped, so the finish-reason mapping survives both the usage-chunk path and the stream-end fallback.

Tests

  • TestOutboundTransformer_TransformStream_MapsCompletedStatusToFinishReason: incomplete / content_filter / failed / cancelled / canceled / nil status / completed + tool calls
  • TestInboundTransformer_TransformStream_MapsFinishReasonToCompletedStatus: length / content_filter / error / cancelled / canceled / unknown / stop / tool_calls
  • TestInboundTransformer_TransformStream_UsageBeforeFinishReasonKeepsMappedStatus: usage chunk arriving before the finish_reason chunk

go test ./llm/transformer/openai/responses passes (the websocket executor test fails locally on Windows with a wsarecv error unrelated to these changes; it passes in CI).

Summary by CodeRabbit

  • Bug Fixes

    • Improved stream status handling so completion, incomplete, failed, and cancelled responses are preserved correctly.
    • Added accurate finish reasons for length limits, content filtering, errors, cancellations, and tool calls.
    • Prevented usage updates from overwriting previously determined terminal statuses.
  • Tests

    • Added coverage for inbound and outbound status and finish-reason mappings, including usage events received before completion details.

Two bugs in the Responses streaming conversion:

1. outbound (Responses upstream -> Chat Completions client)
   response.completed always mapped to finish_reason stop or tool_calls,
   ignoring the response's status field. Truncated responses
   (status: incomplete) were reported as a clean stop, and failures
   (status: failed) were swallowed entirely. The status is now mapped:
   incomplete -> length / content_filter (from incomplete_details),
   failed -> error, cancelled -> cancelled. Unknown or missing status
   keeps the old stop/tool_calls inference.

2. inbound (Chat Completions upstream -> Responses client)
   response.completed was always built with status: completed, so a
   stream truncated by the upstream ('length') or rejected by content
   filters reached the Responses client as a success. finish_reason is
   now mapped onto the response status (length -> incomplete with
   max_output_tokens details, content_filter -> incomplete, error ->
   failed, cancelled -> cancelled), and the final fallback only sets
   completed when no terminal status was mapped.
Copilot AI lite review requested due to automatic review settings August 7, 2026 15:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 763796d6-21a4-4189-9857-b5d7a1d78cb8

📥 Commits

Reviewing files that changed from the base of the PR and between dba642a and 2dbd994.

📒 Files selected for processing (4)
  • llm/transformer/openai/responses/inbound_stream.go
  • llm/transformer/openai/responses/inbound_stream_test.go
  • llm/transformer/openai/responses/outbound_stream.go
  • llm/transformer/openai/responses/outbound_stream_test.go

📝 Walkthrough

Walkthrough

The stream transformers now preserve terminal statuses from Chat Completions finish reasons and map Responses API statuses back to finish reasons. Tests cover incomplete, failed, cancelled, completed, usage ordering, content filtering, and tool-call cases.

Changes

Responses stream terminal status mapping

Layer / File(s) Summary
Inbound finish-reason mapping and preservation
llm/transformer/openai/responses/inbound_stream.go, llm/transformer/openai/responses/inbound_stream_test.go
Maps length, content_filter, error, and cancellation reasons to Responses API statuses. Preserves mapped statuses when usage arrives before the terminal finish-reason chunk.
Outbound status mapping
llm/transformer/openai/responses/outbound_stream.go, llm/transformer/openai/responses/outbound_stream_test.go
Maps incomplete, failed, cancelled, completed, and tool-call outcomes to Chat Completions finish reasons. Tests cover fallback and incomplete-detail cases.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested reviewers: copilot, looplj

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main fix: preserving truncation and failure statuses during Responses streaming conversion.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants