Skip to content

Round-trip Anthropic extended-thinking blocks with signatures for multi-turn tool use #250

@maragubot

Description

@maragubot

Background

PartTypeThought was added to the gai abstraction in #257 to surface streamed reasoning content alongside text. On the request side, the Anthropic client currently returns a typed error when a caller passes gai.PartTypeThought back as message history:

case gai.PartTypeThought:
    err := fmt.Errorf("anthropic: %w", errThoughtRoundTripUnsupported)
    span.RecordError(err)
    span.SetStatus(codes.Error, "unsupported part type")
    return gai.ChatCompleteResponse{}, err

The error message embeds a link to this issue. That is fine for single-turn use and forces callers to filter PartTypeThought from message history before sending it back, but it blocks the natural multi-turn pattern (collect streamed parts, feed them back as next-turn history) — and it makes extended-thinking-with-tool-use simply unusable on multi-turn flows, since the Anthropic API requires those signed blocks to be echoed back.

The Anthropic constraint

When the Anthropic API streams an extended-thinking block, it includes a signature, and for some content a redacted_thinking variant carrying an opaque data blob instead of plain text. For multi-turn requests that combine tool calls with extended thinking, the API requires the original signed thinking block (and any redacted_thinking blocks) to be echoed back verbatim in the next turn — otherwise the request is rejected. See https://docs.claude.com/en/docs/build-with-claude/extended-thinking for the full constraint.

Because gai.ThoughtPart(text string) exposes thoughts as a plain string payload, there is currently nowhere to carry the signature or the redacted-thinking data. Round-tripping the part would lose the signature, and Anthropic would reject the next turn — so we hard-error in the gai client today rather than produce a silent failure several frames later on the wire.

What a fix would entail

  1. Extend gai.Part (or ThoughtPart specifically) to carry opaque provider-specific metadata. Concrete shape to evaluate during implementation:
    • (a) Provider any field on Part set by the originating client and consumed only by the same provider's request builder. Honest about leakage; uses any.
    • (b) Sibling part types per provider (AnthropicThoughtPart, GoogleThoughtPart — see Round-trip Gemini 3.x thought_signature for multi-turn tool use #256) gated behind an interface. Avoids any but explodes the type surface.
  2. On the stream-read side, the Anthropic client populates the metadata from the SDK's ThinkingBlock.Signature / RedactedThinkingBlock.Data.
  3. On the request-build side, the Anthropic client emits ThinkingBlockParam{Signature: ...} or RedactedThinkingBlockParam{Data: ...} — replacing today's hard error with a real round-trip.
  4. The Google and OpenAI clients can either ignore the new metadata (current behaviour) or carry it through as opaque pass-through.
  5. The errThoughtRoundTripUnsupported error path (and its tests) goes away.

This issue and #256 (Gemini 3.x thought_signature) have the same shape — both want opaque per-provider metadata on gai.Part. The two should be designed together so we don't end up with two parallel mechanisms.

Why deferred

The cleanest API surface for opaque per-provider metadata is non-trivial: a typed field leaks Anthropic specifics into the gai abstraction, while a generic map[string]any is loose. We chose to ship the simpler "thoughts are output-only on Anthropic" model in #257 and resolve the round-trip story when a real caller hits the constraint or when we have head-room to design the metadata shape carefully.

Workaround until fixed

Callers who need multi-turn extended-thinking + tool use on Anthropic must filter gai.PartTypeThought parts out of the message history themselves before sending the next turn. Doing so loses the thinking context for the model, but unblocks the request.

See docs/decisions.md (entry: "Per-client ThinkingLevel constants (2026-04-29)") for the broader principle that gai does not pre-validate provider constraints, and the docs/diary/2026-04-29-per-client-thinking-levels.md Step 7 narrative for the round-2 reasoning behind the typed-error choice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions