Stream long Anthropic requests while preserving reasoning settings and timeout semantics #2643

Problem and impact

The Anthropic adapter always uses client.messages.create(). With the installed Anthropic SDK, a request allowing 64,000 output tokens raises a local ValueError before inference:

Streaming is required for operations that may take longer than 10 minutes.

This prevents using the output allowance requested by the AIRO joint elicitation, including Claude Opus 5 with adaptive thinking and output_config.effort="max".

In our live replication attempts, the initial 64,000-token requests failed before inference. Reducing the allowance to 20,000 permitted calls, but one Opus turn then used the entire allowance on thinking and returned no text. A later explicit amendment reduced effort from max to high. That allowed the study to continue, but changed the intended experimental configuration. Lowering the token allowance is therefore not an adequate adapter fix.

Inspected implementation and environment

• EDSL checkout: 75a8ec03474137d1d426d8e0a281920da4a4a012; the cited adapter was unmodified locally.
• Installed Anthropic SDK: 0.80.0 (also matches the checkout's ^0.80.0 dependency constraint).
• anthropic_service.py already forwards thinking and output_config, then calls messages.create() at line 244 and returns response.model_dump().
• LanguageModel timeout uses EDSL_API_TIMEOUT, whose default is 60 seconds. The cached call wraps execution in asyncio.wait_for(). Streaming does not by itself remove this outer deadline; actual remote overrides require verification.

Deterministic SDK reproduction (no credentials or network)

from anthropic._base_client import BaseClient

# Diagnostic against installed SDK 0.80.0; this is a private SDK helper,
# not a proposed dependency for EDSL's production implementation.
BaseClient._calculate_nonstreaming_timeout(None, 64000, None)
# Raises ValueError: Streaming is required ...

The helper estimates 3600 * max_tokens / 128000 seconds and rejects nonstreaming requests whose estimate exceeds 600 seconds. It can also enforce a model-specific nonstreaming token limit. Do not treat 20,000 as a universal provider limit.

Proposed implementation

Use the SDK streaming interface for long requests and assemble the complete response internally:

async with client.messages.stream(**create_kwargs) as stream:
    response = await stream.get_final_message()
return response.model_dump()

The caller can continue to await one final dictionary; this does not require streaming UI output or changing ordinary survey answer APIs.

• Select a documented policy for when to stream. Streaming all Anthropic calls is an option; a conservative long-request branch is another. A prototype using streaming above 20,000 tokens passes mocked checks but is not a complete capability policy.
• Preserve the requested max_tokens, thinking, and output_config exactly. Retain current model-specific temperature behavior.
• Preserve all final content blocks, signatures, stop reason, response identifier, and usage from model_dump(); do not concatenate only text deltas and lose the rest.
• Close stream/client resources on success, provider failure, timeout, and cancellation. Distinguish an interrupted stream from a completed response.
• Configure and document an adequate total inference deadline for long reasoning calls, including the EDSL wrapper and remote task/worker limits. Prefer explicit per-run or per-call timeout propagation if supported; do not globally make every ordinary call wait indefinitely.
• Do not automatically change reasoning effort or retry a partially consumed stream without making retry/cost implications visible.

Acceptance tests

• A mocked 64,000-token request uses streaming and never invokes nonstreaming create() on that path.
• get_final_message() is awaited and the returned dictionary retains text, thinking/signature blocks, stop reason, response ID, and usage.
• thinking={"type":"adaptive"} and output_config={"effort":"max"} reach the SDK unchanged.
• Cover ordinary short calls according to the chosen policy, including legacy temperature handling and models where temperature must be omitted.
• Cover stream errors, timeout, and cancellation with resource cleanup and no false successful answer.
• Exercise the path through EDSL's outer timeout wrapper, not just the adapter method, with deterministic fake timing.
• Confirm serialization/remote execution preserves the requested allowance and timeout configuration.

Validation already performed and rollout

A candidate streaming branch was compiled and invoked against a fake SDK client at 20,000 and 64,000 tokens. Both checks passed; the large request used streaming, retained adaptive/max settings, awaited the final message, and returned its content/usage. The SDK rejection above was also reproduced directly. The existing Anthropic and response-parsing test files passed 21 tests, with two Pydantic deprecation warnings. These are local/mocked checks, not live validation of the patch.

Deploy to the remote inference workers as well as local EDSL installations. Then run a bounded, fresh remote smoke test with a large permitted output budget and inspect effective settings, final stop reason, usage, and worker version. A test that happens to produce few output tokens still validates acceptance of the large allowance; a separate timeout/cancellation test must establish long-running behavior.

Related: #2628 (token defaults and reasoning exhaustion), #2626 (answer-health metadata). This issue changes transport so a requested allowance can be honored; those issues address defaults and how incomplete answers are represented.

Companion issues: #2642 (OpenAI reasoning forwarding) and #2644 (native sessions, completion statuses, and request/cache identity).