Support native tool conversations with completion statuses and auditable request/cache identity #2644

Outcome

Support resumable, provider-native research conversations through EDSL while retaining inspectable Jobs/Results, caching, costs, and remote execution. A model should be able to request a tool, receive the corresponding result in the next native turn, and continue without flattening the exchange into a new user prompt. Each turn should expose a structured completion status and an auditable request identity.

This groups three coupled requirements:

1. Native messages, tool declarations/calls/results, and continuation state.
2. Explicit completed/tool-requested/truncated/refused/error outcomes with complete raw responses and usage.
3. Effective-request provenance and cache keys that distinguish endpoints and all semantically relevant input.

Motivation and boundaries

The AIRO runner at commit 646da9a2 accumulates native conversation history and preserves Anthropic thinking blocks during tool use. Its research tools search, read paginated pages, and accept partial forecast submissions. Re-encoding this interaction as JSON instructions inside repeated free-text survey questions is an approximation, even if the visible words and final probabilities are saved correctly.

EDSL should own faithful inference transport and durable records. A study controller can own the research loop, tool execution, stopping rules, and probability validation. Tavily search/extraction can remain external; neither a Firecrawl integration nor a new general-purpose agent framework is a prerequisite. The initial implementation should support application-executed tools for OpenAI Responses and Anthropic Messages. Other endpoints/providers should advertise support explicitly and fail clearly when unsupported.

An internal serializable request/response type is appropriate; this proposal does not require a new top-level Jobs dimension. Align public configuration with existing Model/PromptPlan work.

Related issues: coordinate rather than duplicate

• Feature: Jinja-templated conversation history in PromptPlan #2641 proposes Jinja-templated, fixed user/assistant history in PromptPlan. Its initial scope explicitly excludes tools and automatic conversation replay. Reuse its rendering/message plumbing where useful; this issue adds dynamic tool turns and continuation state. Do not repeatedly template provider-returned content as executable Jinja.
• Support OpenAI Responses API tools such as web search #2594 requests built-in OpenAI Responses tools and preservation of citations/tool traces. Share tool configuration, serialization, and capability handling. Application-executed cross-provider tools also need stable call IDs and explicit tool results.
• Expose empty or truncated model answers as answer-health metadata #2626 covers answer-health metadata. Reuse/extend that contract for incomplete/refused/error outcomes instead of adding a conflicting status system.
• Apply reasoning-aware output-token defaults consistently across inference services #2628 covers reasoning-aware token defaults, truncation, and failure provenance.
• Model.to_dict() serializes a default temperature (0.5) that contradicts runtime behavior #2510 covers differences between serialized parameters and actual provider parameters.

Treat this as an integration issue with linked implementation work, not a replacement for those narrower proposals.

Findings from the checkout

Inspected commit: 75a8ec03474137d1d426d8e0a281920da4a4a012. The source files cited here were unmodified locally.

• LanguageModel exposes user/system prompt calls. Its cached inference path forwards prompt strings, files, and optionally a response schema, rather than an ordered native message/tool exchange.
• OpenAI MessageBuilder constructs system/user messages; Anthropic constructs one user message.
• RawResponseHandler first tries a fixed text path. Anthropic thinking-only truncation falls back to empty text; refusal text is extracted as ordinary text; when the first content block is text, subsequent text blocks can be omitted. Responses already has some incomplete-response handling, so extend rather than discard that logic.
• CacheEntry hashes model, parameters, system/user prompts, and iteration. Service is stored but excluded from identity. The call-site cache parameters use configured model parameters; response schema is forwarded later, outside those cache inputs.

Deterministic reproductions

No provider calls are needed:

from edsl.language_models.raw_response_handler import RawResponseHandler
from edsl.caching.cache_entry import CacheEntry

h = RawResponseHandler(["content", 0, "text"], inference_service="anthropic")
print(h.get_generated_token_string({
    "content": [{"type": "thinking", "thinking": "..."}],
    "stop_reason": "max_tokens",
}))  # "": no explicit truncation in the extracted text

print(h.get_generated_token_string({
    "content": [{"type": "text", "text": "I cannot provide that."}],
    "stop_reason": "refusal",
}))  # ordinary text extraction; no refusal classification here

print(h.get_generated_token_string({
    "content": [{"type": "text", "text": "first"},
                {"type": "text", "text": "second"}],
    "stop_reason": "end_turn",
}))  # "first": second block omitted

args = dict(model="gpt-6-astra", parameters={"reasoning_effort": "xhigh"},
            system_prompt="system", user_prompt="hello", output="{}")
assert CacheEntry(**args, service="openai").key == \
       CacheEntry(**args, service="openai_v2").key
# Actual equality despite different API endpoints.

These probes describe extraction/cache behavior, not a claim that every downstream question type accepts the returned text as a valid answer.

Proposed contract

Native request and continuation

• Preserve ordered roles and typed content, tool definitions and choices, call IDs, tool results, and provider-specific continuation material. Retain signed/opaque thinking blocks as needed for continuation without reinterpreting or rewriting them.
• Define provider adapters explicitly. Preserve native semantics rather than silently flattening, merging turns, dropping blocks, or changing endpoints. Validate missing/duplicate tool IDs and unsupported combinations before dispatch.
• Allow a caller to submit one turn, receive tool requests, execute tools externally, append their results, persist the session, and resume it later. Keep concurrent interview histories isolated.
• Store immutable turn records with stable session/turn/parent identifiers. Tool receipts should identify the call they satisfy and preserve the supplied result/error. A resumed completed turn must not trigger duplicate inference or tool execution implicitly.
• Retain ordinary prompt-only behavior. Decide how fixed PromptPlan history and dynamic session history compose; document that order and avoid appending history twice.

Response and status

• Return normalized completion status alongside provider stop/status reasons, ordered content blocks, all tool calls, visible text, usage/cost, response ID, and raw response. Suggested statuses: completed, tool_requested, truncated, refused, error; align naming with Expose empty or truncated model answers as answer-health metadata #2626.
• Keep answer presence/validation separate from completion status. A parseable partial answer can still be truncated; a successful tool request can legitimately contain no answer text.
• Preserve all text blocks in order for the text convenience view while retaining the original structured response. Do not treat reasoning content as an answer.
• Preserve diagnostics and available usage for failure/refusal/truncation. Unknown usage remains unknown, not silently zero. Retain per-attempt evidence when retries occur.
• Make policy changes explicit: no silent effort downgrade, token-limit increase, forecast repair, or retry around a refusal. Callers choose whether and how to continue incomplete work.

Provenance and cache identity

• Separate requested configuration from the effective provider request assembled after adapter filtering/defaults. Record provider/endpoint, requested and returned model identifiers, relevant SDK/adapter version, request fingerprint, and any applied parameter adjustments. Record this on the worker that actually sends the request.
• Never include authorization headers or API keys in exported request metadata. Keep existing access controls for prompts, research evidence, and opaque provider continuation blocks.
• Version the cache identity. Include service/API endpoint, effective generation settings, ordered messages/content/tool results, tool definitions/choice, output schema, relevant file identities, and iteration. Include a semantics version so a parameter-forwarding bug fix cannot silently reuse a response produced by the old behavior.
• Canonicalize object-key order without losing meaningful message/list order or rewriting signed content. Transport-only choices such as stream delivery should not unnecessarily fragment equivalent request identities.
• Define a migration policy: ordinary legacy entries may remain readable, but a native session must not reuse an entry that lacks the information needed to prove equivalence. Avoid a blanket destructive cache purge.
• Reflect history/tools in prompt previews, token/cost estimation, and durable Jobs/Results serialization. Preserve cache-hit provenance separately from newly billed inference.

Acceptance criteria

• Mock a two-turn search/read exchange for both OpenAI Responses and Anthropic Messages. Assert exact outgoing roles/content, tool definitions, call IDs, and result associations.
• Preserve Anthropic signed thinking/opaque blocks and provider-specific Responses continuation items across save/load/resume.
• Cover multiple tool calls, tool errors, text plus tools, malformed IDs, unsupported providers, and concurrent sessions without state leakage.
• Cover thinking-only truncation, partial text plus truncation, refusal, ordinary completion, and multiple text blocks. Preserve raw stop reason, usage, and ordered content through Results round trips.
• Identical requests reuse cache; changing only endpoint, message role/order/content, tool definition/result, output schema, or effective reasoning setting changes identity. Object-key order alone does not.
• A change in request semantics cannot reuse a pre-fix result merely because the original serialized Model parameters are unchanged.
• Round-trip the request, continuation, outcomes, and provenance through Jobs/Results and remote worker serialization. Unsupported worker capabilities fail clearly instead of downgrading behavior.
• Existing prompt-only jobs remain compatible, and previews/estimates include the added history/tool payloads.
• Run one bounded remote integration after worker deployment to establish that native tool turns and effective-request metadata survive the actual service path.

Implementation sequence and validation evidence

1. Agree on shared message/outcome/request-identity contracts, coordinating the related issues above.
2. Implement and mock-test native transport for the two initial providers, including ordinary prompt compatibility.
3. Integrate durable continuation, cache migration, Results/preview/cost paths, and worker capability handling.
4. Compare EDSL transport with the authors' runner using fixed evidence and matched model/settings; separately run the live research procedure with the authors' tool provider. New stochastic calls or a changed information date should not be expected to reproduce historical probabilities exactly.

The extraction/cache probes above passed against the inspected checkout; 21 existing targeted Anthropic/response-parsing tests also passed. These findings identify gaps not covered by those tests. No native-session feature has been implemented or deployed by this audit. The companion issues for OpenAI reasoning forwarding and long Anthropic streaming are prerequisites for testing the intended AIRO model settings.

Prerequisites: #2642 (OpenAI reasoning forwarding) and #2643 (long Anthropic streaming).