Skip to content

fix: report failed cloud tools as errors - #139758

Merged
steipete merged 4 commits into
mainfrom
codex/worker-tool-error-outcome
Sep 6, 2026
Merged

steipete merged 4 commits into
mainfrom
codex/worker-tool-error-outcome

Conversation

@steipete

@steipete steipete commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where users running cloud computer tools would see failed operations recorded as successful. A live macOS run returned a screenshot error while its saved tool result said isError: false and its trajectory said success: true. That also gave the next model request incorrect outcome metadata.

Why This Change Was Made

Classify structured tool failures at the shared session lifecycle, before completion events and transcript persistence. Executed results are classified before plugin middleware; each middleware result preserves an explicit error override, including intentional recovery. Pre-execution failures use the existing terminal hook because they never enter execution middleware.

This keeps one finalized outcome for local and remote embedded sessions. It adds no protocol, configuration, schema, timeout, retry, or downstream display workaround. The underlying locked/asleep macOS display is a separate readiness issue; this PR does not fix display capture or bypass screen locking.

User Impact

Failed tools are reported truthfully to users and models. Successful commands, including completed commands with nonzero exit codes, retain their existing semantics. Plugins can explicitly recover from errors without discarding diagnostic details. Optional file absence remains an intentional successful query result.

Evidence

Final head: 6f45aeec60f9c74dfb6734cc3350bd96a2dba653.

Real OpenAI cloud-agent verification on the retained macOS host used a fresh worker built from this exact revision. Each result was matched by tool call ID in both live trajectory and committed transcript:

Operation Trajectory isError / success Committed isError
Screenshot while native display unavailable true / false true
Successful uname and task-file absence check false / true false
Read with optional:true, returning not_found false / true false
Screenshot after ordinary WebVNC wake, actual 1024×768 image false / true false

Connecting WebVNC in view-only mode alone did not restore native screenshot readiness: that additional failed capture is retained as an error. A single normal wake click allowed capture. The positive image shows the normal OS lock screen, not unlocked app control; no OS login, TCC reset, sleep-policy change, or security bypass was used for this final verification. The task-only worker is reclaimed and its session archived; the retained host remains available.

The same mismatch was observed before the fix. Regression tests drive actual adapted tools through a real session loop and reopen its SQLite transcript: returned errors, execution and preparation throws, missing tools, successful results, completed nonzero commands, explicit recovery, and middleware-added failure, with and without plugin handlers. The original source fails both variants. The explicit-recovery and middleware-combination defects found during review were separately reproduced and fixed.

  • 98 focused tests pass across seven files; scoped lint passes.
  • Full clean-machine changed gate passed on 9c1ba9ec5a4b231ea290afe4829d0c1980b7c64f (760.603 seconds). Environment run. The refresh preserves the identical three-file patch (stable patch ID c0a90acc3bcf3719fd4c3a07a3d3d1efc4142287).
  • Exact-head CI passed. Earlier checks caught and repaired test typing/lint issues. The first refreshed CI encountered an unrelated main history fixture; this branch consumes its already-landed repair from fix(gateway): align history test fixture with projection state #139794 rather than duplicating it.
  • Independent reviews are clean through P2 after addressing recovery semantics. Latest ClawSweeper review on the final head has no actionable findings or before-merge requests.
  • Canonical runtime and Control UI build passed, followed by supervised Gateway restart. Configuration and installed service definition remained byte-identical; the offline profile backup was verified before the update.

Production delta: +6 lines across existing lifecycle owners; tests: +152. This connects the existing classifier at the owning boundaries and removes a redundant no-result branch. No new abstraction or persisted state is introduced. No visual layout changes. Cloud startup latency and macOS display availability remain separate follow-ups, not claims of this repair.

@clawsweeper

clawsweeper Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S maintainer Maintainer-authored PR labels Sep 6, 2026
@clawsweeper clawsweeper Bot added P2 Normal backlog priority with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 6, 2026
@clawsweeper

clawsweeper Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed September 6, 2026, 2:00 AM ET / 06:00 UTC.

ClawSweeper review

What this changes

The PR classifies structured tool failures before completion events and transcript storage while preserving explicit plugin recovery.

Merge readiness

✅ Ready for maintainer review

This remains a useful, focused fix: current main still misses structured failures at the session outcome boundary. No introduced correctness or security blocker was found, and the maintainer-authored PR is protected from cleanup closure.

Priority: P2
Reviewed head: 6f45aeec60f9c74dfb6734cc3350bd96a2dba653

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused lifecycle repair with meaningful regression coverage and no supported blocking finding.
Proof confidence 🌊 off-meta tidepool Not applicable: The maintainer-authored exemption applies: the captured body supplies original Mac failure observations and session-boundary regression claims, while after-fix Mac verification remains pending and is not counted as completed proof.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: The maintainer-authored exemption applies: the captured body supplies original Mac failure observations and session-boundary regression claims, while after-fix Mac verification remains pending and is not counted as completed proof.
Evidence reviewed 9 items Pinned introduction and review continuity: The supplied complete introduction contains two production changes and one regression file. Comparison with the previous reviewed head found no changes in those files; the previous completed review recorded no findings.
Current main still needs the repair: Fetched main returns without classification when no tool-result handlers exist and otherwise forwards the incoming isError flag. The adapter converts execution exceptions into resolved results containing details.status=error, establishing the mismatch from source.
Adapter failure representation: executeAdaptedToolOperation catches ordinary execution failures and returns buildToolExecutionErrorResult, whose error identity lives in structured details rather than a thrown exception.
Findings None None.
Security None None.

How this fits together

OpenClaw’s session lifecycle receives tool results and plugin overrides, then produces completion events, saved transcripts, and the next model request. This change makes those consumers receive consistent error metadata.

flowchart TD
  A[Tool execution or preflight] --> B[Structured result]
  B --> C[Session error classification]
  C --> D{Execution started?}
  D -->|Yes| E[Plugin overrides and recovery]
  D -->|No| F[Final outcome]
  E --> F
  F --> G[Completion events]
  F --> H[Transcript and model context]
Loading

Before merge

None.

Agent review details

Security

None.

PR surface

Source +6, Tests +152. Total +158 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 2 14 8 +6
Tests 1 152 0 +152
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 166 8 +158

Review metrics

Metric Value Why it matters
Outcome coverage 8 outcomes × 2 middleware configurations across 3 observable boundaries The regression checks agreement among completion events, model context, and stored transcripts while protecting recovery semantics.

Technical review

Best possible solution:

Keep outcome classification at the shared session lifecycle, with explicit middleware recovery authoritative and one finalized flag feeding events, persistence, and model context.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: an adapted tool exception becomes a structured error result that current main can finalize with isError=false. No runtime reproduction was executed during this read-only review.

Is this the best way to solve the issue?

Yes. Reusing the existing classifier before middleware and at the pre-execution outcome boundary repairs the producer while retaining explicit recovery and completed-command semantics.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against d1dce5857a3a.

Labels

Label justifications:

  • P2: Incorrect tool outcome metadata is a concrete reliability defect with a focused repair and no demonstrated emergency-level impact.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The maintainer-authored exemption applies: the captured body supplies original Mac failure observations and session-boundary regression claims, while after-fix Mac verification remains pending and is not counted as completed proof.

Evidence

What I checked:

  • Pinned introduction and review continuity: The supplied complete introduction contains two production changes and one regression file. Comparison with the previous reviewed head found no changes in those files; the previous completed review recorded no findings. (6f45aeec60f9)
  • Current main still needs the repair: Fetched main returns without classification when no tool-result handlers exist and otherwise forwards the incoming isError flag. The adapter converts execution exceptions into resolved results containing details.status=error, establishing the mismatch from source. (src/agents/sessions/agent-session-base.ts:258, d1dce5857a3a)
  • Adapter failure representation: executeAdaptedToolOperation catches ordinary execution failures and returns buildToolExecutionErrorResult, whose error identity lives in structured details rather than a thrown exception. (src/agents/agent-tool-definition-adapter.ts:256, 6f45aeec60f9)
  • Recovery and finalization ordering: The session classifies executed results before middleware, preserves explicit false overrides, and limits terminal-hook classification to outcomes where execution never started. The runner classifies each returned middleware result rather than reclassifying retained diagnostic details after recovery. (src/agents/sessions/agent-session-base.ts:259, 6f45aeec60f9)
  • Owned runtime boundary: The runtime facade reaches the repository-owned packages/agent-core implementation through src/plugin-sdk/agent-core.ts. Its finalization applies afterToolCall before afterToolOutcome, and both completion events and tool-result messages consume finalized.isError. No changed behavior depends on the Codex runtime contract. (packages/agent-core/src/agent-loop.ts:1328, 6f45aeec60f9)
  • Behavioral regression coverage: The complete new test exercises eight outcomes with and without recovery middleware, asserting completion metadata, next-request context, and typed SQLite transcript reload. It includes preflight failure, adapted exceptions, explicit recovery, middleware-added failure, missing tools, and successful completed nonzero commands. Tests were inspected, not executed. (src/agents/sessions/sdk.tool-outcomes.test.ts:24, 6f45aeec60f9)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • gaoanze888: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (3 earlier review cycles)
  • reviewed 2026-09-06T05:00:16.411Z sha 93d7320 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-06T05:09:13.464Z sha 1dfa291 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-06T05:41:44.188Z sha 9c1ba9e :: needs maintainer review before merge. :: none

@steipete
steipete marked this pull request as ready for review September 6, 2026 05:02
@steipete
steipete merged commit 56e96f6 into main Sep 6, 2026
180 checks passed
@steipete
steipete deleted the codex/worker-tool-error-outcome branch September 6, 2026 06:30
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Sep 7, 2026
* fix: report failed cloud tools as errors

* test: verify tool outcomes through typed transcript reload

* refactor: keep tool outcome proof focused and lint clean
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling maintainer Maintainer-authored PR P2 Normal backlog priority with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant