fix(compaction): count nested tool result content by LZY3538 · Pull Request #99391 · openclaw/openclaw

LZY3538 · 2026-07-03T06:12:02Z

What Problem This Solves

Fixes an issue where users in tool-heavy Codex sessions could see compaction repeatedly retain the full transcript when recent context was dominated by Codex tool-result blocks. Those blocks were preserved in transcript content as nested toolResult payloads, but the compaction estimator did not count their text, so the cut point could collapse back to the start of the transcript.

Why This Change Was Made

The shared compaction estimator now counts nested toolResult/tool_result text payloads in the same content-block pass that already accounts for text and image blocks. The compaction summary serializer now preserves the same nested tool-result payloads exactly once before summarized history is dropped, using text first and falling back to string content only when text is absent.

Dependency contract checked against OpenAI Codex protocol sources in sibling ../codex: codex-rs/protocol/src/items.rs for tool-call/result item shapes and codex-rs/protocol/src/dynamic_tools.rs for dynamic tool output content items.

User Impact

Tool-heavy Codex conversations can choose a real compaction cut point instead of no-oping when the newest retained transcript contains nested tool-result output. When older Codex tool-result output is compacted into the replacement summary, that payload remains available to the summarizer instead of being silently omitted.

Evidence

Red first: node scripts/run-vitest.mjs packages/agent-core/src/harness/compaction/compaction-trailing-toolresult.test.ts failed before the estimator implementation because the new nested toolResult case estimated 0 tokens and retained entry index 0.
Red first for review fix: node scripts/run-vitest.mjs packages/agent-core/src/harness/compaction/utils.test.ts failed before the serializer implementation because serializeConversation returned an empty string for Codex nested toolResult content.
Runtime compaction proof with prepareCompaction + compact fake stream captured the actual summarization prompt:

{
  "firstKeptEntryId": "entry-3",
  "summarizedRoles": ["user", "assistant", "toolResult"],
  "summarizedToolPayloadChars": 120039,
  "promptContainsMarker": true,
  "markerOccurrencesInPrompt": 1,
  "compactOk": true,
  "summary": "summary from captured prompt"
}

node scripts/run-vitest.mjs packages/agent-core/src/harness/compaction/utils.test.ts packages/agent-core/src/harness/compaction/compaction.test.ts packages/agent-core/src/harness/compaction/compaction-trailing-toolresult.test.ts packages/agent-core/src/harness/compaction/compaction-image-tokens.test.ts src/agents/embedded-agent-runner/transcript-file-state.test.ts (5 files, 35 tests passed)
pnpm tsgo:core
pnpm tsgo:core:test
pnpm exec oxfmt --check packages/agent-core/src/harness/compaction/utils.ts packages/agent-core/src/harness/compaction/utils.test.ts packages/agent-core/src/harness/compaction/compaction.ts packages/agent-core/src/harness/compaction/compaction-trailing-toolresult.test.ts
git diff --check

Known local proof gap: node scripts/run-vitest.mjs extensions/codex/src/app-server/run-attempt.context-engine.test.ts was attempted earlier, but this local Windows worktree does not have the codex-context-test-sandbox backend registered. The run failed before exercising this compaction change, with turn/start not observed and an unhandled sandbox backend registration error.

clawsweeper · 2026-07-03T06:14:20Z

Codex review: needs maintainer review before merge. Reviewed July 3, 2026, 4:53 AM ET / 08:53 UTC.

Summary
The PR updates agent-core compaction token estimation and summary serialization to count nested toolResult/tool_result text payloads, with focused regression tests.

PR surface: Source +18, Tests +80. Total +98 across 4 files.

Reproducibility: yes. Current main creates nested Codex toolResult transcript blocks while the estimator and summary serializer ignore those nested payloads; the focused regression shape can reproduce the zero-token estimate and summary-prompt omission from source.

Review metrics: none identified.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #99375
Summary: This PR is a candidate fix for the focused nested tool-result compaction estimator and serializer issue; the broader Slack/context-ceiling issue overlaps but includes additional product behavior.

Members:

canonical: bug(compaction): toolResult blocks estimate to 0 tokens — cut point collapses, compaction permanently no-ops (safeguard futile loop) #99375 - The PR body uses closing syntax for this focused bug, and the changed compaction paths match the reported failure.
same_root_cause: fix(compaction): count nested toolResult block payloads in estimateTokens #99386 - This open sibling PR targets the same nested tool-result compaction bug but review context shows a less clean landing path.
superseded: fix(compaction): count toolResult blocks in content char estimator #99430 - That sibling PR was reviewed as superseded by this cleaner proof-positive branch for the same focused compaction bug.
partial_overlap: bug(compaction): safeguard mode allows sessions to grow to context ceiling — wedge causes "Something went wrong" with no in-channel recovery on Slack; cost in one session #90639 - The broader report includes related context growth symptoms plus safeguard threshold, Slack recovery, and product-policy work outside this narrow fix.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Risk before merge

[P1] This changes session compaction accounting and summary serialization for transcript entries that may be dropped after summarization, so maintainers should treat it as session-state sensitive even with focused tests.
[P1] The runtime proof exercises production compaction functions with a fake stream, not a full live Codex gateway session; maintainers may still request live proof before landing.
[P1] The PR should close the focused nested tool-result bug only; broader safeguard threshold and Slack recovery behavior remain in bug(compaction): safeguard mode allows sessions to grow to context ceiling — wedge causes "Something went wrong" with no in-channel recovery on Slack; cost in one session #90639.

Maintainer options:

Land the focused compaction fix after normal checks (recommended)
Maintainers can accept the remaining proof-scope risk because the patch covers both cut-point estimation and summary serialization for the focused nested tool-result bug.
Request live Codex gateway proof first
Before merge, ask for redacted live or packaged Codex proof showing compaction chooses a real cut and preserves a nested tool-result marker in the generated summary path.
Consolidate with a sibling branch instead
If a sibling PR is chosen as the landing branch, require it to include the same estimator and serializer repair before closing this PR as superseded.

Next step before merge

No automated repair is selected because there is no discrete code defect; maintainers should decide whether the focused proof is enough to land this session-state change.

Security
Cleared: The diff only changes TypeScript compaction logic and focused Vitest coverage; it does not touch dependencies, CI, package scripts, secrets, permissions, or code-execution boundaries.

Review details

Best possible solution:

Land one focused compaction fix that counts and summarizes nested Codex tool-result payloads exactly once, while keeping broader safeguard and Slack recovery work separate.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main creates nested Codex toolResult transcript blocks while the estimator and summary serializer ignore those nested payloads; the focused regression shape can reproduce the zero-token estimate and summary-prompt omission from source.

Is this the best way to solve the issue?

Yes. The shared agent-core compaction estimator and summary serializer are the narrow owner boundary; changing only Codex projection would leave the generic accounting and summary asymmetry in place.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 1fef99962edf.

Label changes

Label justifications:

P1: The PR targets a source-reproducible compaction bug that can leave tool-heavy Codex sessions repeatedly retaining transcript history and risking context overflow.
merge-risk: 🚨 session-state: The diff changes compaction accounting and summary serialization for transcript entries that may be dropped from session history.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal-style runtime proof from production compaction functions showing a real cut point and the nested tool-result marker included once in the captured summary prompt.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal-style runtime proof from production compaction functions showing a real cut point and the nested tool-result marker included once in the captured summary prompt.

Evidence reviewed

PR surface:

Source +18, Tests +80. Total +98 across 4 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	2	26	8	+18
Tests	2	81	1	+80
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	4	107	9	+98

What I checked:

Repository policy read and applied: Root AGENTS.md was read fully; its whole-path review, session-state merge-risk, and direct Codex source inspection requirements apply to this Codex compaction patch. (AGENTS.md:15, 1fef99962edf)
No scoped package policy found: No scoped packages/**/AGENTS.md file was present for the touched package path, and no matching maintainer-note file was found.
Current main estimator gap: On current main, countContentBlockChars counts only text and image blocks, and estimateTokens delegates array-valued toolResult content to that helper, so nested type: "toolResult" payloads contribute zero. (packages/agent-core/src/harness/compaction/compaction.ts:249, 1fef99962edf)
Current main serializer gap: On current main, serializeConversation filters tool-result message content down to plain type: "text" blocks, so nested Codex tool-result payloads are omitted from compaction summary prompts. (packages/agent-core/src/harness/compaction/utils.ts:156, 1fef99962edf)
Codex app-server creates the affected shape: The Codex app-server projector creates persisted role: "toolResult" messages with nested type: "toolResult" blocks carrying both content and text strings. (extensions/codex/src/app-server/event-projector.ts:1918, 1fef99962edf)
PR head unifies estimator and serializer extraction: The PR head imports getCompactionContentBlockText into compaction.ts and uses the same helper in serializeConversation, so counted nested tool-result text is also included in the summary prompt exactly once. (packages/agent-core/src/harness/compaction/utils.ts:109, 590432751417)

Likely related people:

yetval: Authored recent merged fixes for the same compaction cut-point accounting family, including oversized trailing tool results and image-block token accounting. (role: recent compaction estimator contributor; confidence: high; commits: f826a665a217, 2ece2945ae67; files: packages/agent-core/src/harness/compaction/compaction.ts, packages/agent-core/src/harness/compaction/compaction-trailing-toolresult.test.ts, packages/agent-core/src/harness/compaction/compaction-image-tokens.test.ts)
steipete: Carried the agent-core extraction that established this package boundary and added the PR follow-up commit unifying estimator and serializer extraction. (role: agent runtime refactor contributor and recent branch maintainer; confidence: high; commits: bb46b79d3c14, 590432751417; files: packages/agent-core/src/harness/compaction/compaction.ts, packages/agent-core/src/harness/compaction/utils.ts)
omarshahine: Recently touched the Codex event-projector path adjacent to the nested tool-result transcript shape that this PR accounts for. (role: recent Codex projector contributor; confidence: medium; commits: 9b9a124cc520; files: extensions/codex/src/app-server/event-projector.ts)
vincentkoc: Recent Codex app-server history intersects the projection and transcript-shape area used as the source shape for this compaction fix. (role: adjacent Codex/runtime contributor; confidence: medium; commits: 2b75806197ab; files: extensions/codex/src/app-server/event-projector.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

LZY3538 · 2026-07-03T07:00:19Z

@clawsweeper re-review

clawsweeper · 2026-07-03T07:00:22Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

steipete · 2026-07-03T09:21:21Z

Land-ready verification for exact head 59043275141710ff6516d956c9b1d8cecda1f61f:

Compaction now uses one canonical block-text extractor for both token estimation and summary serialization, covering nested toolResult and tool_result shapes without double-counting duplicate text/content.
Direct Codex contract check: dynamic tool output is text-bearing in ../codex/codex-rs/protocol/src/dynamic_tools.rs:58-72 and carried on dynamic tool items in ../codex/codex-rs/protocol/src/items.rs:197-213; OpenClaw mirrors it as nested toolResult content in extensions/codex/src/app-server/event-projector.ts:1918-1939.
Focused compaction coverage — 10 passed; transcript file-state — 25 passed; Codex run-attempt — 116 passed; Codex context-engine — 29 passed.
Production prepareCompaction + compact probe — a 120,027-character nested result survived into the split-turn summarization prompt exactly once and compaction completed.
Final local and full-branch structured autoreviews — clean.

Proof gap: the production compactor used an injected deterministic summary stream rather than a paid external model call, allowing exact prompt inspection. No screenshot is useful for transcript accounting. Thanks @LZY3538 for the fix.

steipete · 2026-07-03T09:21:38Z

Merged via squash.

Prepared head SHA: 59043275141710ff6516d956c9b1d8cecda1f61f
Landed commit: 413acc2e634dd977383a68597f7fc99f322b206f

* fix(compaction): count nested tool result content * fix(agent-core): unify compaction block extraction --------- Co-authored-by: Peter Steinberger <steipete@gmail.com>

openclaw-barnacle Bot added size: S triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jul 3, 2026

clawsweeper Bot mentioned this pull request Jul 3, 2026

fix(compaction): count nested toolResult block payloads in estimateTokens #99386

Closed

openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jul 3, 2026

clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 3, 2026

LZY3538 force-pushed the codex/99375-toolresult-compaction branch from 506e5b0 to 3c8d6f8 Compare July 3, 2026 06:25

clawsweeper Bot added P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jul 3, 2026

LZY3538 force-pushed the codex/99375-toolresult-compaction branch from 3c8d6f8 to d6016f0 Compare July 3, 2026 06:57

steipete self-assigned this Jul 3, 2026

clawsweeper Bot mentioned this pull request Jul 3, 2026

fix(compaction): count toolResult blocks in content char estimator #99430

Closed

LZY3538 and others added 2 commits July 3, 2026 09:45

fix(compaction): count nested tool result content

0a595f4

fix(agent-core): unify compaction block extraction

5904327

steipete force-pushed the codex/99375-toolresult-compaction branch from d6016f0 to 5904327 Compare July 3, 2026 08:45

steipete merged commit 413acc2 into openclaw:main Jul 3, 2026
80 checks passed

clawsweeper Bot mentioned this pull request Jul 3, 2026

Prompt history is not append-only mid-session → Anthropic prompt-cache thrash (two reproducible invalidators) #99495

Closed

github-actions Bot mentioned this pull request Jul 3, 2026

📡 Upstream Digest — 2026-07-03 10:27 UTC curtismercier/openclaw-mods#1196

Open

BillBOT33 mentioned this pull request Jul 6, 2026

compaction: delivery-mirror usage poisons tokensBefore→null; assembler rejects the record, so sessions never truncate (safeguard thrash) #100982

Closed

clawsweeper Bot mentioned this pull request Jul 9, 2026

fix(compaction): use truncateUtf16Safe for compaction summary truncation #102538

Closed

clawsweeper Bot mentioned this pull request Jul 17, 2026

fix(compaction): retain CJK sessions within token budget #109824

Closed

Navigation Menu

Uh oh!

fix(compaction): count nested tool result content - #99391

fix(compaction): count nested tool result content#99391
steipete merged 2 commits into
openclaw:mainfrom
LZY3538:codex/99375-toolresult-compaction

LZY3538 commented Jul 3, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 3, 2026 •

edited

Loading

Uh oh!

LZY3538 commented Jul 3, 2026

Uh oh!

clawsweeper Bot commented Jul 3, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 3, 2026

Uh oh!

Uh oh!

steipete commented Jul 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

LZY3538 commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LZY3538 commented Jul 3, 2026

Uh oh!

clawsweeper Bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 3, 2026

Uh oh!

Uh oh!

steipete commented Jul 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

LZY3538 commented Jul 3, 2026 •

edited

Loading

clawsweeper Bot commented Jul 3, 2026 •

edited

Loading

clawsweeper Bot commented Jul 3, 2026 •

edited

Loading