Skip to content

perf(cron): reuse normalized history projection - #140170

Merged
steipete merged 1 commit into
mainfrom
codex/perf52-cron-projection
Sep 6, 2026
Merged

steipete merged 1 commit into
mainfrom
codex/perf52-cron-projection

Conversation

@steipete

@steipete steipete commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Automation history requests incur avoidable per-record overhead as the number of stored runs grows.

Why This Change Was Made

Use the existing full schema as the single status normalizer, then materialize the five legacy optional fields on its private result instead of copying it again. Input tasks, field exclusion and insertion order, authored-detail precedence, recovery decisions, and all database/configuration/concurrency behavior remain unchanged. Production −2 lines; the existing wire-record test expands by 13 lines.

User Impact

Synthetic 50/500/2,000-record projector batches improved 45–57% in both orders. The real history-page caller, including SQLite reading/hydration, improved in 23 of 24 nonempty comparisons; the remaining 2,000-row ascending case was 4.06% slower forward and 33.84% faster reverse. This does not claim faster whole-Gateway turns or a consistent memory reduction.

Evidence

  • 55/55 tests passed across the complete history, event-codec, and task-run recovery files; all 29 normal changed checks passed. Managed P2 review was scoped-clean, with no findings.
  • Fixed B0/C0/C1/B1 fresh-process cohort on ce7eb85ae413282d0d46b2b661d11788d5fde671: 20 rows per phase, three warm calls and 12 fixed samples per row. All 5,040 full wrapper returns (240 warm + 4,800 timed) matched, including undefined-valued own fields, insertion order, and reference graph. No retries or favorable subset selection.
  • The table includes every row and all 7 adverse comparisons out of 40. Six adverse comparisons are empty controls. Percentages compare candidate/baseline; negative means faster.
Rows Actual operation Forward Reverse
0 Projector batch +4.75% -25.88%
0 History asc +3.78% +3.48%
0 History desc -2.95% +5.03%
0 History desc; status=error +7.85% -1.73%
0 History asc; query=needle -2.14% +3.76%
50 Projector batch -45.12% -49.59%
50 History asc -27.52% -26.59%
50 History desc -27.55% -23.06%
50 History desc; status=error -30.89% -27.07%
50 History asc; query=needle -31.88% -27.50%
500 Projector batch -56.66% -53.70%
500 History asc -33.65% -31.81%
500 History desc -16.89% -28.62%
500 History desc; status=error -19.45% -32.38%
500 History asc; query=needle -31.57% -34.11%
2000 Projector batch -52.65% -55.26%
2000 History asc +4.06% -33.84%
2000 History desc -26.46% -32.12%
2000 History desc; status=error -45.43% -33.17%
2000 History asc; query=needle -32.28% -33.64%

Across the same 12 sample means per row, 7/40 mean comparisons and 8/40 maximum comparisons were slower. These are maxima of observed sample means, not population-tail estimates. Notable adverse observations: 50-row descending history maxima +5.45%/+4.87%; 2,000-row descending history forward maximum +40.37%; empty status-filtered history means +56.77%/+51.58% and maxima +367.77%/+370.99%. Every underlying sample remains retained.

Full-process wall improved 12.83%/11.32%, but includes imports, fixture creation, assertions and evidence writing. Forward kernel/sample-tree RSS rose 0.88%/1.10%; system CPU rose 8.85%/10.25% across the two orders. No memory or whole-Gateway improvement is claimed. All children closed and the exact candidate was restored. Independent retained-data audit passed: it decoded all 5,040 returns from 115,009,918 bytes, verified every journal ordinal and timing sum, and independently checked complete values, own-key order and bidirectional reference relationships.

@clawsweeper

clawsweeper Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@openclaw-barnacle openclaw-barnacle Bot added size: XS maintainer Maintainer-authored PR labels Sep 6, 2026
@steipete
steipete marked this pull request as ready for review September 6, 2026 14:32
@clawsweeper clawsweeper Bot added P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 6, 2026
@clawsweeper

clawsweeper Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed September 6, 2026, 10:39 AM ET / 14:39 UTC.

ClawSweeper review

What this changes

Reduces automation-history processing overhead by reusing the normalized result and expands coverage for status normalization, immutable inputs, and legacy optional fields.

Merge readiness

✅ Ready for maintainer review

The optimization remains useful: fetched main and the latest release retain the redundant work. No actionable correctness or security defect was found, and this maintainer-authored PR is protected from automatic closure.

Priority: P2
Reviewed head: 1277ac1f7438e5068391695cb4bd5cf30ccf4ed1

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused optimization with meaningful compatibility coverage and candid production-caller benchmark results.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (live_output): The supplied benchmark results exercise the changed projection through the real SQLite-backed history reader in fresh processes, reporting equivalent complete returns and mostly improved nonempty timings; raw samples were not independently rerun.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The supplied benchmark results exercise the changed projection through the real SQLite-backed history reader in fresh processes, reporting equivalent complete returns and mostly improved nonempty timings; raw samples were not independently rerun.
Evidence reviewed 8 items Introduced change verified: The merge-base-to-head diff matches the supplied introduction evidence: only the history projection and its existing compatibility test change.
Normalization and object ownership: The full schema already accepts ok/error/skipped and maps invalid or missing status to undefined. The parser constructs a fresh allowlisted entry, so assigning the five fixed optional fields cannot mutate the input task or expose additional detail fields.
Production caller: The history reader projects SQLite task rows before status filtering, sorting, and pagination; recovery also consumes the projected status. Neither caller's decisions nor the stored representation change.
Findings None None.
Security None None.

How this fits together

Automation history reads stored task records from SQLite and converts them into run-history responses. The same conversion also supplies stored outcomes to execution recovery.

flowchart LR
  A[Stored task records] --> B[History reader]
  B --> C[Normalize and allowlist fields]
  C --> D[Materialize legacy optional fields]
  D --> E[Filter and paginate history]
  E --> F[CLI and Gateway responses]
  C --> G[Stored outcome recovery]
Loading

Before merge

None.

Agent review details

Security

None.

PR surface

Source -2, Tests +13. Total +11 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 3 5 -2
Tests 1 15 2 +13
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 18 7 +11

Review metrics

Metric Value Why it matters
Reported history benchmark 23 of 24 nonempty comparisons improved The measured caller includes SQLite reading and hydration, supporting a practical benefit beyond isolated projection timing.

Technical review

Best possible solution:

Keep one status-normalization path and reuse its private result while preserving the existing history and recovery contracts.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a functional bug reproduction: this is a performance optimization, supported by source-visible redundant work and reported baseline/candidate benchmarks.

Is this the best way to solve the issue?

Yes. Reusing the existing normalizer and its fresh result is a narrow solution that preserves field exclusion, input immutability, and legacy output properties.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against b5b64a8d5133.

Labels

Label changes:

  • add P2: This is a bounded performance improvement to automation-history reads without evidence of an urgent user-facing failure.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The supplied benchmark results exercise the changed projection through the real SQLite-backed history reader in fresh processes, reporting equivalent complete returns and mostly improved nonempty timings; raw samples were not independently rerun.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The supplied benchmark results exercise the changed projection through the real SQLite-backed history reader in fresh processes, reporting equivalent complete returns and mostly improved nonempty timings; raw samples were not independently rerun.

Label justifications:

  • P2: This is a bounded performance improvement to automation-history reads without evidence of an urgent user-facing failure.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The supplied benchmark results exercise the changed projection through the real SQLite-backed history reader in fresh processes, reporting equivalent complete returns and mostly improved nonempty timings; raw samples were not independently rerun.
  • proof: sufficient: Contributor real behavior proof is sufficient. The supplied benchmark results exercise the changed projection through the real SQLite-backed history reader in fresh processes, reporting equivalent complete returns and mostly improved nonempty timings; raw samples were not independently rerun.

Evidence

What I checked:

  • Introduced change verified: The merge-base-to-head diff matches the supplied introduction evidence: only the history projection and its existing compatibility test change. (src/cron/task-run-detail.ts:341, 1277ac1f7438)
  • Normalization and object ownership: The full schema already accepts ok/error/skipped and maps invalid or missing status to undefined. The parser constructs a fresh allowlisted entry, so assigning the five fixed optional fields cannot mutate the input task or expose additional detail fields. (src/cron/task-run-detail.ts:139, 1277ac1f7438)
  • Production caller: The history reader projects SQLite task rows before status filtering, sorting, and pagination; recovery also consumes the projected status. Neither caller's decisions nor the stored representation change. (src/cron/task-run-history.ts:123, 1277ac1f7438)
  • Compatibility coverage: The expanded existing test covers six status inputs, freezes the task and detail, checks all five legacy own properties, and retains internal-field exclusion assertions. These protect observable contracts without adding a production test seam. Tests were inspected, not executed during this read-only review. (src/cron/task-run-history.test.ts:476, 1277ac1f7438)
  • Current-main necessity: Fetched main still performs the separate status check and spreads the parsed entry into another object; the central optimization is not already present. (src/cron/task-run-detail.ts:331, b5b64a8d5133)
  • Latest-release comparison: The v2026.9.2 source also retains both redundant operations. This PR preserves its legacy optional-field behavior. (src/cron/task-run-detail.ts, 3928bad9badf)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit affda2e into main Sep 6, 2026
248 checks passed
@steipete
steipete deleted the codex/perf52-cron-projection branch September 6, 2026 14:47
@steipete

steipete commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer Maintainer-authored PR P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: XS status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant