Skip to content

improve(skills): review retained conversation for durable learning - #139656

Merged
obviyus merged 1 commit into
mainfrom
codex/skill-review-history-prompt
Sep 6, 2026
Merged

obviyus merged 1 commit into
mainfrom
codex/skill-review-history-prompt

Conversation

@obviyus

@obviyus obviyus commented Sep 6, 2026 •

Copy link
Copy Markdown
Collaborator

What Problem This Solves

Skill Workshop receives the retained conversation, but its experience prompt limited review to the latest turn, separating earlier corrections from later evidence that they worked.

Why This Change Was Made

The prompt connects requirements, corrections, attempts, and observed results across retained history. Duplicate patch instructions and an unused input are removed. The existing trigger, saved-transcript boundary, provider selection, permissions, and proposal/apply flow remain unchanged.

User Impact

Earlier lessons remain eligible after a routine final exchange. Review still abstains for covered lessons, routine work, unsupported guesses, and private facts, and can stage at most one proposal. No write access is added.

Evidence

All 40 focused scheduling, prompt, authoring, and real Workshop-tool tests passed. Tool tests retained read-before-write, pending-only mutation, stale-read rejection, and the single-mutation limit. Formatting, lint, and changed-page checks passed; full anchor checking found unrelated missing external pages. Seven synthetic multi-turn scenarios produced the expected proposals or abstentions with tool execution denied. The old prompt also captured the earlier correction in a paired check, so this clarifies instruction scope without claiming improved learning rate or a deterministic missed capture.

@clawsweeper

clawsweeper Bot commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation size: S maintainer Maintainer-authored PR labels Sep 6, 2026
@clawsweeper clawsweeper Bot added P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 6, 2026
@clawsweeper

clawsweeper Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed September 5, 2026, 11:05 PM ET / September 6, 2026, 03:05 UTC.

ClawSweeper review

What this changes

The PR asks Skill Workshop to learn from the full retained conversation, removes repeated prompt mechanics and prose-pinning assertions, and corrects collection-review documentation.

Merge readiness

✅ Ready for maintainer review

Keep open: the clarification remains useful against inspected main, and no actionable introduced defect was found. The maintainer-authored, protected PR is not a cleanup-close candidate.

Priority: P3
Reviewed head: e93557b7543254f02a16fc2f6699c6cb2c9b531c

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused, coherent clarification with retained runtime safeguards and appropriately limited validation claims.
Proof confidence 🌊 off-meta tidepool Not applicable: The repository-state MEMBER association exempts this PR from the external-contributor gate. Reported real-provider trials exercise prompt decisions with tools denied; they do not prove proposal application or improved learning rates, neither of which this diff claims.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: The repository-state MEMBER association exempts this PR from the external-contributor gate. Reported real-provider trials exercise prompt decisions with tools denied; they do not prove proposal application or improved learning rates, neither of which this diff claims.
Evidence reviewed 8 items Introduced scope: The complete host-pinned introduction patch contains only the prompt, two related test files, and self-learning documentation. The verified test-merge comparison preserves unrelated base changes.
Still useful against main: Inspected main still instructs the reviewer to consider only the last turn. The branch replaces that instruction without adding another review mechanism. Latest-release source inspection could not complete because its blob was unavailable locally and network resolution failed; no shipped-fix claim is made.
Existing runtime boundary: The production runner already opens model context through the saved boundary, validates the source session, allocates one mutation, and allows only skill_workshop execution. These controls are unchanged.
Findings None None.
Security None None.

How this fits together

Skill Workshop reviews saved conversation context after eligible foreground work and can stage one reusable-skill proposal. The existing configured pipeline controls whether that proposal is applied.

flowchart TD
  A[Eligible completed conversation] --> B[Saved transcript boundary]
  B --> C[Detached Workshop review]
  D[Existing skills and usage] --> C
  C --> E{New durable learning?}
  E -->|No| F[No change]
  E -->|Yes| G[One pending proposal]
  G --> H[Configured apply pipeline]
Loading

Before merge

None.

Agent review details

Security

None.

PR surface

Source -12, Tests -38, Docs -6. Total -56 across 4 files.

View PR surface stats
Area Files Added Removed Net
Source 1 9 21 -12
Tests 2 4 42 -38
Docs 1 11 17 -6
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 4 24 80 -56

Review metrics

None.

Technical review

Best possible solution:

Keep learning scope aligned with the retained conversation while preserving the existing bounded, proposal-only review pipeline.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a deterministic bug reproduction: source confirms the instruction mismatch, while the reported paired trial succeeded with both prompts.

Is this the best way to solve the issue?

Yes: updating the existing review instruction and deferring patch mechanics to the tool contract is a narrow solution that preserves runtime enforcement.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 77ee04bad2b0.

Labels

Label changes:

  • add P3: This is a bounded prompt clarification and documentation correction without an established runtime regression.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The repository-state MEMBER association exempts this PR from the external-contributor gate. Reported real-provider trials exercise prompt decisions with tools denied; they do not prove proposal application or improved learning rates, neither of which this diff claims.

Label justifications:

  • P3: This is a bounded prompt clarification and documentation correction without an established runtime regression.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The repository-state MEMBER association exempts this PR from the external-contributor gate. Reported real-provider trials exercise prompt decisions with tools denied; they do not prove proposal application or improved learning rates, neither of which this diff claims.

Evidence

What I checked:

  • Introduced scope: The complete host-pinned introduction patch contains only the prompt, two related test files, and self-learning documentation. The verified test-merge comparison preserves unrelated base changes. (src/skills/workshop/experience-review-prompt.ts:100, e93557b75432)
  • Still useful against main: Inspected main still instructs the reviewer to consider only the last turn. The branch replaces that instruction without adding another review mechanism. Latest-release source inspection could not complete because its blob was unavailable locally and network resolution failed; no shipped-fix claim is made. (src/skills/workshop/experience-review-prompt.ts:110, 77ee04bad2b0)
  • Existing runtime boundary: The production runner already opens model context through the saved boundary, validates the source session, allocates one mutation, and allows only skill_workshop execution. These controls are unchanged. (src/skills/workshop/experience-review.ts:149, e93557b75432)
  • Implementation ownership: The review runner explicitly selects the OpenClaw harness and proposal-only mode. No changed dependency contract or Codex-specific compatibility claim establishes a sibling-Codex inspection requirement. (src/skills/workshop/review-run.ts:36, e93557b75432)
  • Retained executable safeguards: Remaining tool-boundary tests exercise forbidden apply/evaluate operations, the one-mutation budget, read-before-update, and pending updates. Tool schema descriptions retain exact-span preparation mechanics removed from the review prompt. (src/agents/tools/skill-workshop-tool.review.test.ts:64, e93557b75432)
  • Documentation correction: The canonical Workshop guide describes weekly isolated cron turns, normal file editing, automation history, and no collection-wide rollback. The inspected cron monitor implements that existing behavior; this PR changes its description, not storage semantics. Public docs: docs/tools/skill-workshop.md. (docs/tools/skill-workshop.md:109, e93557b75432)

Likely related people:

  • obviyus: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Peter Steinberger: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@obviyus
obviyus merged commit 96feee2 into main Sep 6, 2026
183 checks passed
@obviyus
obviyus deleted the codex/skill-review-history-prompt branch September 6, 2026 03:09
@obviyus

obviyus commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

Landed in 96feee2da216.

  • Change: the experience-review prompt now connects earlier requirements and corrections with results across the retained conversation. Eligibility, transcript checkpoints, permissions, and proposal/apply behavior are unchanged.
  • Review: exact-head local ClawSweeper passed at Platinum with no findings, security concerns, or requested changes. The GitHub review agrees.
  • Proof: 40 focused tests and seven local model-backed prompt scenarios passed, with independent source and semantic review. The baseline also passed its paired case; this is a clearer instruction, not a measured learning-rate gain or automatic-apply proof.
  • CI: exact-head run 34007923580 passed. No red-CI exception or rebase was needed.
  • Size: net −12 production, −38 test, and −6 documentation lines: −56 total.

Merged the reviewed head through the guarded landing workflow with attribution verified.

vincentkoc added a commit that referenced this pull request Sep 6, 2026
* commit 'a2886546df92c3a601db538b1b70125e3519a16f':
  test(reports): deduplicate native process census setup (#139596)
  fix(ui): avoid refetching unrelated catalog hosts (#139570)
  test(compaction): consolidate classification proof at its owner (#139564)
  fix(apple): harden native state file permissions (#139623)
  improve(skills): review retained conversation for durable learning (#139656)
  fix(docker): stage bundled plugin runtime dependencies under the packaged plugin root (#139117)
  test: close HTTP fixtures before destroying their sockets (#139595)
  refactor(agents): simplify attempt execution and recovery (#139597)
  test(gateway): share HTTP authority scenarios (#139611)
  test(gateway): distinguish final CLI results from draft text (#139605)
  fix(shell-env): preserve CLI terminal ownership during login imports (#139308)
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Sep 6, 2026
…penclaw#139656)

Clarify experience review's evidence scope to include the full retained conversation. Consolidate duplicate instructions without changing eligibility, transcript checkpoints, permissions, or the proposal/apply pipeline.

Remove obsolete prompt fixtures and correct stale collection-review documentation. Production net -12 lines; total net -56 lines. Verified with 40 focused tests and seven local model-backed prompt scenarios, with independent source and semantic review. The paired baseline also passed; this is not a measured learning-rate improvement.

Co-authored-by: Ayaan Zaidi <hi@obviy.us>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation maintainer Maintainer-authored PR P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant