Skip to content

feat(engineering): add agent-run-forensics skill - #1016

Open
xizhuomengcontin wants to merge 1 commit into
alirezarezvani:devfrom
xizhuomengcontin:feature/agent-run-forensics
Open

xizhuomengcontin wants to merge 1 commit into
alirezarezvani:devfrom
xizhuomengcontin:feature/agent-run-forensics

Conversation

@xizhuomengcontin

Copy link
Copy Markdown

Adds one skill, engineering/agent-run-forensics/.

What it does

It gives the agent one rule and the tooling to keep it: when a question is about something that
already happened, read the trace before answering
— do not reconstruct it from memory.

Agents are confidently wrong about their own history. A session summary written from memory is
missing the tool results, the exit codes, and the files that changed without anyone mentioning
them, so "why did you overwrite that file" gets a fluent answer rather than a true one. This skill
has the agent open the recording of the run, walk the causal chain to the step in question, and
answer from what is actually in it — and, where it helps, replay that run offline so the failure
happens again in front of you.

It is the forensic counterpart to zero-hallucination-coder: that one stops the agent inventing
facts about code it has not read, this one stops it inventing facts about work it has already done.

The discipline I care most about is in step 3: the trace labels every causal edge recorded (the
recorder watched it happen) or inferred (derived at query time from a named rule), and the skill
requires the agent to keep those apart in its answer instead of collapsing both into one confident
sentence.

Conventions checklist

  • Targets dev.
  • Frontmatter is exactly name and description — no license, metadata, version,
    author, category or anything else.
  • 184 lines, well under the 500 limit.
  • Opinionated and actionable (specific tools, specific flags, a table for which tool answers
    "what" vs "why").
  • Anti-patterns section included — seven of them, each naming a concrete failure.
  • Cross-references zero-hallucination-coder, agent-harness, handoff, llm-cost-optimizer
    and chaos-engineering.
  • No scripts/, references/ or assets/ — the skill is self-contained.

Two things I want to flag rather than have you find

1. Layout. CONVENTIONS.md documents <domain>/<skill-name>/SKILL.md, so that is what I used.
Looking at dev, though, 50 of the 94 skills under engineering/ actually live at
<domain>/<name>/skills/<name>/SKILL.md with a .claude-plugin/plugin.json and a README.md
alongside, and 38 more use a third shape. Tell me which one you want and I will restructure in this
PR — I did not want to guess against your own written convention.

Related: existing skills do not cross-link with relative paths, so I wrote the cross-references as
plain skill names in backticks rather than inventing a link format.

2. Disclosure. The skill drives an external tool — the orcareplay npm package and its MCP
server — and I maintain that tool. So, plainly:

  • It is Apache-2.0, installs with npm i -g orcareplay, and runs entirely locally. No account, no
    hosted service, no metered tier, and nothing is sent anywhere. The prerequisite block in the
    skill says exactly what is needed.
  • The skill's honest limits are in the body, not buried: replay blocks model-provider egress, not
    the network
    (recorded shell commands execute for real, so it is not a sandbox); replay is
    destructive in-place unless worktree: true is passed; and a matching replay is not a
    determinism result
    , because the model is not re-asked.
  • If a vendor-dependent skill is not something you want in the library regardless of licence, that
    is a fair call and I will close this myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants