feat(engineering): add agent-run-forensics skill - #1016
Open
xizhuomengcontin wants to merge 1 commit into
Open
xizhuomengcontin wants to merge 1 commit into
xizhuomengcontin wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds one skill,
engineering/agent-run-forensics/.What it does
It gives the agent one rule and the tooling to keep it: when a question is about something that
already happened, read the trace before answering — do not reconstruct it from memory.
Agents are confidently wrong about their own history. A session summary written from memory is
missing the tool results, the exit codes, and the files that changed without anyone mentioning
them, so "why did you overwrite that file" gets a fluent answer rather than a true one. This skill
has the agent open the recording of the run, walk the causal chain to the step in question, and
answer from what is actually in it — and, where it helps, replay that run offline so the failure
happens again in front of you.
It is the forensic counterpart to
zero-hallucination-coder: that one stops the agent inventingfacts about code it has not read, this one stops it inventing facts about work it has already done.
The discipline I care most about is in step 3: the trace labels every causal edge
recorded(therecorder watched it happen) or
inferred(derived at query time from a named rule), and the skillrequires the agent to keep those apart in its answer instead of collapsing both into one confident
sentence.
Conventions checklist
dev.nameanddescription— nolicense,metadata,version,author,categoryor anything else."what" vs "why").
zero-hallucination-coder,agent-harness,handoff,llm-cost-optimizerand
chaos-engineering.scripts/,references/orassets/— the skill is self-contained.Two things I want to flag rather than have you find
1. Layout. CONVENTIONS.md documents
<domain>/<skill-name>/SKILL.md, so that is what I used.Looking at
dev, though, 50 of the 94 skills underengineering/actually live at<domain>/<name>/skills/<name>/SKILL.mdwith a.claude-plugin/plugin.jsonand aREADME.mdalongside, and 38 more use a third shape. Tell me which one you want and I will restructure in this
PR — I did not want to guess against your own written convention.
Related: existing skills do not cross-link with relative paths, so I wrote the cross-references as
plain skill names in backticks rather than inventing a link format.
2. Disclosure. The skill drives an external tool — the
orcareplaynpm package and its MCPserver — and I maintain that tool. So, plainly:
npm i -g orcareplay, and runs entirely locally. No account, nohosted service, no metered tier, and nothing is sent anywhere. The prerequisite block in the
skill says exactly what is needed.
the network (recorded shell commands execute for real, so it is not a sandbox); replay is
destructive in-place unless
worktree: trueis passed; and a matching replay is not adeterminism result, because the model is not re-asked.
is a fair call and I will close this myself.