Agent trajectory diagnosis + rubric wash CLI
Find where an Agent run first fails → drop the cascade of useless retries → export a clean segment for eval / SFT.
Local-first · zero npm deps · rule-rubric by default (no API key).
git clone https://github.com/ktz03/traj-wash.git
cd traj-wash
node bin/traj-wash.js evaluate examples/bad.trajectory.json
node bin/traj-wash.js wash examples/bad.trajectory.json --out examples/washed.jsonOn the bundled bad example: 7 steps → 4, rubric 0.37 → ~0.54, failure onset kept as evidence.
npm test # or: npm run demo| Command | What it does |
|---|---|
evaluate <file> |
Rule checks + rubric scores + failure onset index |
wash <file> |
Drop post-onset cascade fails; write washed JSON |
export <file> |
Alias of wash |
--rubric path |
Rubric JSON (default rubrics/default.json) |
--out path |
Output path |
{
"steps": [
{
"role": "assistant",
"content": "...",
"tool": "run_tests",
"tool_input": {},
"tool_result": "FAIL...",
"ok": false
}
]
}Cascading failed tool retries pollute Agent eval and SFT data. traj-wash locates failure onset, washes the noisy tail, and reports effective action ratio plus score before/after.
MIT