Typed judgment layer for coding agents — gates from PRD to ship. Jev-ready.
Built for [ZCode] but provider-agnostic: vendor the src/ folder into any
TypeScript agent harness.
Frontier models generate. Judges decide. Code owns side effects.
An agent's biggest lie is "I'm done." This module calls that lie with cheap, typed, threshold-checked judgments — today via any OpenAI/Anthropic-compatible LLM (JSON-forced), tomorrow via TypeSafe Jev by changing one constructor.
import { JudgmentHarness } from './src/core';
import { preExecGate, doneGate } from './src/gates';
import { llmJsonBackend } from './src/backends/llm-json';
const backend = llmJsonBackend({
baseUrl: 'https://api.z.ai/api/anthropic/v1',
apiKey: process.env.GLM_API_KEY!,
model: 'glm-5.3-flash',
});
const harness = new JudgmentHarness({ backend, logDir: '.judgments' });
// Before any destructive command:
const verdict = await harness.run(preExecGate, commandString);
if (verdict.action === 'human') return askHumanApproval();
// Completion check — one Noul per acceptance criterion, batched:
const done = await harness.run(
doneGate(['adds min-w-0 to the select trigger', 'updates the snapshot']),
diffText,
);
if (done.action === 'loop') agent.continueWith(done.failures); // feeds back the failing ACs| Gate | Phase | On fail |
|---|---|---|
prd-gate |
PRD intake — measurable criteria? ambiguity? | human |
coverage-gate(ac, req) |
spec extraction — does each AC verify its requirement? | loop |
plan-gate |
planning — scope creep, risky ordering, reversibility | loop |
pre-exec-gate |
before shell commands — irreversible? blast radius? | human |
diff-gate |
after each step — matches task? out-of-scope files? | loop |
done-gate(acs) |
completion — one Noul per acceptance criterion | loop |
route-gate |
PR — breaking? needs human? | human |
ci-triage-gate |
CI failure — real vs flake | loop |
release-gate |
ship — changelog, rollback plan, migrations | human |
llmJsonBackend— any OpenAI/Anthropic-compatible chat endpoint with JSON forcing (GLM, GPT, Claude, local). Works today. ~7–15s per batched call.jevBackend— TypeSafe Jev (POST /v1/systemone). Stub with the documented contract; adjust field mapping on first live call when keys ship. Expected 70–500ms at ~$0.042/M input tokens.- Write your own: implement
JudgeBackend(one method).
Every harness.run() appends {gate, judgments, action, stateHash} to
logDir/decisions.jsonl. Replay old PRs/tasks you know the outcome of to tune
thresholds before trusting any backend in production. Thresholds are per
gate — loosen them for side projects, tighten for work repos.
npm install
npm test # deterministic: policy engine, parser, gates with fake backend
npm run test:live # 3 real GLM calls: blocks rm -rf, passes grep, loops on incomplete diff- Live-tested on GLM via JSON forcing (see
src/live.test.ts) — judgments were sensible on first contact but LLM judges drift: keep thresholds conservative (≥0.9 require, ≤0.2 forbid) and calibrate from the log. - The Jev backend is a contract stub — their early-access hadn't shipped keys at time of writing. The interface is the stable part.