The quality gate for AI‑assisted development — in a single Rust binary.

Evaluate before you build. Score before you ship. Catch regressions before they ship for you.

Install · Quick start · How it works · Eval harness · Metrics · CLI

AI agents are fast — and confidently wrong. They ship code that doesn't match the ask, skip validation because "it looks right," and burn tokens building the wrong thing. The cost isn't the model call; it's the rework, the missed regression, the silent drift between "what was requested" and "what was delivered."

Janus is a quality gate that sits between intent and merge. It classifies risk at intake, forces acceptance criteria before code, proves each criterion with evidence, and runs behavior‑regression evals on your skills — all from one binary, all backed by one SQLite database.

Named after the Roman god of gates and transitions — two faces, one looking to the past (your baseline), one to the future (your current run). Janus guards the doorway between them.

_{All counts above are verifiable from the source tree and harness-cli --help. See Metrics & impact.}

# One‑liner — installs the binary + git hooks into your project
cd ~/your-project
curl -fsSL https://raw.githubusercontent.com/hoainho/janus/main/scripts/install-harness.sh | bash -s -- --yes

# Initialize the SQLite state database
scripts/bin/harness-cli init

# npm (cross-platform, picks the right prebuilt binary)
npm install -g @nano-step/janus

# From a local clone of Janus
bash ~/janus/scripts/install-harness.sh --yes

# Build from source
git clone https://github.com/hoainho/janus.git
cd janus && cargo build --release

# Include the experimental eval harness (opt-in — off by default)
cargo build --release --features eval

# 1 — Record what you're about to build (Janus classifies the risk lane)
harness-cli intake --type feature --summary "Add OAuth login" --lane normal

# 2 — Open a story to track acceptance criteria
harness-cli story add --id US-001 --title "OAuth login flow" --lane normal

# 3 — ...implement...

# 4 — Verify the story against its evidence, then record a trace
harness-cli story verify US-001
harness-cli trace --summary "Implemented OAuth login" --outcome pass --story US-001

# 5 — See the whole proof matrix at a glance
harness-cli query matrix

Guarding a skill against regressions is just as direct:

harness-cli eval baseline --skill=code-reviewer     # snapshot today's behavior
harness-cli eval run --skill=code-reviewer --mode=2tier   # smoke first, escalate failures
harness-cli eval diff                                 # what changed vs baseline?

Janus runs two cooperating engines over one database: the Harness (governs how a change moves from intent to merge) and the Eval Harness (governs whether your skills still behave as they did).

Every change enters one gate and is routed to a risk lane — tiny, normal, or high-risk. The lane decides which of the 9 gates apply.

flowchart TD
    START([User intent]) --> INTAKE[Feature intake]
    INTAKE -->|classify risk| LANE{Risk lane?}
    LANE -->|tiny| PATCH[Patch and validate]
    LANE -->|normal or high-risk| PROPOSE[Propose then deep-design]
    PROPOSE --> T1[T1 Acceptance criteria]
    T1 --> M1[M1 Recall prior fixes]
    M1 --> T2[T2 AC to test matrix]
    T2 --> IMPL[Implement]
    IMPL --> T3[T3 Evidence per AC]
    T3 --> REVIEW[Review gate reviewer not implementer]
    REVIEW --> T4[T4 Reconciliation]
    T4 --> M2[M2 Persist for recall]
    PATCH --> M2
    M2 --> ARCHIVE([Archive])

    style START fill:#6366F1,color:#fff
    style ARCHIVE fill:#10B981,color:#fff
    style REVIEW fill:#8B5CF6,color:#fff

flowchart LR
    subgraph Pre
        T0[T0 Pre-flight]
        T1[T1 AC extraction]
        M1[M1 Recall]
    end
    subgraph During
        T2[T2 Matrix]
        T3[T3 Evidence]
        REV[Review]
    end
    subgraph Post
        P1[P1 Push permission]
        P2[P2 Jira tier]
        T4[T4 Reconciliation]
    end
    subgraph Persist
        M2[M2 Persist]
    end
    T0 --> T1 --> M1 --> T2 --> T3 --> REV --> P1 --> P2 --> T4 --> M2

⚗️ Experimental / opt-in. The eval harness is not compiled into the default binary. Build it with cargo build --release --features eval (or cargo install --features eval); without that flag the harness-cli eval … subcommand is absent. The 9 quality gates above ship in every build — only this eval layer is gated.

The eval harness answers one question on every push: does this skill still do what it did at baseline? Pick a mode, run the checks, and on a regression Janus attributes the likely cause instead of just turning red.

flowchart TD
    RUN[harness-cli eval run] --> MODE{Mode}
    MODE -->|smoke| S[1 sample / case]
    MODE -->|full| F[3 samples / case]
    MODE -->|2tier| TT[smoke first, escalate failures]
    S --> CHK[Run 9 check kinds]
    F --> CHK
    TT --> CHK
    CHK --> Q{All pass?}
    Q -->|yes| PASS[VERDICT: PASS]
    Q -->|no| ATTR[Attribute the cause]
    ATTR --> CLASS{Class}
    CLASS -->|SKILL_CHANGED| R[VERDICT: REGRESSION]
    CLASS -->|FIXTURE_STALE| R
    CLASS -->|MODEL_CHANGED| R
    CLASS -->|CROSS_SKILL_CHANGE| R
    CLASS -->|UNKNOWN_DRIFT| R
    PASS --> GATE{Promoted?}
    R --> GATE
    GATE -->|warn-only| W[exit 0]
    GATE -->|blocking and regression| B[exit 12 push blocked]

    style PASS fill:#10B981,color:#fff
    style R fill:#F43F5E,color:#fff
    style B fill:#F43F5E,color:#fff
    style W fill:#F59E0B,color:#fff

harness-cli eval run --skill=my-skill --mode=smoke   # fast — 1 sample
harness-cli eval run --skill=my-skill --mode=full    # thorough — 3 samples
harness-cli eval run --skill=my-skill --mode=2tier   # smoke, then escalate only failures

When something regresses, the diff alone rarely tells you why. Janus does:

SKILL_CHANGED · FIXTURE_STALE · MODEL_CHANGED · CROSS_SKILL_CHANGE · UNKNOWN_DRIFT

schema_version: 2
id: review-must-flag-sql-injection
mode: deterministic
skill_under_test: code-reviewer
skills_loaded: [code-reviewer]
description: "Reviewer must flag a raw SQL string interpolation"

budget:
  max_tokens: 50000
  max_seconds: 180

prompt: "Review src/db.ts and report security issues."

checks:
  - kind: file_exists
    path: review.md
  - kind: output_contains
    text: "sql injection"
    normalize: [whitespace, case]
  - kind: llm_judge
    target_file: review.md
    rubric: "Must identify the unsanitized query as a security risk"
    samples: 3

cp crates/harness-cli/hooks/pre-push     .git/hooks/   # auto-eval changed skills on push
cp crates/harness-cli/hooks/sync-publish .git/hooks/   # block publish on regression
cp crates/harness-cli/hooks/opencode-stop.sh .git/hooks/   # eval when a session ends

These come straight from the build and the test suite — reproduce them yourself:

Evals are real model calls, so Janus treats spend as a first‑class concern:

• Per‑case caps — budget.max_tokens and budget.max_seconds on every case.
• Daily budget — EVAL_BUDGET_USD (default $2.00) with a running ledger.
• Pricing aware — token cost computed from a built‑in model price table.
• Per‑run accounting — duration and cost recorded per run in SQLite for eval trend.

⚠️ The numbers below are an illustrative projection of a typical team workflow, not measured benchmarks of this repo. They show the kind of payoff a quality gate is designed to produce; your mileage will vary. Only the table above is measured.

harness-cli init                                       # initialize the database
harness-cli intake --type=X --summary=Y --lane=Z       # record + classify work
harness-cli story add --id=X --title=Y --lane=Z
harness-cli story update --id=X --status=Y
harness-cli story verify X                              # verify one story
harness-cli story verify-all
harness-cli decision add --id=X --title=Y
harness-cli trace --summary=X --outcome=Y [--story=Z]
harness-cli score-trace
harness-cli backlog add --title=X --pain=Y
harness-cli propose                                    # suggest harness improvements
harness-cli audit                                      # detect doc/state drift

harness-cli query matrix        # story proof matrix
harness-cli query intakes
harness-cli query traces
harness-cli query decisions
harness-cli query backlog
harness-cli query stats
harness-cli query gcr           # gate-compliance metric
harness-cli query friction

harness-cli eval run --skill=X [--mode=smoke|full|2tier] [--strict]
harness-cli eval baseline --skill=X
harness-cli eval diff
harness-cli eval status [--skill=X]
harness-cli eval promote --skill=X            # warn-only -> blocking
harness-cli eval trend --skill=X [--last=N]
harness-cli eval accept --skill=X --case=Y
harness-cli eval apply --skill=X              # show fix proposals
harness-cli eval ab --skill-a=X --skill-b=Y
harness-cli eval rebaseline --skill=X
harness-cli eval analyze --skill=X
harness-cli eval suggest --skill=X
harness-cli eval goal --skill=X --type=Y --target=Z
harness-cli eval quality --skill=X

harness-cli  (single Rust binary)
├── domain.rs / application.rs / infrastructure.rs / interface.rs
├── eval/
│   ├── scoring.rs       # 9 check kinds + normalization
│   ├── attribution.rs   # 5 attribution classes
│   ├── case.rs          # YAML parsing + fixtures (path-traversal guarded)
│   ├── spawn.rs         # runs the skill under test
│   ├── budget.rs · pricing.rs   # daily budget + cost accounting
│   ├── storage.rs       # SQLite: runs, baselines, goals, suggestions
│   ├── quality.rs · context.rs  # prompt/response/context scoring
│   └── stability.rs · diff.rs · report.rs (JUnit/SARIF) · ...
├── hooks/               # pre-push · sync-publish · opencode-stop
└── scripts/
    ├── install-harness.sh
    └── schema/          # 8 SQL migrations

git clone https://github.com/hoainho/janus.git
cd janus
cargo build
cargo test          # 58 tests

See CONTRIBUTING.md. Issues and PRs welcome at github.com/hoainho/janus.

MIT © 2025–2026 Hoài Nhớ.