Marmelab's AI harness

A Claude Code plugin that turns a code-change request into a reviewed, merged change, driven by a team of subagents working in git worktrees.

The point is not the agents, it is the enforcement. Every gate is a hook, so it fires whether or not the model cooperates. A prompt that says "never merge yourself" is a suggestion; a PreToolUse hook that refuses git merge is not. As this repo's own dependency rule puts it: a deny is ~100% respected, a CLAUDE.md line is not.

The orchestrator classifies the request, the planner breaks it into tickets, a developer implements each one in its own worktree, the quality-reviewer reviews it, the merger merges it into a session branch, and the whole feature is reviewed once more before promotion. Around that, hooks:

• create and tear down the worktrees, so no agent invents its own layout
• run the validation chain (format, typecheck, lint, unit) on every developer stop, and reject the stop until it is green
• refuse to dispatch a merger without a recorded review verdict
• refuse to promote while approved work sits unmerged
• run the e2e suite once, at end of feature, only after the feature review approved, and reject the orchestrator's stop if it tries to finish while that suite is red
• block the commands that would make the pipeline look healthy while being broken: merging outside the merger, launching arbitrary containers, opening a headed browser
• size each review by a difficulty tier computed from the plan and the diff, so a two-file fix pays a sonnet pass, a schema change pays opus, and a ticket the planner scored critical pays fable

A test suite covers those hooks, and CI runs it on every push. That coverage is the product: an untested guard fails silently, and one of them had been inert for months before a test caught it. npm test states the current count.

/plugin marketplace add marmelab/AIHarness
/plugin install aiharness

Then declare your project's facts in harness.config.json at the repo root. The minimum:

{
  "name": "myapp",
  "containers": { "allow": [] },
  "validation": {
    "steps": [
      {
        "id": "typecheck",
        "kind": "typecheck",
        "command": "npm run typecheck"
      },
      {
        "id": "unit",
        "kind": "unit",
        "command": "npm test",
        "changedScoped": true
      }
    ],
    "extraForbidden": ["build", "e2e"]
  },
  "roles": {
    "orchestrator": { "model": "sonnet", "pipeline": false },
    "planner": { "model": "opus", "pipeline": true },
    "developer": { "model": "sonnet", "pipeline": true },
    "quality-reviewer": { "model": "opus", "pipeline": true },
    "merger": { "model": "haiku", "pipeline": true }
  },
  "review": {
    "tiers": {
      "trivial": { "model": "sonnet" },
      "normal": { "model": "sonnet" },
      "hard": { "model": "default" },
      "critical": { "model": "fable" }
    }
  }
}

This repo's own harness.config.json is a working reference, and node scripts/check-config-sync.mjs tells you whether your roles cover the hook matchers.

review.tiers runs in both directions: below the reviewer's own model the harness names a cheaper one, and at critical it names fable, above it. The escalation is the one rewrite that fails downward, since a runtime that ignores model there reviews with the agent's declared model instead. That model is the floor of every failure path and is what every review cost before the tiers existed, so none of them reviews more weakly than the untiered harness did. Whatever a tier names also needs a rate in scripts/lib/session-cost.mjs, or the cost report prices it at the sonnet fallback.

deploy and app are absent from the defaults on purpose: a capability exists if and only if its block is present. Omit one and the feature is simply off, with no warning, which is the failure mode worth knowing about before you go looking for a bug.

app takes smokeCommand, portBase, and optionally portArg, strictPortArg, hashRouting, demoMode. deploy takes relevantGlobs (required when the block is present).

Read by hooks. These change what the harness enforces.

Read by agent prompts. Instructions, not enforcement: an agent can ignore them.

Declarative. Validated or defaulted, but nothing reads them. Setting them changes nothing today.

The harness is opt-in per request: nothing routes through it until you ask, with #harness or "use the agent team".

The plugin declares a TypeScript language server (via npx, so nothing to install). It pays off in an interactive session: an LSP call resolves a symbol through the type system without going through the shell, where Claude Code analyses each command before running it, costing seconds per call.

The pipeline agents do not get it, and do not look for it. Every harness agent is dispatched by the orchestrator, which is itself a subagent and so cannot ask for a foreground dispatch. They all run in the background, and a background subagent has LSP pruned from its tool set: four open runtime reports, no fix, and the one confirmed workaround is the foreground dispatch a nested subagent cannot request. Measured over one full run: 21 agents, 0 LSP calls, 357 Bash calls.

So they answer symbol questions with scripts/ts-symbols.mjs, which reaches the same TypeScript program from Bash, and rules/code-search.md tells them not to spend a turn probing for the tool. There is nothing to configure for the pipeline. The rest of this section is about your own interactive sessions.

Only one LSP server can own a file extension. The runtime registers the first one and the others never start; the order is undefined and there is no priority field, so a plugin cannot win, yield, or even detect that it lost. If you also have the official typescript-lsp plugin enabled, it may claim .ts first, and it ships no binary, so every call answers Executable not found in $PATH. Nothing fails; your interactive sessions just fall back to grep.

Two things report it, both automatic:

• Every session start prints one line naming the conflicting plugin and the remedy.
• npm run check (in this repo) fails on the same condition.

To fix it, disable the other plugin (/plugin, then Manage, then toggle it off) or set it to false in enabledPlugins. Check the user scope (~/.claude/settings.json), not just the project's: a plugin enabled there is invisible to anything the project does, which is exactly how this went unnoticed for two full runs. Alternatively, keep the other plugin and install the binary it expects (npm install -g typescript-language-server typescript).

Verify in an interactive session: an LSP workspaceSymbol call on any TypeScript symbol should return locations rather than an error.

The split is documented in HARNESS-SPLIT.md, including the measurements it rests on. In short: this repo owns the hooks, the agent team, the mechanics rules and the generic skills. You own harness.config.json and your domain skills. Backends and launcher surfaces are adapters, dormant until your config activates them.

This repo used to be a copy-paste kit aimed at working WITHOUT a harness. Five of those files were redundant once the harness arrived and were dropped; HARNESS-SPLIT.md says which, and what already owns each of them.

One caveat worth knowing before adopting it for a non-Supabase stack: four agent prompts still inline Supabase specifics, so you would have to edit them. Moving those into the adapter is the next planned step.

npm install
npm test

No runtime dependencies: the hooks use only node builtins, and vitest is the single devDependency.

Copy hooks/, agents/, skills/, commands/ and scripts/ into your project's .claude/, and merge hooks/hooks.json into your .claude/settings.json (replacing ${CLAUDE_PLUGIN_ROOT} with $CLAUDE_PROJECT_DIR/.claude). The core resolves its own files in either layout.

rules/ is the one directory to copy SELECTIVELY rather than wholesale — see rules/README.md for which files an agent benefits from and which are documentation for whoever works on the harness. Unlike the rest, a project's .claude/rules/*.md is delivered to every one of its subagents, so what you copy there is paid on every dispatch.

One thing does not carry over: the browser tools. A plugin exposes an MCP server as mcp__plugin_aiharness_<server>__<tool>, which is the name the developer and the quality-reviewer declare, so a .mcp.json of your own naming the server playwright grants them nothing (silently, which is how this went unnoticed for weeks). Name that server plugin_aiharness_playwright to keep them.