cowork-harness

Unofficial. An independent project, not affiliated with, endorsed by, or supported by Anthropic. "Claude" and "Claude Cowork" are Anthropic's. This harness emulates Cowork's observable runtime contract and drives Anthropic's own agent binary from your local Claude Desktop install — it bundles no Anthropic code, and it is not Cowork.

Scriptable, CI-friendly test harness that reproduces Claude Cowork's observable runtime contract closely enough to test the skills you write — across many scenarios, headless, in CI — without the (locked) Desktop app. It reproduces not just Cowork's behavior but its limitations: sealed filesystem, default-deny egress, MCP-only cross-boundary — so a green test has cleared the constraints that break skills in Cowork. That is a far stronger signal than a bare claude -p run, and it is not a guarantee: this is an emulator of the contract, and the deliberate divergences are catalogued in docs/fidelity-gaps.md.

And because every run is recorded, you get the thing a transcript can't give you: evidence of what the agent actually did, not what it said it did — which skill was invoked (if any), which files a sub-agent really read, which hosts it reached, which options a person was really shown. See Why not just claude -p or the Agent SDK?.

Debugging a run? → docs/debugging.md — a separate page, not a section below.

New here? Start by running a committed cassette replay and browsing examples/ (see examples/README.md) to see green runs before any setup — then read docs/boundary.md (the limitations model) and docs/session.md (the file you'll author).

What this is and isn't. This is an emulator of the contract, not the Desktop runtime. Cowork runs a session in one of two lanes: local — the Desktop app driving the agent on your own machine against an Apple Virtualization.framework microVM sandbox (on the pinned baseline the agent loop runs on the host and reaches into the VM for shell; a VM-loop configuration that runs the whole agent inside the microVM also exists — see DESIGN.md, "Which Cowork? — both are implemented") — or remote, where the agent runs in an Anthropic-hosted cloud container (the default for new sessions since 2026-07-07; local stays available). This harness emulates the local lane's runtime, and holds a run to either lane's delivery contract via a scenario's lane: key — the lanes deliver files differently, which is the part that changes skill behaviour (see docs/fidelity-gaps.md, "File delivery"). You cannot drive the local microVM from a script (Cowork's session control plane is closed off; see DESIGN.md §1 for why). What you can faithfully reproduce is everything that actually changes how a skill behaves: the same agent binary in cowork mode (CLAUDE_CODE_IS_COWORK=1 — there is no --cowork flag), the same mount layout, the same egress allowlist, and the same permission/question protocol. That's what this project does.

Zero-friction preview — no token, no Docker. A committed cassette replays from a fresh clone (the example cassette ships in the repo; just Node ≥ 22):

git clone https://github.com/yaniv-golan/cowork-harness && cd cowork-harness
npm ci && npm run build
node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json

(Installing globally — npm install -g "cowork-harness@^3.7.0" — gives you the cowork-harness CLI for your own scenarios and cassettes; the bundled example above also replays from a global install — see the $(npm root -g) path below.)

Full setup → Quick start.

Three ways to use this project. Each row is the whole hook — follow the link for the full page.

Not sure a harness is what you need? The next two sections are the argument.

A skill's behavior under Cowork is determined by four things, all reproducible outside the VM:

The permission/question protocol is the backbone, and it's the most stable surface — it's the documented Agent SDK control protocol (can_use_tool, hook_callback, mcp_message, …). Everything fragile (agent version, mount paths, allowlist contents) is pushed into a versioned baseline that you re-sync per release. See Maintenance.

Design principle: fail loud, never silently wrong. An unscripted question, a stale cassette, an unadded skill file, a missing capability — every one of these is a hard error or a loud warning, never a silent pass. Where you see "fails loud" or "no silent false-greens" elsewhere in this doc, it's this same principle applied to one specific mechanism.

Both are excellent, and if you're building an agent you should use them. This harness exists for a narrower job: testing a skill the way Cowork will actually run it. Five things it gives you that are hard or impossible to reach from a plain CLI session or your own SDK loop — each one anchored to the mechanism or the record field that backs it, so you can check the claim rather than take it:

Two more that matter in practice:

• A blocking gate you can actually answer. AskUserQuestion blocks: it is a question to a human, and claude -p has no human. A gated skill under a plain CLI run either stalls at the gate or never reaches the code behind it — so the half of your skill that lives past the first question is untestable, not merely awkward to test. The harness answers over the same can_use_tool control protocol Desktop uses, from your scenario's scripted answers: — or from a live decider when you're still discovering what it asks — so the gate genuinely fires and is answered deterministically. on_unanswered: decides what an unscripted gate does (fail loud by default). See scenario.md, decider-dir.md.
• Token-free CI. Record a run once, commit the cassette, and every PR re-runs it deterministically at zero spend — assertions, tool stream, gate answers and all. A cassette replays in well under a second with no token, no Docker and no model call (the three shipped examples replay in ~0.6s total), which is what makes an always-on per-PR gate affordable. See cassette.md.

What it doesn't do. It runs and records; it does not design your experiment. Comparing "with skill" against "without skill" credibly — scrubbing tells, shuffling, judging blind, unblinding after grading — is still yours to build; the harness contributes the run execution and the control arm (--ablate-skill, one arm per invocation). And it emulates the contract, not the Desktop runtime: see Limitations and fidelity-gaps.md for what it deliberately does not reproduce.

You author one file. It is the prompt, the answers to any questions the skill asks, and what must be true when it finishes:

# scenarios/pdf.yaml
session: ../sessions/default.yaml
fidelity: container

prompt: |
  Summarize report.pdf and write the action items to outputs/actions.md

answers:                                   # the same control channel Desktop's question UI uses
  - when_question: "Which output format"   # regex on an AskUserQuestion gate
    choose: "Markdown"
  - when_tool: Bash
    allow_if: "!command.includes('rm -rf')"
    else: deny

expect_denied: ["evil.example.com"]        # this host must be refused egress

assert:
  - user_visible_artifact: outputs/actions.md   # the file actually reached the user
  - tool_called: Write
  - transcript_contains: "action items"

cowork-harness run scenarios/pdf.yaml prints a verdict and writes the whole run to disk:

✓ success [container] · 7 tools · 24.3s · $0.18
   [provenance] model=claude-sonnet-5  skill=offered,invoked  ablated=false
   guards: capability-use ✓  permissive-auto-allow ✓  host-path ✓  outputs-delete ✓

That skill=offered,invoked is the part a transcript cannot give you: whether the skill was selected, not just whether the answer looked right. offered,NOT-invoked on a green run means the model solved it without your skill — which is a finding, not a pass.

Two runs are often worth more than one. The same scenario at two fidelity tiers, diffed, is a discovery technique in its own right: a skill that passes at container and fails at hostloop has told you something specific about where it will break, and the difference is the finding. See Fidelity tiers.

Prove your assertions can fail. The harness fails loud rather than passing silently, but it cannot stop you writing an assertion that could never go red. Break the thing under test on purpose once — move the file, rename the skill — and confirm you get the ✗. An assertion you have never seen fail is not yet evidence.

A real one, from someone dogfooding a skill of their own. The prompt named their skill outright; the run came back green and the answer read fine. The record said otherwise:

skillsInvoked: []          # the skill was offered and never invoked
toolCounts:    {}          # zero tools — it never read anything
finalMessage:  "This is a quick syntax question, not a full skill-creation
                workflow, so I'll just answer it directly."

The model declined the skill and answered from its own knowledge. The guidance being tested was correct and simply never consulted — so the run was measuring the model, not the skill. No transcript of the answer would have shown that, because the answer was fine.

They widened the skill's description in response, re-ran the same probe, and it now invokes — then pinned the fix with a scenario so it cannot regress silently. That is the loop this is for: find a failure the output hides, fix it, and verify with the same instrument that found it. The snapshot above is a moment, not a standing bug — reproduce it today against that skill and it passes, which is the point.

That is the general shape: not "did the output look right", but "did the thing I am shipping actually run, under the constraints it will meet in production".

Requirements at a glance (a summary — full detail in Prerequisites on the CLI page)

• Free demo (replay): Node ≥ 22 — nothing else (no Docker, token, or Claude Desktop).
• Global npm install -g: ships the runnable examples/ subtrees — replays/, scenarios/, sessions/, skills/, data/ — under $(npm root -g)/cowork-harness/, so replay and run examples/scenarios/… both work from one; matrices/, answer-policies/ and probes/ still need a source checkout. Full detail in Prerequisites on the CLI page.
• lint (optional, token-free): also needs python3 on PATH — the scenario linter shells out to it (PyYAML is bundled); a missing python3 is a hard exit 127.
• Live tiers need three things:
  • Claude Desktop, opened once — stages the agent; nothing is bundled.
  • A Claude token — real per-run cost, runs take minutes; mint one with claude setup-token (needs the claude CLI: npm i -g @anthropic-ai/claude-code).
  • A runtime — Docker (arm64) for container (default) / hostloop, or Lima (Apple-VZ) for microvm.
  • The protocol tier skips the runtime + the staged agent but still calls a real model, so it still needs the token. Run cowork-harness doctor --tier <t> to check exactly what a given tier needs.
• Platform: best on macOS Apple Silicon; Windows is not supported for the live tiers (use the token-free replay); sync and microvm are macOS-arm64 only. Full detail in Prerequisites on the CLI page.

This page vs. the other four. Fidelity is documented in five places, on purpose — each answers a different question:

Question	Page
Which tier should I pick?	this section — the decision table
What does each tier enforce?	docs/boundary.md
What does each tier NOT reproduce?	docs/fidelity-gaps.md
Why is it built this way?	DESIGN.md § 2 Parity matrix
I only have the installed plugin	references/fidelity-and-answers.md — offline snapshot

L0  protocol-only     claude -p stream-json (the agent's JSON-lines I/O format) on the host. No sandbox,
                      no egress control. Fastest. Pure-logic / inner-loop assertions.

L1  container parity  Pinned agent in cowork mode inside an arm64 Linux container with the real
   (recommended)      mount layout and a default-deny egress proxy enforcing the synced allowlist.
                      Reproducible, CI-native (Docker/Podman). The faithful-yet-maintainable sweet spot.

L2  microvm parity    Optional. Agent inside a real Linux microVM (Lima/Apple-VZ) with a guest
   (opt-in, heavy)    default-deny iptables firewall funnelling to the same allowlist proxy as L1.
                      VM-grade escape isolation; egress transport equals L1's HTTP-CONNECT proxy (the
                      HTTP tunneling method used for HTTPS-through-a-proxy) — no gVisor netstack
                      reproduced. Not for CI; periodic high-fidelity checks only.

    ─── loop-mode overlays (orthogonal to L0/L1/L2: they pick WHERE the loop runs, not isolation) ───

    hostloop          Cowork's PRODUCTION split-execution. The agent loop is a NATIVE process spawned
                      directly on the host (no container around the file tools, matching production) —
                      native Bash/WebFetch are disabled and routed host-side via the workspace SDK-MCP
                      server — an in-process MCP server the Agent SDK talks to directly (tool name
                      mcp__workspace__bash), whose bash calls route into a Docker VM sidecar; web_fetch
                      routes via host curl. A PreToolUse path-containment hook is the security boundary
                      for real filesystem access at this tier — see docs/boundary.md.

    Both container and hostloop also declare the skills/plugins skill/plugin-discovery SDK-MCP servers
    (mcp__skills__list_skills/suggest_skills, mcp__plugins__list_plugins/search_plugins/
    suggest_plugin_install) alongside cowork/workspace — see docs/fidelity-gaps.md.

    cowork            Auto-picks hostloop vs container the way Cowork itself does — decoded from
                      GrowthBook gate 1143815894 (Cowork's internal feature-flag system) in the
                      synced baseline. "Do what real Cowork does."

Every tier above reproduces Cowork's desktop-local lane: the agent runs on your machine, shell commands land in a Linux sandbox rooted at /sessions/<id>, attached folders appear under /sessions/<id>/mnt/<name>, and finished files reach the user through present_files.

Cowork also has a remote lane, where the session runs server-side in an ephemeral cloud container that reaches your machine over a link. There the filesystem, the shell tool, and file delivery are all different — folders arrive under $HOME/mnt/, deliverables go to /mnt/user-data/outputs/ and are handed over with SendUserFile, and the environment prompt is authored by the server rather than by Desktop. Which lane you get is a Cowork setting ("Only on this computer", Settings → Cowork), and it has been observed off — i.e. remote — on a current install.

The harness cannot execute the remote lane: that container is Anthropic's, not something a local tool can stand up. What it does instead is refuse to fake it. Declare lane: remote on a scenario and the assertions that depend on observing a local filesystem degrade honestly — file_absent reports evidence-unavailable rather than passing, and delivery is reported as unobservable — so a green never means more than it should.

What this means for you. Behaviour-shaped conclusions travel between lanes: whether your skill triggers, how it sequences tools, which questions it asks, whether it respects a permission gate. Environment-shaped conclusions do not: anything asserting a path, a mount, or a delivery mechanism is a statement about the local lane specifically. Scope your claims accordingly, and if you are probing real Cowork to compare, turn "Only on this computer" on first or you will be measuring a lane this tool does not model.

Decision guide — fidelity: takes exactly one of these five values (protocol/container/microvm vary isolation strength; hostloop/cowork are overlays that instead pick where the loop runs — there's no combining the two groups):

Note: cowork above is one value of fidelity:. Two other cowork-named settings are unrelated — permission_parity: cowork (a session setting for how unscripted tool calls are treated; see Two files: session + scenario) and "cowork mode" (the CLAUDE_CODE_IS_COWORK=1 env flag every live tier passes to the agent binary, regardless of which fidelity tier you picked).

Recorded cassettes pin the tier a fidelity: cowork scenario actually resolved to as effectiveFidelity, which backs the resolved-tier/unverifiable-tier staleness classes — see docs/cassette.md.

Set the tier with fidelity: in a scenario, or --fidelity on skill / chat / critique / probe-dispatch.

                      ┌────────────────────────────────────────────────┐
  scenario.yaml ────► │  cowork-harness  (TypeScript CLI)              │
                      │    baseline loader ◄── baselines/desktop-*.json│
                      │    runtime selector  ──►  L0 / L1 / L2         │
                      └───────────────────────┬────────────────────────┘
                                              │  spawns + speaks stream-json
                      ┌───────────────────────▼────────────────────────┐
                      │  Agent: the STAGED Cowork binary               │
                      │    claude-code-vm/<ver>/claude                 │
                      │    (NOT `claude -p` from your PATH)            │
                      │    CLAUDE_CODE_IS_COWORK=1                     │
                      │    --input-format / --output-format stream-json│
                      │    cwd = /sessions/<id>                        │
                      │    mnt/uploads · mnt/<folder-name> · plugins   │
                      └───────────────────────┬────────────────────────┘
            decision control request          │  outbound network (egress)
            (tool · question · dialog)        │  default-deny → allowlist
                      ┌───────────────────────▼────────────┐    ┌────────────────────────┐
                      │  AgentSession ──► Decider ──► Run  │    │  Egress proxy          │
                      │  protocol · policy · turn loop     │    │  default-deny;         │
                      │  + RunRecord                       │    │  allowlist = synced    │
                      │                                    │    │  vmAllowedDomains()    │
                      └────────────────────────────────────┘    └────────────────────────┘

The diagram above shows the VM-loop path (container/microvm). At hostloop fidelity the agent loop instead runs as a native host process with no container around it, routing shell/file access through a workspace SDK-MCP server into a VM sidecar — see the Spawn contract section in DESIGN.md for detail. Only the L0 protocol tier spawns claude from your PATH. Every other tier runs a binary staged by your own Claude Desktop install: container/microvm bind-mount the Linux VM ELF (claude-code-vm/<ver>/claude), while hostloop spawns the separate native macOS binary (claude-code/<ver>/claude.app/…) directly on the host, with no container around it — two different staged binaries, in two different version namespaces. That is what makes the run Cowork-shaped rather than CLI-shaped. See Why not just claude -p.

• AgentSession speaks the Agent SDK control protocol over stream-json, emitting a typed event stream. When the agent emits a decision request (a tool permission, an AskUserQuestion, or a request_user_dialog/elicitation), the Decider resolves it — scripted answers: first, then the cowork/strict permission default, then the on_unanswered policy (fail/prompt/first/llm) — and Run drives the turn loop and builds the RunRecord (decisions, the sub-agent dispatch tree, egress, cost).
• Egress proxy (L1/L2) enforces the synced allowlist; default-deny. Domains come from the baseline, plus per-scenario extra_allow.
• The platform baseline is the single source of release-specific truth. Code rides the stable protocol; data tracks the release.

See DESIGN.md for the full parity matrix, the known deltas vs. real Cowork, and the threat-model notes on egress.

• One lane, deliberately. Every tier models Cowork's desktop-local lane. The remote (cloud) lane runs server-side with a different filesystem, shell tool, delivery mechanism and a server-authored prompt; no tier reproduces it, and lane: remote exists to make the resulting blind spots refuse to grade rather than pass. See Which Cowork lane this models.
• Not the full Desktop network transport. L1 is a container, not a VM; L2 is a real Apple-VZ microVM but still does not reproduce Cowork's gVisor netstack — its egress is the same allowlist proxy as L1 (with a guest iptables firewall in front). If your skill depends on VM-kernel specifics, validate at L2; if it depends on packet-level gVisor behavior, no tier reproduces it.
• Cowork in-guest context is partial. Desktop supplies host-loop staging, runtime mountPath RPC, and the bridge. We reproduce the filesystem and cowork mode, not those host-side services. Skills that call Desktop-only host RPCs won't run here (they wouldn't be portable anyway).
• The agent binary is the staged ELF (claude-code-vm/<ver>/claude), bind-mounted from your own Claude Desktop install — nothing Anthropic-owned is bundled or installed. There is no npm path; override the path with COWORK_AGENT_BINARY. Check licensing/ToS for your use.
• Egress fidelity is domain-exact against a pinned list, transport-approximate at L1 and L2. Allow/deny is decided per domain the way Cowork decides it, but the allowlist itself is a hand-curated reconstruction rather than an extraction — the first-party deployment delivers it per session from the server, not in the app bundle, so it cannot be read out — and the baseline's network.$comment flags four entries as unverified as VM egress. The packet-level gVisor netstack is reproduced at neither tier; both use a default-deny allowlist proxy (L2 adds a guest iptables firewall).

These are documented per-tier in DESIGN.md so a green test means what you think it means.

This repo is built to be driven by agents, not just read by humans:

• AGENTS.md — the canonical agent-instructions file (architecture seams, the build gate, invariants, ethos). Read it before changing code. Also indexed in llms.txt.
• Companion skill — .claude/skills/cowork-harness/ teaches an agent to drive the harness; install it via the marketplace (see above).
• Machine-readable interfaces — stable --output-format json envelope on stdout, deterministic exit codes (0/1/2/3, with a couple of documented per-command exceptions — see SPEC.md for the full table), and --help on every command.
• JSON Schemas — schema/scenario.schema.json and schema/session.schema.json describe every field of the YAML you author (generated from the source schemas; npm run schema). schema/protocol.v1.json (hand-authored) schemas the harness's own control-channel wire protocol, with a golden vector pack at fixtures/protocol/v1/ — see docs/protocol.md. schema/critique-report.json describes critique's JSON report / critique-report.json artifact for automation consumers (budget pacers, harvesters) — descriptive, not §12-frozen while critique is EXPERIMENTAL (see SPEC.md §12).

AGENTS.md, SPEC.md, and DESIGN.md are shipped in the npm package (see package.json files) — a global install has them locally too, not just on GitHub.

From 1.0.0 this project follows semver. What that covers is enumerated in SPEC.md §12 — the CLI commands/flags and exit codes, the scenario/session/baseline/RunResult/cassette/protocol schemas, the documented COWORK_HARNESS_* (+ COWORK_AGENT_BINARY/COWORK_AGENT_IMAGE) env vars, and the packaged Action's inputs/outputs. Human-readable terminal text is explicitly not part of the contract — parse the --output-format json envelope, not stdout. As of 1.0.0, a backwards-incompatible change to a covered surface is a major bump.

The latest shipped baseline — what baseline: latest resolves to (cowork-harness list) — is desktop-2.2553.1. Release-by-release verification notes (what was re-verified against which live agent/asar) are recorded in CHANGELOG.md; the feature catalogue this section would otherwise duplicate lives in the sections above.