web-companion

A spec-first SDK that turns any website into something an AI agent can drive through the same UI a human would use — the cursor flies, the button gets clicked, the existing onClick runs. No parallel business code, no fidelity gap.

The contract is a single JSON document at /.well-known/companion.json that declares:

• tools — UI action sequences expressed as click / fill / select / check / wait_for steps
• resources — structured data the AI can read by extracting from the DOM

The runtime dispatches real DOM events against the user's actual elements. add_to_cart calls the same button's onClick your user would press.

  ┌──────────────┐                ┌──────────────────────────────┐
  │  agent host  │   MCP / ws     │           web page           │
  │  (claude     │ ◀────────────▶ │  @web-companion/sdk          │
  │   code,      │                │  + visible cursor            │
  │   etc.)      │                │  + DSL executor              │
  └──────────────┘                └──────────────────────────────┘
                                       loads /.well-known/companion.json
                                       dispatches real DOM events
                                       runs the page's own onClick/onChange

Same companion.json, same DSL, same fidelity — only the transport differs.

You have claude code (or any other stdio-MCP host) on your laptop and you want it to drive your dev server.

# 1. on the page side: drop the sidecar into your React/Vue/vanilla app.
#    It mounts the runtime + cursor, and dials the local bridge over ws.
npm install @web-companion/sidecar

import { Sidecar } from '@web-companion/sidecar/react';

export function App() {
  return (
    <>
      {/* your existing app */}
      <Sidecar backendUrl="ws://127.0.0.1:8765/ws" />
    </>
  );
}

# 2. on the desktop side: run the bridge. It speaks stdio MCP up to claude
#    code and ws down to the page. First connect from a new origin will
#    prompt you to allow / deny.
npm install -g @web-companion/local-bridge
web-companion-bridge start

// 3. tell claude code about the bridge (claude_desktop_config.json):
{
  "mcpServers": {
    "web-companion": {
      "command": "web-companion-bridge",
      "args": ["start"]
    }
  }
}

Now claude code sees tools like <originSlug>--<sessionShort>:add_to_cart and calls them; the bridge forwards to the matching ws session; the cursor flies, the button gets clicked. The bridge meta-tool companion_list_sessions is your "which tab am I working with" lookup.

You're shipping a SaaS where end-users get their own AI assistant running on your servers. The page connects out to your backend; your backend hosts the agent and pushes tools/call down.

The reference implementation in examples/reference-backend is intentionally agent-less — it shows the routing skeleton (HS256 JWT identity → ws session lookup → MCP Streamable HTTP relay) without binding you to a particular LLM. Drop your agent on top of SessionRegistry.request().

// page side — same Sidecar component, just a remote URL + per-user token.
import { Sidecar } from '@web-companion/sidecar/react';

<Sidecar
  backendUrl="wss://agent.yoursaas.com/ws"
  token={signedJwtForCurrentUser}
/>

# pull up the reference backend to see the wiring end-to-end
cd examples/reference-backend
pnpm dev                                # listens on 127.0.0.1:3001
pnpm sign-token alice                   # mint a HS256 JWT

Backend exposes:

• ws://…/ws?token=<JWT> — page sdk dials here
• http://…/mcp — desktop MCP client posts here with Authorization: Bearer <JWT>
• http://…/health — debug only

Tools surface to the MCP client as <userId>:<toolName>. Two browsers with two different tokens get fully disjoint tool namespaces; cross-tenant attempts are rejected with HTTP 403. See examples/reference-backend/README.md for the file map, multi-user demo recipe, and claude_desktop_config.json snippet.

If your users are on Chrome 146+ Canary with chrome://flags → "WebMCP for testing", the page can register its tools directly with the browser's navigator.modelContext. The agent lives in the browser; no bridge, no backend.

import { CompanionRuntime, attachCursor } from '@web-companion/sdk';
import { registerCompanionWithWebMCP } from '@web-companion/webmcp';

const runtime = new CompanionRuntime(attachCursor({}, {}));
await runtime.load();

registerCompanionWithWebMCP(runtime, {
  onUnsupported: (info) => console.warn('WebMCP unavailable:', info.reason),
});

What happens when the browser-side agent calls add_to_cart({id:'mocha'}): the adapter routes through runtime.invokeTool, cursor flies, real MouseEvent('click') is dispatched, the button's existing onClick fires, WebMCP returns { ok: true, stepCount: 1 }. Resources surface as read_<name> tools. Falls back silently in non-WebMCP browsers — safe to call unconditionally.

A tool is a sequence of UI steps. Every target, value, and field selector can contain {paramName} placeholders interpolated from the tool's params at invocation time.

{
  "name": "checkout",
  "description": "Place the order.",
  "steps": [
    { "type": "click", "target": "[data-ai-tool='checkout']" }
  ]
}

Cursor flies to the element, plays a click ripple, dispatches MouseEvent('click', { bubbles: true }). Whatever onClick you have runs.

{
  "name": "search",
  "description": "Search the catalog.",
  "params": {
    "type": "object",
    "properties": { "query": { "type": "string" } },
    "required": ["query"]
  },
  "steps": [
    { "type": "fill",     "target": "[data-ai='search-input']", "value": "{query}" },
    { "type": "click",    "target": "[data-ai='search-submit']" },
    { "type": "wait_for", "target": "[data-ai='results']", "timeoutMs": 3000 }
  ]
}

fill uses the native React-compatible value setter so controlled inputs sync; wait_for polls via MutationObserver so async results region is waited on, not raced.

Non-wait_for steps default to a 1500ms wait for their target to appear; no immediate fail on race conditions.

Multi-route apps can scope a tool to a URL pattern, a DOM marker, or both:

{
  "name": "checkout",
  "where": {
    "url":    "**/cart",                  // optional, glob over location.pathname+search+hash
    "marker": "[data-ai-view='cart']"     // optional, DOM marker for SPAs
  },
  "steps": [ /* ... */ ]
}

Both fields are AND'd; omit where for site-global tools. When the agent invokes the tool from the wrong page, the runtime throws WrongPageError with {currentUrl, currentMarkers, expectedWhere} so the agent can decide whether to navigate first.

Beyond per-tool error handling, where: also drives server-side filtering in v0.4 — see the next section.

A resource is a pure DOM-extraction rule — no JavaScript runs, no parallel data path.

{
  "name": "cart",
  "description": "Current cart contents.",
  "schema": { /* JSON Schema for the result */ },
  "extract": {
    "type": "list",
    "selector": "[data-ai='cart-item']",
    "fields": {
      "id":    { "from": "attr",  "attr": "data-id" },
      "name":  { "from": "text",  "selector": "[data-ai='item-name']" },
      "price": { "from": "text",  "selector": "[data-ai='item-price']" }
    }
  }
}

type: 'single' returns one object; type: 'list' returns one per matched element. from options:

selector on a field is optional — omit to read the source from the item element itself.

A flat 5-tool catalog is fine for the coffee-shop demo. A real SaaS dashboard with 100+ flows isn't — token bloat, name collisions, and an agent that can't tell which tools are usable from the current page.

v0.4 introduces three mechanisms (modules / namespacing / server-side filter), v0.5 adds an identity axis (where.roles), and v0.6 adds a hierarchy axis (nested modules with N-level flow.subflow.tool names). All are opt-in additions inside version: "0.2" — no spec bump.

A v0.2 index companion.json can declare modules instead of (or alongside) inline tools / resources:

{
  "version": "0.2",
  "modules": [
    {
      "name": "checkout",
      "url":  "./companion/checkout.json",
      "where": { "marker": "[data-ai-view='cart']" }
    },
    {
      "name": "search",
      "url":  "./companion/search.json",
      "where": { "marker": "[data-ai-view='search']" }
    }
  ]
}

Each module file is itself a CompanionSpec (v0.2, no nested modules). The SDK loader fetches them in parallel after the index, AND-merging each module's per-module where: into every tool/resource inside.

Tools declared inside a module get their flow name prefixed at the runtime surface: a tool named submit inside the checkout module becomes checkout.submit. The agent sees the namespaced form, but the raw name in the JSON file stays unqualified.

invokeTool('checkout.submit', input) routes to the namespaced entry; invokeTool('submit', input) resolves only to a site-level tool of that exact name. Identifiers are validated against /^[A-Za-z][A-Za-z0-9_-]*$/ — the . is reserved.

Once a catalog is split by flow with where: clauses, both @web-companion/local-bridge and the reference-backend automatically:

• Pre-filter tools/list to only the entries whose where: matches the page's current state.
• Push notifications/tools/list_changed when the SDK's PageStateTracker reports a navigation or marker change.
• Accept _meta: { "scope": "all" } on tools/list as an opt-out for agents that want the full catalog.

Three new meta tools (companion_pages / companion_flows / companion_tools) let the agent introspect:

So the agent's natural first move on connect becomes: call companion_pages → see "I'm on /cart, flow cart is active" → call companion_tools(flow='cart') → see just the four cart tools instead of all 87 in the site catalog.

Same where: clause, new optional field: roles: string[]. The SDK collects the current user's roles from <body data-wc-user-roles="..."> (or <meta> / explicit override — see the RFC), pushes them inside page/changed, and the bridge / backend AND the role check into the existing passesWhere.

{
  "name": "admin",
  "url":  "./companion/admin.json",
  "where": {
    "marker": "[data-ai-view='admin']",
    "roles":  ["admin"]
  }
}

PagesSummary also surfaces userRoles so the agent can reason about its identity context. The full RFC, including the security disclosure ("this is ergonomics, not authorization — the server still does RBAC"), lives in docs/v0.5-auth-aware-filter.md.

The v0.4 hierarchy was one level deep — flow.tool. v0.6 lifts the loader's nested modules forbidden guard and lets a module file declare its own modules: [] array, recursing up to a configurable depth (default maxDepth: 3). Surface names compose by joining every level with .:

// public/.well-known/companion/cart.json — parent module
{
  "version": "0.2",
  "modules": [
    { "name": "advanced", "url": "./cart/advanced.json" }
  ],
  "tools": [
    { "name": "add_to_cart", "description": "...",
      "steps": [{ "type": "click", "target": "[data-ai-tool='add-cart-{id}']" }] }
  ]
}

// public/.well-known/companion/cart/advanced.json — nested leaf
{
  "version": "0.2",
  "tools": [
    { "name": "apply_coupon", "description": "...",
      "steps": [ ... ] }
  ]
}

The catalog now exposes both cart.add_to_cart (inline) and cart.advanced.apply_coupon (nested), with where: cascaded through every level. companion_flows entries gain a parent?: string and depth: number so renderers can build a tree; the default companion_tools(flow=) query is a prefix match (flow='cart' returns both direct and descendant tools).

When to nest: catalogs past ~50 top-level modules, with a natural domain split, sharing lifecycle. The annotator playbook's Nested modules section in Step 4 has the full rubric. Full RFC: docs/v0.6-nested-modules.md.

This section is for agent authors, not site integrators. It answers "my agent connected to a web-companion-instrumented site — what should its first five calls be?". It also untangles two readings of "CLI" that confuse the push-vs-poll question.

Most agents people write for this protocol fall into one of two shapes:

• Long-lived agent — IDE / desktop / continuous loop. Claude Code, Cursor, Claw, a daemon-style headless agent — these run as a long-lived process, spawn (or hold open) an MCP transport, and process user prompts one after another. From inside the user's terminal they look like a CLI, but the process never exits between turns. Push notifications matter here.
• One-shot CLI — gh-style imperative. A script like webcompanion-tool-call shop add_to_cart --id mocha that starts, fires a single tool call, and exits. Lifetime shorter than a page-state diff. Push doesn't help — the CLI is gone before any notification could arrive.

The protocol works for both, but the recommended invocation sequence and the "do I need to listen for notifications" answer differ. Pick the shape that matches what you're writing and follow that recipe.

Long-lived or one-shot, the first round-trip is the same:

1. initialize — the MCP SDK does this for you; you get back capabilities.tools.listChanged: true from both @web-companion/local-bridge and examples/reference-backend. (If you need server-driven notifications, this is your gate — log a warning if it's false.)
2. tools/call companion_pages — { currentUrl, matchedMarkers, currentFlows, userRoles }. Now you know where the user is and what identity context applies. One round-trip, no need to scrape the page yourself.
3. tools/call companion_flows (optional) — full flow inventory with active booleans. Use this when the user asked something open-ended ("what can you do here?") and you want to enumerate capabilities without pulling the full tool catalog.
4. tools/list (or companion_tools(flow="...")) — the page-filtered tool catalog. Default scope is "tools whose where: passes the current page state + user roles". Pass _meta: { scope: "all" } only when you genuinely need the unfiltered site map (debugging, agent diagnostics).
5. tools/call <name> — pick a tool, invoke it. The SDK runs the DSL steps against real DOM; you get a JSON result back.

That's the whole baseline contract. Everything else is optimization.

The "do I need to listen for notifications/tools/list_changed" question only has interesting answers if your process outlives a single user-page-state diff (~150ms typical). For each scenario:

Polling tools/list every N seconds is not part of any recommended recipe — the protocol already gives you on-demand fresh state at every meta tool / tools/list call, and a server-pushed delta when you keep the transport open. Polling would just waste round-trips.

Stdio transport (mode 1): the bridge is a child process of the agent. Both sides write JSON-RPC messages to the pipe at any time. When server.sendToolListChanged() fires, the bridge writes {"jsonrpc":"2.0","method":"notifications/tools/list_changed"} to stdout; the agent's MCP client reads it via stdin and calls the notification handler.

Streamable HTTP transport (mode 2): the client opens /mcp with Accept: text/event-stream; the server keeps that response open as an SSE stream. Server-initiated messages get framed as event: message\ndata: <JSON-RPC>\n\n. Same notification, different wire.

Both transports require the server to declare capabilities.tools.listChanged: true at initialize time — both web-companion surfaces do, so the agent can safely subscribe. Push isn't a web-companion invention; it's bog-standard MCP. See the linked transport docs in docs/v0.4-spec-at-scale.md if you want the wire-level detail.

If you're writing a "run once, exit" script and just need the filtered tool catalog at a moment in time, the whole flow is six lines of pseudocode:

const transport = new StreamableHTTPClientTransport(url, { headers });
const client = new Client({ name: 'demo', version: '0' });
await client.connect(transport);
const pages = await client.callTool({ name: 'companion_pages', arguments: {} });
const tools = await client.listTools();  // already filtered
await client.callTool({ name: '<flow>.<tool>', arguments: { ... } });

No notification listener. No polling. The server-side filter does the work each time you ask.

The protocol is designed so an AI annotator — even one with limited context — can read a page's source, identify interactive elements, and emit a companion.json without writing business code. Four properties make this safe:

1. No business logic in the spec. The annotator never references a JS function. It only points at DOM elements.
2. Selectors are plain CSS. Use whatever's already on the element (aria-*, role, classes, text content) or add a data-ai-* attribute as an anchor when existing markup is unstable.
3. Step semantics are explicit. Every step is one of five known kinds. No arbitrary code path.
4. Fidelity is structural. Real DOM events on actual elements; whatever the user's onClick does is what the agent triggers.

A typical pass over an existing React app:

1. Identify interactive elements you want to expose (buttons, inputs, dropdowns).
2. If their existing selectors aren't stable, add data-ai-* attributes — marker only, no logic change.
3. Identify data the AI should be able to read (cart list, product info). Add data-ai-* markers to the wrapper and field-bearing children.
4. Write companion.json referencing those markers.

No state-management changes. No onClick rewrites. The annotator is annotating, not refactoring.

Recommended path (v0.4): your existing AI coding agent is the annotator. Read docs/annotator-playbook.md — it's the framework-agnostic manual any agent (Claude Code, Cursor, Claw, etc.) follows to do the four steps above. Claude Code users can also load the bundled skill at .claude/skills/web-companion-annotate/ and invoke /web-companion-annotate <path>.

For CI / batch annotations without an interactive agent, @web-companion/annotator is a Claude API-backed CLI MVP that does steps 1–4 from a single .tsx file (suggestions only, doesn't mutate source) — see its NOTE.md for when to pick which route.

type CompanionSpec = {
  version: '0.1' | '0.2';
  modules?: ModuleRef[];          // 0.2 only
  tools?: ToolSpec[];             // site-level (no flow)
  resources?: ResourceSpec[];     // site-level (no flow)
};

type ModuleRef = {
  name: string;                   // [A-Za-z][A-Za-z0-9_-]* — becomes the flow namespace
  url: string;                    // resolved relative to the parent spec
  description?: string;           // shown by `companion_flows`
  where?: WhereSpec;              // AND'd into every contained capability
};

type ToolSpec = {
  name: string;                   // identifier — no `.` (reserved for namespacing)
  description: string;
  params?: JsonSchema;
  where?: WhereSpec;
  steps: Step[];                  // at least one
};

type ResourceSpec = {
  name: string;
  description: string;
  schema: JsonSchema;             // shape of the returned data
  where?: WhereSpec;
  extract: ExtractConfig;
};

type WhereSpec = {
  url?: string;                   // glob over location.pathname+search+hash
  marker?: string;                // CSS selector; presence in DOM
  roles?: string[];               // v0.5 — user-roles intersection (ergonomics, not RBAC)
};                                // at least one of url/marker/roles required

type Step =
  | { type: 'click';    target: string }
  | { type: 'fill';     target: string; value: string }
  | { type: 'select';   target: string; value: string }
  | { type: 'check';    target: string; checked?: boolean }
  | { type: 'wait_for'; target: string; timeoutMs?: number };

type ExtractConfig =
  | { type: 'single'; selector: string; fields: Record<string, FieldExtract> }
  | { type: 'list';   selector: string; fields: Record<string, FieldExtract> };

type FieldExtract =
  | { from: 'text';    selector?: string }
  | { from: 'attr';    selector?: string; attr: string }
  | { from: 'value';   selector?: string }
  | { from: 'checked'; selector?: string };

A live companion.schema.json (draft 2019-09) is published from the spec package at packages/spec/companion.schema.json for editor autocomplete and external validators.

The 0.2 schema is a strict superset of 0.1 — every existing 0.1 file keeps parsing unchanged. Opt into the new shape at your own pace:

1. Bump version to '0.2'. Required to use the modules field.
2. For each conceptual flow, move its tools/resources into companion/<flowName>.json. Each module file is itself a v0.2 CompanionSpec with modules: [] (one level deep, enforced).
3. Replace the moved entries in companion.json with a modules ref each:

   "modules": [
     { "name": "checkout", "url": "./companion/checkout.json",
       "where": { "marker": "[data-ai-view='cart']" } }
   ]

4. (Optional) Add per-flow where: to the module ref — this is what activates the server-side filter. Without it, every module's tools stay site-wide.

The full reference design lives in docs/v0.4-spec-at-scale.md.

For backwards compatibility safeguards:

• A v0.1 file may not declare modules (rejected by the parser).
• Tool/resource/module identifiers ([A-Za-z][A-Za-z0-9_-]*) are enforced in both versions — . was de-facto unused, now reserved.
• _meta: { scope: "all" } on tools/list bypasses the v0.4 filter, so an agent that doesn't know about the filter still works.

No version bump. Drop roles: onto any where: (tool, resource, or module ref). Wire <body data-wc-user-roles="..."> to your auth store, and the SDK does the rest:

// public/.well-known/companion.json
{
  "version": "0.2",
  "modules": [
    { "name": "admin", "url": "./companion/admin.json",
      "where": { "marker": "[data-ai-view='admin']", "roles": ["admin"] } }
  ]
}

// In your root component
useEffect(() => {
  if (user.role === 'anonymous') {
    document.body.removeAttribute('data-wc-user-roles');
  } else {
    document.body.setAttribute('data-wc-user-roles', user.role);
  }
}, [user.role]);

That's it. v0.4 specs continue parsing and running unchanged; new roles: fields are inert against pre-v0.5 SDKs (they just don't filter on it). See docs/v0.5-auth-aware-filter.md for the full security disclosure — the filter is ergonomics, not an authorization gate.

No version bump. Move a group of related modules into a sub-directory and reference them via a parent module's modules: [] array:

public/.well-known/
├── companion.json              # index — still flat refs at the top
├── companion/
│   ├── ecommerce.json          # parent — only `modules: [...]`
│   ├── ecommerce/
│   │   ├── cart.json
│   │   ├── checkout.json
│   │   └── browse.json
│   └── support.json

Surface names automatically become ecommerce.cart.add_to_cart, ecommerce.checkout.submit, etc. — agents that hardcoded the older flat names (cart.add_to_cart) need to be updated. The loader caps at maxDepth: 3 by default; pass loadCompanionSpec(url, { maxDepth: 4 }) for deeper trees, but >=4 is almost always a sign the catalog should be flattened. v0.5 SDKs loading a v0.6 nested spec fail loud via onModuleError ("nested modules forbidden"), so the breakage is visible rather than silent.

packages/
  spec/          @web-companion/spec          Zod schema (v0.1 + v0.2 incl. v0.5 where.roles) + TS types + parser/validator + companion.schema.json
  sdk/           @web-companion/sdk           Runtime: registry, recursive-modules loader (mergeWhere + v0.6 N-level nesting + maxDepth), dsl-executor, dom-extractor, cursor, where-check (roles), ws-client (userRoles), PageStateTracker (4-level role fallback), meta-tools helpers (v0.6 parent/depth + prefix-match)
  sidecar/       @web-companion/sidecar       Headless connector for mode 2 — React/Vue/Vanilla entries
  local-bridge/  @web-companion/local-bridge  Mode 1 — stdio MCP ↔ ws bridge; origin allowlist; navigation grace; server-side filter (url+marker+roles) + meta tools
  webmcp/        @web-companion/webmcp        W3C WebMCP adapter — `navigator.modelContext.registerTool` from a CompanionSpec
  annotator/     @web-companion/annotator     LLM-backed source → spec+marker suggestions; Claude Opus 4.7
examples/
  coffee-shop/                                vite 6 + react 19 end-to-end demo (v0.2 spec, 5 modules + 1 nested cart.advanced sub-flow = 15 tools / 7 resources incl. v0.5 admin + v0.6 cart.advanced). Three Playwright suites: default (8), mode-1 bridge (2), mode-2 backend (5).
  reference-backend/                          Skeleton remote agent backend for mode 2: ws + JWT + MCP Streamable HTTP + v0.4/v0.5 filter / meta tools; agent-less by design.
  with-sidebar/                               Demo: in-page chat sidebar (the old @web-companion/react package, repositioned in v0.3 — see its NOTE.md).

Build chain: spec → sdk → {sidecar, local-bridge, webmcp, annotator, with-sidebar} → examples/*. After modifying any package, pnpm -r build before exercising the demo.

pnpm install
pnpm -r build
pnpm -r typecheck
pnpm --filter coffee-shop dev --host 127.0.0.1
pnpm --filter coffee-shop test:e2e

# explore the reference backend
cd examples/reference-backend
pnpm dev
pnpm sign-token alice

Drop a Playwright spec under examples/coffee-shop/e2e/ if you're touching the cursor or DSL executor.