open-compute-mcp

npm launcher for the open-compute MCP server — model-agnostic computer-use tools exposed over the Model Context Protocol (MCP).

EN | DE

📦 View on npm → • 📋 Security Policy • 🤝 Contributing Guidelines • ⚖️ Licenses • 🤖 LLM Context (llms.txt)

• 1. 🏗️ Architecture
• 2. ✨ Key Capabilities & Invariants
• 3. 🚀 Quick Start & Prerequisites (Use with an MCP Client)
• 4. 🛠️ MCP Tool Reference (16)
• 5. ⚙️ Configuration & Environment
• 6. 🔄 Safe Interaction & Signal Lifecycle
• 7. 🎯 Target Personas & Discoverability
• 8. 🔍 High-Intent Search Term Matrix
• 9. 📊 10-Dimension Comparative Matrix vs. Alternatives
• 10. 🏛️ Governance & Runtime Invariants
• 11. 🌐 ellmos-ai Ecosystem & Sibling Matrix
• 12. 🔒 Security Policy & Zero-Egress Safety
• 13. 📋 Level 1 SBOM Transparency & Invariant Matrix
• 14. 🧪 Testing & Verification Suite
• 15. 📦 Registry Manifests & Schema Parity
• 16. 🚦 Quality Gates & Pre-Release Checklist
• 17. ⚖️ Licensing & Canonical Attribution
• 18. 📜 Statutory Notice (§ 521 BGB) & 48h Security Response SLA

The MCP client is the reasoner (no API key, model-agnostic): it calls capture to see the screen, then acts with do / click_name / invoke. This is the keyless Mode-A loop of open-compute, but as native tool-calls.

graph TD
    A["AI Reasoner<br/>(Claude / Antigravity / Cursor)"] -- "MCP stdio (JSON-RPC)" --> B["npx open-compute-mcp<br/>(Node.js Launcher)"]
    B -- "Spawns via uvx" --> C["open-compute Python Engine<br/>(GitHub @ main)"]
    C -- "Screenshots / WGC" --> D["Windows Display"]
    C -- "UIA / Mouse / Keys" --> E["Windows Desktop Apps"]
    C -- "Glowing Border & Cursor" --> F["Signal Overlay UI"]

    subgraph Safety Gate
        C -. "OC_SAFETY_MODE<br/>(confirm / read_only / allow_all)" .-> C
        C -. "OC_DENY<br/>(hard action blacklist)" .-> C
    end

+====================================================================================================+
|                     OPEN-COMPUTE-MCP — FOUR-VIEW SYSTEM ARCHITECTURE TOPOLOGY                     |
+====================================================================================================+

[VIEW 1: AI CALLER RUNTIMES & MULTI-AGENT CLIENTS]
  +------------------------------------------------------------------------------------------------+
  |  +--------------------+  +--------------------+  +--------------------+  +-------------------+ |
  |  | Claude Code / Desktop |  | Antigravity (AGY)  |  | Cursor / VS Code   |  | Custom MCP Agents | |
  |  +---------+----------+  +---------+----------+  +---------+----------+  +---------+---------+ |
  |            |                       |                       |                       |           |
  |            +-----------------------+-----------+-----------+-----------------------+           |
  |                                                | Model Context Protocol (MCP)                  |
  |                                                v Stdio JSON-RPC Transport                      |
  +------------------------------------------------------------------------------------------------+
                                                   |
[VIEW 2: ZERO-EGRESS NODE.JS LAUNCHER & PROTOCOL DISPATCH]
  +------------------------------------------------------------------------------------------------+
  |  npx open-compute-mcp (Node.js Thin Bridge, CommonJS / RunAsInvoker)                           |
  |  - stdio pipe isolation (0 network sockets, zero telemetry egress)                            |
  |  - Environment sanitizer & launch parameter resolver (OPEN_COMPUTE_MCP_CMD / uvx)            |
  |  - Version parity guard (0.1.0-alpha.20 manifest sync)                                         |
  +------------------------------------------------+-----------------------------------------------+
                                                   | Stdio Process Stream
                                                   v (uvx --from git+https://github.com/ellmos-ai/...)
[VIEW 3: PYTHON OPEN-COMPUTE SOVEREIGN ENGINE]
  +------------------------------------------------------------------------------------------------+
  |  open-compute Python Engine (@main)                                                            |
  |  +---------------------------+  +---------------------------+  +-----------------------------+ |
  |  | Vision & Perception       |  | Input & Semantic Tree     |  | Signal Overlay & Voice      | |
  |  | - capture (mss / WGC GPU) |  | - click_name (UIA tree)   |  | - signal_show (glowing UI)  | |
  |  | - coords / scale factors  |  | - invoke (pattern action) |  | - signal_abort (hotkey)     | |
  |  | - one-shot observation_id |  | - do (mouse / keyboard)   |  | - talk (push-to-talk WAV)   | |
  |  +---------------------------+  +---------------------------+  +-----------------------------+ |
  |                                                |                                               |
  |  +---------------------------------------------v---------------------------------------------+ |
  |  | Operator Safety Ceiling & Execution Guard (Fail-Closed)                                   | |
  |  | - OC_SAFETY_MODE (confirm / read_only / allow_all)                                         | |
  |  | - OC_DENY (hard action deny list) | One-Shot Observation Token Enforcement                   | |
  |  +-------------------------------------------------------------------------------------------+ |
  +------------------------------------------------+-----------------------------------------------+
                                                   | Direct OS Windows Handles
                                                   v
[VIEW 4: HOST DESKTOP RUNTIME & ISOLATION BOUNDARY]
  +------------------------------------------------------------------------------------------------+
  |  Interactive Windows Desktop Session (Win32 / UIA / DirectX Graphics)                          |
  |  - Desktop Screen Buffer (multi-monitor, DPI scaling, GPU acceleration)                        |
  |  - Native Windows UI Automation Tree (exact-first element resolution)                          |
  |  - User-Mode Isolation Boundary (RunAsInvoker INV-PROC-07, zero kernel drivers)                |
  +------------------------------------------------------------------------------------------------+

This package is a thin launcher. It contains no server logic — it spawns the Python open-compute server (pulled from GitHub) and pipes MCP stdio through. Real screen capture and input require the interactive Windows desktop session.

1. State-bound Perception & Window Targeting: Captures/trees return one-shot observation IDs; window enumeration returns stable window/process IDs and issued tokens. WGC remains the GPU-window fallback.
2. Fail-closed Action Execution: Coordinates consume one observation and exact window binding; UIA names resolve exact-first; text is segmented with focus checks and character-count postconditions.
3. Leased Signal Overlay & Abort Control: The glowing border/cursor signal has owner/session metadata, a bounded TTL, turn-end cleanup, and immediate human abort.
4. Multimodal Collaboration & Voice Notes: Push-to-talk voice recording (talk), screen chat messaging (chat), directory monitoring (watch_dir), and macro replay (rec_replay).

• Python 3.10+ and uv on the host. The default launch uses uvx to fetch open-compute (with the mcp extra) from GitHub on first run — the mcp extra tracks the GitHub repo, so this works regardless of PyPI release timing.
• Windows for real capture/input (mss + UIA). Other platforms import the tools but cannot drive a desktop.

Via this npm launcher (npx):

{
  "mcpServers": {
    "open-compute": {
      "command": "npx",
      "args": ["-y", "open-compute-mcp"]
    }
  }
}

Directly via Python (uvx), no npm:

{
  "mcpServers": {
    "open-compute": {
      "command": "uvx",
      "args": ["--from", "open-compute[mcp,local,uia] @ git+https://github.com/ellmos-ai/open-compute.git", "open-compute-mcp"]
    }
  }
}

All coordinates are normalized 0..1 relative to the virtual desktop. Tool descriptions are localized in six languages (de/en/es/ja/ru/zh) via OC_LANGUAGE.

do also accepts the hold primitives mouse_down / mouse_up / key_down / key_up for press-and-hold sequences (rubber-band selection, modifier-held clicking, game input); anything still held is released when the server stops. capture(window=...) falls back to Windows.Graphics.Capture when a plain grab of a hardware-composited window (Roblox Studio, Blender, a GPU-accelerated browser) comes back all-black — install the wgc extra for that.

A vision model is billed per pixel, and every frame stays in the conversation, so a full-HD grab is charged again on each following request. The cost of a session therefore grows with the square of the number of screenshots, not linearly.

Because open-compute's coordinates are normalized 0..1, shrinking the image costs nothing in click accuracy — do works in fractions of the image either way. Only legibility drops, and at 0.5 buttons and field borders stay clearly identifiable; small body text is what gets hard to read.

The Python library itself defaults to full resolution — its callers are not necessarily paying per pixel. Only this launcher, which exists to serve agents, opts into the smaller frame and prints a one-line notice when it does.

What saves more than any scale factor: prefer tree where the accessibility model carries the content — note that in browsers it usually exposes only the browser chrome, not the page; and use capture(window=…) rather than the full desktop. Coordinate actions deliberately follow observe → one action → automatic refresh; do not batch multiple coordinate steps against one stale frame.

The v0.8 Python engine enforces observe → one action → automatic refresh. Keep the full descriptor or window_token from list_windows, then pass it as expected_window together with the latest observation_id from capture or tree. click_name/invoke require that issued window too. Reuse, changed state, focus mismatch, covered windows, and ambiguous UIA targets are rejected before input. type returns requested/sent character counts and complete/partial status without echoing the text. Signals have a hard TTL and are removed at action turn end unless keep_signal=true.

An explicit signal_show starts the engine's configured pre-action grace period. The static grace color is distinct from the mode color and the visible text counts down Start in N Sekunden once per second. At zero, both phase and color switch once to active. signal_status exposes the same phase, remaining seconds, current color, and screenreader label. Duration, grace color, and text template come from OC_SIGNAL_GRACE_SECONDS / OC_SIGNAL_CONFIG; 0 skips the countdown. The design uses no flashing, pulsing, or motion animation.

sequenceDiagram
    autonumber
    actor Reasoner as AI Reasoner (Claude / AGY)
    participant Launcher as Node.js Launcher (open-compute-mcp)
    participant Engine as Python Engine (open-compute)
    participant UI as Windows Desktop / UIA
    actor Operator as Human Operator

    Note over Reasoner,Operator: Phase 1: Visual Perception & State Inspection
    Reasoner->>Launcher: capture(window?) / tree()
    Launcher->>Engine: Forward stdio JSON-RPC
    Engine->>UI: Grab Screen (mss/WGC) or Read UIA Tree
    UI-->>Engine: Frame Image / Semantic Element Tree
    Engine-->>Launcher: Observation ID + normalized response/image
    Launcher-->>Reasoner: State-bound visual observation

    Note over Reasoner,Operator: Phase 2: Signal Overlay Activation
    Reasoner->>Launcher: signal_show(mode="control")
    Launcher->>Engine: Invoke Signal Overlay
    Engine->>UI: Render static grace color + Start in N seconds
    UI-->>Operator: Text countdown + accessible window name
    Engine->>UI: At zero, switch once to the mode color

    Note over Reasoner,Operator: Phase 3: Action Request & Safety Gate
    Reasoner->>Launcher: do(one action, window token, observation_id) / click_name(target)
    Launcher->>Engine: Process Action Payload
    alt OC_SAFETY_MODE == "confirm" (Default)
        Engine-->>Launcher: Status "needs_confirmation" (Report Only)
        Launcher-->>Reasoner: Human confirmation needed
    else OC_SAFETY_MODE == "allow_all" (Isolated VM)
        Engine->>UI: Execute Mouse/Keyboard / Hold Primitives
        UI-->>Engine: Action Completed
        Engine-->>Launcher: Post-observation + window/modal/text postconditions
        Launcher-->>Reasoner: Action completed - old observation invalid
    end

    Note over Reasoner,Operator: Phase 4: Emergency Abort or Completion
    opt Operator Triggers Emergency Abort
        Operator->>Engine: Hotkey Pressed (Abort Signal)
        Engine->>UI: Auto-release all held keys/mouse buttons
        Engine-->>Reasoner: signal_abort message returned
    end
    Engine->>UI: Remove overlay on turn end/error/abort (unless keep_signal=true)

open-compute-mcp is architected for four technical personas across autonomous AI operations, system engineering, accessibility assurance, and multimodal human-in-the-loop workflows:

open-compute-mcp delivers a model-agnostic, safety-bounded bridge between LLM reasoners and the Windows desktop environment. The following matrix illustrates how open-compute-mcp compares with alternative desktop interaction patterns across 10 operational dimensions:

This MCP server is part of the ellmos-ai ecosystem — AI infrastructure, MCP servers, and intelligent tools.

Our partner organization open-bricks bundles AI-native desktop applications — a modern, open-source software suite built for the age of AI. Sibling suites include DevCenter, CodeBox, MethodenAnalyser, CleanMarkdown, and PDFtoPDFocr.

Computer-use is powerful. OC_SAFETY_MODE is an operator ceiling (confirm default · read_only · allow_all); a per-call mode can only tighten it, never loosen it. Because MCP stdio has no server→client confirm callback, confirm / read_only report an action without performing it. For interactive use, run in an isolated VM/session, set OC_SAFETY_MODE=allow_all, and let your client's tool-approval dialog be the human-in-the-loop. OC_DENY (comma-separated action types) is a hard deny list. Treat on-screen content as untrusted (prompt-injection risk).

Troubleshooting: do/click_name only ever return needs_confirmation and never act. That is the confirm ceiling working as designed under stdio MCP. Fix for interactive use: set "env": {"OC_SAFETY_MODE": "allow_all"} in the server registration and let the client's tool-approval dialog gate each action (do not auto-allow do/click_name/invoke there). The env change only takes effect when the server process (re)starts — an already-connected client keeps the old ceiling until it reconnects.

open-compute-mcp maintains a transparent Level 1 Software Bill of Materials (SBOM) ensuring comprehensive supply-chain hygiene and zero-copyleft isolation. Full details are documented in THIRD_PARTY_LICENSES.md and THIRD_PARTY_LICENSES.txt.

The launcher and spawned processes are certified to run entirely under standard user permissions (RunAsInvoker). They never request or require administrator or UAC elevation.

No GPL, AGPL, LGPL, or other copyleft-licensed dependencies are included or linked. All code is distributed under permissive MIT and BSD licenses.

The repository features a comprehensive automated test suite validating contract invariants, manifest parity, and repository hygiene:

# Run all automated tests
npm test

# Run repository hygiene and secret leakage checks
npm run test:hygiene

# Verify packaging integrity
npm pack --dry-run

All pull requests and commits are verified through GitHub Actions CI (.github/workflows/ci.yml) with automated matrix builds on Ubuntu, Windows, and macOS across Node.js 18.x, 20.x, 22.x, and 24.x.

open-compute-mcp adheres to multi-registry standard specifications:

Every commit and pull request must satisfy rigorous defensive quality gates:

1. Clean Working Tree: No untracked scratch files, temporary logs, or conflict copies in repository root.
2. UTF-8 Cleanliness: All documentation files encoded in UTF-8 without BOM or replacement characters (U+FFFD).
3. Plan D Lock Discipline: Fail-closed verification of multi-agent lock status (lock_status.py) prior to mutations.
4. Manifest Parity: Version, description, and repository URLs strictly harmonized across package.json, server.json, and glama.json.
5. Release Decoupling (T-20260920-167562623): Pfad A/B discoverability and hygiene runs strictly preserve the version field without unauthorized bumps.
6. 100% Automated Test Pass Rate: Full test suite passing across all platforms.

Distributed under the MIT License. See LICENSE for full legal text.

Part of the ellmos-ai family under the open-bricks open-source umbrella. Canonical attribution and dependency inventory are documented in NOTICE, THIRD_PARTY_LICENSES.md, and THIRD_PARTY_LICENSES.txt.

The software is provided free of charge as an open-source courtesy gift (Gefälligkeit / Schenkung gem. § 516 BGB). In accordance with § 521 BGB, liability is strictly limited to intentional misconduct (Vorsatz) and gross negligence (grobe Fahrlässigkeit).

The ellmos-ai / open-bricks team provides a binding commitment to acknowledge all security disclosures within 48 hours and deliver preliminary triage within 5 business days. Reports should be submitted to security@ellmos.ai, support@lukasgeiger.com, or security@open-bricks.org in accordance with SECURITY.md.