Phanthy Motus

中文文档 | Official Website

Give Embodied AI a Real Soul. PhanthyMotus is a next-generation, open-source framework and platform for Embodied AI Agents. Built upon a robust ROS2 foundation, it seamlessly bridges diverse sensor inputs with advanced robot execution. By enabling flexible integration of World Models, LLMs, and VLMs, PhanthyMotus transforms traditional hardware into soulful, intelligent assistants capable of perceiving, thinking, and acting independently in the real world.

Install and run with a single command:

curl -fsSL https://motus.phanthy.com/install.sh | sudo bash

Or specify a version:

curl -fsSL https://motus.phanthy.com/install.sh | sudo bash -s <tag>

The install script will automatically install Docker (if needed), pull the latest Agent Core image, and start the service.

Open http://<device-ip>:15678 to access the Web Dashboard.

Browse available versions and images at the Resource Center.

Deploy hardware drivers from phanthymotus-driver. Drivers automatically register with Agent Core on startup — no manual configuration needed.

See CONTRIBUTING.md for building and running from source code.

• Visual Orchestration — Drag-and-drop web dashboard for connecting devices, sensors, and AI models on a canvas
• MCP Data Bus — Unified Model Context Protocol interface for all hardware devices
• Driver-Inferred Topics — Output ROS2 topics are declared by drivers, not computed by the core. The canvas calls each driver's info action (passing instance_id for sensors or input_topic for processors) to get the exact topic path before the device starts, keeping all topic naming logic inside the driver
• Event-Driven Agent Loop — LLM-powered reasoning with multi-turn tool calling, driven by real-time sensor events
• ROS2 Integration — Native DDS bridge for seamless ROS2 topic relay and monitoring
• Pluggable Perception — Modular ASR/TTS stack with multi-instance support and local inference (Jetson)
• Web Dashboard — Real-time device monitoring, agent activity stream, and configuration — all from the browser

Editable source: docs/architecture.svg — re-export the PNG after changing it.

The platform runs a single sense → think → act loop:

Hardware → Driver·Sensor → Perception → Agent Loop → ActuCore → Driver·Actuator → Hardware

• Drivers (L1) — One MCP server per device. Every tool declares a type, and the Agent Core treats each type differently: sensor (data streams), actuator (executable actions), processor (data transforms), resource (static assets such as URDF). Sensor and actuator tools normally live in the same driver process — the diagram splits them by direction of data flow, not by deployment.
• Perception (L2, ports 15720 / 15721) — Turns raw streams into semantics: ASR, TTS, VLM captions, vision understanding, face recognition.
• ActuCore (L2, port 15730) — The execution-model side of the same layer, shipped in this repository as actucore/: VLA policies, navigation, grasping, locomotion, whole-body control. It is a card host, structurally identical to Perception — each execution model attaches as a processor card, so any model that takes a goal and emits motion commands plugs in the same way. It currently ships no cards; the models are chosen per robot. See actucore/README.md for the card contract.
• Agent Loop (L3, port 15678) — FastAPI + ros2_bridge.py: event collector, layered L1–L4 prompt, tool dispatch, ACP barrier, history compaction, steering / interrupt, task store, subagent manager, skills, memory.
• Two bypass lanes — The loop can call sensor tools directly, skipping perception; and it can drive actuator tools over MCP JSON-RPC directly, skipping ActuCore. Both are the common path for simple queries and one-shot commands.
• Web Dashboard — Subscribes to every DDS topic on the bus via /ws/bus/{topic}, and to the agent's decision stream via /ws/motus.

Hardware drivers are maintained in a separate repository: phanthymotus-driver.

The Agent Core is designed for continuous operation over days or months. The architecture separates real-time interaction from background intelligence:

┌─────────────────────────────────────────────────────┐
│                   Main Agent Loop                     │
│  • Only processes user interactions (ASR/message)    │
│  • Lean history → stable prefix caching (~90% hit)   │
│  • Uses memory_recall for on-demand context retrieval│
└──────────────┬──────────────────────┬───────────────┘
               │ spawn                │ memory_recall
               ▼                      ▼
┌──────────────────────┐   ┌──────────────────────────┐
│   User Task Subagent │   │    Memory Store (SQLite)  │
│  • Isolated context  │   │  • subagent_conclusions   │
│  • Full tool access  │   │  • chat_history (FTS5)    │
│  • Returns summary   │   │  • daily_summary          │
└──────────────────────┘   └──────────────────────────┘
               ▲
┌──────────────────────┐
│    BG Monitor Agent   │
│  • Sensor analysis   │
│  • Results → DB only │
│  • urgent=true → push│
└──────────────────────┘

Key design principles:

• Main agent stays lean — only user interactions enter the conversation history. Background monitoring conclusions are stored in the memory database, not pushed to the main thread.
• Memory recall on demand — memory_recall tool provides FTS-based retrieval from past conversations, subagent conclusions, and daily summaries. Both main agent and subagents can use it.
• Urgent interrupts only — background subagents only interrupt the main agent for safety-critical alerts (battery critical, hardware faults). Routine reports go to the database silently.
• Daily auto-summary — a scheduled subagent generates daily reports covering user interactions, task completion, anomalies, performance review, and skill discovery opportunities.
• Prefix caching optimized — stable system prompt (L1 + L2-static) is frozen per turn; dynamic status is minimal and placed in user messages to maximize LLM prefix cache hits.

The dashboard at http://<device-ip>:15678 provides:

Add sensors and actuators you need onto the canvas, connect them to the core Agent Loop, and the framework handles data flow and execution automatically. Build your embodied AI agent like stacking building blocks.

Live sensor data visualization — audio waveforms, battery status, 3D skeleton/point cloud, and more.

A multiInstance tool (ASR, TTS, OCR) does not have a fixed output topic. The driver infers it from the input topic the card is connected to — /remote_control/mic + asr → /remote_control/mic/asr — so the same tool on two cards publishes to two different topics, and a card's topic only exists once something has asked the driver (action: info with input_topic, a read).

Two rules follow, both of which were learned the hard way:

• A derived topic is only valid for the input it was derived from. The canvas records which input produced each answer and refetches when the graph changes underneath it (_revalidateDerivedTopics in web/js/canvas.js). Without that, re-pointing a TTS card from one source to another left it publishing to the old source's topic: the dashboard panel watched a topic nothing fed, and there was no sound.
• The saved layout is not a source of truth for them. The monitor dashboard resolves them itself (web/js/topic-derive.js), rather than depending on someone having had the canvas page open — which is why the ASR panel used not to be there until you visited the canvas.

Frontend tests: node --test "agent-core/web/js/*.test.mjs" (no dependencies).

Define the agent's identity, system prompt, and long-term memory directly from the UI.

Browse past agent sessions with full event traces and tool call results.

A community-driven Skill Marketplace where users share and discover skills. Browse and install skills contributed by others, or teach your robot new capabilities using natural language — no coding required.

Open Solutions from the top-left of the dashboard. A solution bundles everything that makes one robot work — canvas topology and per-card config, active skills, prompt files, and tasks — into one shareable package on the Resource Center marketplace.

• Save: pick which blocks to package. The canvas is mandatory; skills, each of the three prompt files, and tasks are optional. Only skills that are already published on the Skill Marketplace can be packaged, so recipients can actually install them.
• Load: Agent Core first checks that every required driver / perception / actucore image is installed (offering one-click install for images already in the local catalog), then lists exactly what will be overwritten before applying.
• Align versions (optional): tick it and each involved container is redeployed at the image tag recorded in the package before the solution is applied — only the tag is taken, the local registry is kept. Agent Core itself is never auto-aligned, since restarting it would abort the load; the dashboard shows the recorded core version so you can upgrade manually if needed.
• Secrets stay home: fields a tool declares sensitive (format: password or x-sensitive: true in its configSchema) are blanked during packaging and reported to the loading user as "needs configuration".

Cards reference devices by MCP server_name, not by the machine-local mcp-<timestamp> id, so a package loads onto a different robot of the same model.

Deploy and manage Agent Core and hardware driver containers from the dashboard.

All services run as Docker containers managed by a single docker-compose.yml at /opt/phanthy-motus/ on the target device.

1. Install: The install.sh script pulls the Agent Core image, extracts the initial docker-compose.yml from the image, and starts the service
2. Add drivers: When you deploy a driver via the Web Dashboard, Agent Core pulls the driver image, extracts its deploy/service.yml fragment, and merges it into the compose file
3. Unified orchestration: All containers (core, drivers, perception, actucore) are managed by the same compose file with docker compose up -d

All driver, perception and actucore containers run with privileged: true and /dev:/dev mounted to access hardware devices (cameras, USB, GPIO). Network is set to host mode for ROS2 DDS communication.

# Example: how a deployed service looks in /opt/phanthy-motus/docker-compose.yml
services:
  agent-core:
    image: registry/core:tag
    network_mode: host
    ipc: host
    pid: host
    privileged: true
    volumes:
      - /dev:/dev
      - /opt/phanthy-motus/data:/work/resource
    ...
  unitree-g1:
    image: registry/drivers/unitree/g1:tag
    network_mode: host
    ipc: host
    pid: host
    privileged: true
    volumes:
      - /dev:/dev
    ...

Hardware driver ports are documented in phanthymotus-driver.

Every container declares log rotation in its deploy/service.yml (or compose fragment): the local driver, max-size: 10m, max-file: 3 — so ~30 MB per container, compressed. Agent Core injects that same policy as a default when a driver image's fragment omits it.

truncate -s 0 on /var/lib/docker/containers/<id>/**/*.log corrupts the log. The file size is reset but the Docker daemon keeps its write offset, so the next write lands past the new end-of-file and the kernel fills the gap with NUL bytes. docker logs then fails outright:

Error grabbing logs: invalid character '\x00' looking for beginning of value   # json-file
Error grabbing logs: error unmarshalling log entry: proto: illegal tag 0       # local

Once that happens the log is unreadable until the file is replaced. A truncate_log.sh helper used to live in deploy/ and was removed for exactly this reason.

Containers started outside the compose/service.yml paths inherit the daemon default, which for json-file is unbounded. Set a floor in /etc/docker/daemon.json so nothing can escape rotation:

{
  "log-driver": "local",
  "log-opts": { "max-size": "10m", "max-file": "3" }
}

Applying this requires restarting the Docker daemon, which stops every container on the host — schedule it rather than doing it mid-session.

The platform can optionally connect to a Resource Center for:

• Browsing and deploying pre-built driver/perception images
• Managing skills and extensions
• Publishing and installing solutions (canvas + skills + prompt + tasks bundles)
• OTA updates

Configure via the RESOURCE_CENTER_URL environment variable.

System hooks provide instant, bypass-LLM actions for time-critical responses. Drivers declare hook bindings via x-hooks in their MCP tool schema; Agent Core fires them directly on system events without waiting for LLM or ACP barrier.

System Event (ASR arrives / LLM starts / error)
  → Agent Core hooks.fire("on_thinking")
  → call_tool_direct() to driver (bypasses barrier + ACP)
  → Driver executes immediately (LED effect, interrupt, etc.)

POST /api/hooks/fire  {"hook": "on_interrupt_all"}
GET  /api/hooks       # list all registered hooks

See phanthymotus-driver/README_dev.md for driver implementation guide.

See CONTRIBUTING.md for development setup, architecture details, and guidelines.

Pull requests can be built and reviewed automatically by commenting /request_bot_review on the PR — see PR_REVIEW_AGENT.md for what it does, how to run it, and its dashboard.

Apache License 2.0