Status: Shipped proxy/control plane, model observability, per-project model pins, review hardening, and native toolbox management for the kept serving backends (ds4/gufo/llama_cpp; v2 L1 purge — see Toolbox Backend Management); the benchmark explorer ships behind the non-default benchmarks cargo feature (v2 R2); future work is explicitly separated below
Language: Rust
Binary: target/release/brainrouter
Implementation baseline: 3691dcc9ee7dee98f8355328d7e2e75f99faafdb (2026-09-07); extended through 32935d1 by the ai-toolbox-cockpit integration rollout, gufo overlay backend, dashboard navigation redesign, per-project pins, and review hardening — see Toolbox Backend Management and docs/design/ai-toolbox-cockpit-integration.md
Config: $XDG_CONFIG_HOME/brainrouter/brainrouter.yaml or ~/.config/brainrouter/brainrouter.yaml; serve --config overrides it. Service environment files supply credentials; the daemon resolves configured environment-variable names.
Repository: https://github.com/ajaxdude/brainrouter
Validation: 391 Rust tests and 38 Node browser-logic tests passed at this baseline; see Validation
- Problem Statement
- Solution
- Architecture Decisions
- Component Map
- Configuration Reference
- Routing Flow
- Review Flow
- Toolbox Backend Management
- Benchmark Explorer
- Model Observability and Regression Alerts
- Bridge Architecture
- HTTP API Reference
- MCP Tools Reference
- Harness Compatibility
- Failure Modes and Mitigations
- Security Model
- What This Is Not
- Validation
- Planned and Unshipped Work
A developer running multiple LLM subscriptions (Anthropic, OpenAI/Copilot, Google, Mistral) and local models constantly hits:
- Rate limits and quota exhaustion -- one provider fails mid-session, work stops.
- Model selection overhead -- manually deciding cloud vs local before every query burns time.
- Review token waste -- every code review cycle through a cloud LLM costs premium quota.
- Harness fragmentation -- each tool (omp, vibe, claude, opencode, codex, droid) has its own provider config; keeping them aligned is manual work.
- Role confusion -- the main author, reviewer, and local subagent pool need independent choices without silently overriding explicitly selected models.
- Unreliable comparisons -- performance numbers lose meaning without artifact, runtime, hardware, workload, configuration, and measurement provenance.
- Operational blind spots -- current request activity, completed-stream measurements, and historical benchmarks need a common inspection surface without treating missing data as zero or adding load to inference.
A single Rust daemon that:
- Separates routing roles. The persisted Main choice handles
auto,brainrouter/auto, and empty model selections; Reviewer has its own local/cloud/default choice; onlysubsorbrainrouter/subsuses the separately selected local pool. Named presets configure compatible combinations without merging these roles. A per-project pin store can override Main, Reviewer, and Subagent for one canonical Git repository root while unpinned projects inherit the global profile. - Preserves explicit choices and fallback semantics. Bare model IDs and
brainrouter/<id>select that local model with no rewrite or model substitution on failure.cloud/<id>sends the exact ID to Manifest. Explicitlocal/cloudaliases select backend defaults, not the Main override. Cloud requests retain the existing local fallback policy on disabled/unavailable Manifest; errors after streaming begins surface to the client rather than replaying output. - Reviews code with runtime policy controls, with explicit local or cloud reviewer IDs, requested-versus-actual reporting, and a reviewer snapshot retained for the lifetime of each review and its continuations. Runtime state can disable review, enable PR-generation guidance injection, and enable HankNDory design-aware review. Review prompts are token-budgeted before dispatch and escalate instead of approving when required evidence is truncated.
- Manages system state via the dashboard: one-click upgrades of llama-swap, resets of the llama.cpp toolbox, start/stop of the Bonsai classifier, and flushing of loaded models — full VRAM control without a terminal. This extends to native, dashboard-driven management of ds4/gufo toolboxes, models, and detached servers (halogen/vllm/r9v were v1 scope, removed by the v2 L1 purge; their catalog feed ids parse as browse-only data) — brainrouter is a native Rust webui for
ai-toolbox-cockpit, sharing its catalog and config.json rather than a divergent brainrouter-only store (see Toolbox Backend Management). - Keeps opt-ins explicit. Bonsai and Manifest are disabled by default. A role set to Auto consults the classifier only when enabled; otherwise it selects local. Applying a cloud or hybrid preset does not enable Manifest.
- Presents a single OpenAI-compatible endpoint to all harnesses, plus an Anthropic-compatible endpoint for harnesses (Claude Code, droid) that speak Anthropic's protocol natively.
- Bridges chat platforms -- Discord and Signal ship as the
bridgesidecar plugin (v2 L2), shelling out toompCLI, bringing LLM access to messaging apps with session management, model selection, and working directory tracking. Commands use the!brprefix. - Presents a simplified dashboard IA. The primary dashboard destinations are Overview, Models, Benchmarks, and Config; an always-visible command strip exposes model scope/role controls, flush/restart/sync actions, health, and quality posture.
- Installs itself into harnesses via an idempotent
installsubcommand that configures 7 harnesses (omp, vibe, opencode, codex, droid, claude, pi) in a single command. - Tracks in-flight requests live. A registry records every request at handler entry (before routing, so the dashboard sees it during model load) and holds it until the stream ends or the user cancels it. The dashboard shows one row per active request — elapsed time, model, user agent, address, session, bytes received, PP progress, and an activity label (tool calling / reasoning / asking a multiple choice question / generating) — with a per-row Cancel button.
- Measures completed requests by unique event ID. Actual first-output timing and provider-reported token counts support bounded recent measurements across both wire protocols. Concurrent turns in the same conversation cannot overwrite each other's samples. Missing prompt-processing speed is not inferred from TTFT.
- Explores imported benchmarks. A normalized SQLite registry, strict Rust boundary models, immutable identities, transactional previews/imports, deterministic matrix expansion, structured inspector, telemetry plots, and bounded CSV/JSONL exports are exposed at
/benchmarks. - Observes model activity without running benchmarks.
/modelscombines exact local model keys, read-only backend metadata, active requests, recent completed measurements, indicative regression warnings, and explicit benchmark references. - Isolates optional work. Unavailable benchmark storage disables only explorer routes. Bounded blocking workers and response budgets keep benchmark work off the async routing executor. No benchmark runner, scheduler, automatic model switching, or outbound alerts are part of these additions.
The configured Bonsai model runs as an external llama-server process (PrismML fork of llama.cpp) on a dedicated port (default 9200). Brainrouter launches it at startup only when bonsai.enabled: true, waits for /health, then classifies through /v1/chat/completions. Decisions include cloud, local/light, and local/deep. Timing depends on hardware and workload; there is no guaranteed classification-latency SLA. BonsaiControl owns start/stop. An Auto role skips classification and chooses local when the classifier is disabled; enabling Bonsai does not change a Main role already pinned to local or cloud.
Bonsai was chosen specifically because it is purpose-trained to understand task complexity and model capability -- not as a general assistant. Using it only for routing preserves VRAM for llama-swap models.
Brainrouter connects to one OpenAI-compatible Manifest endpoint and one configured API-key environment variable. Default cloud selections send model: "auto"; explicit cloud role IDs and cloud/<id> pass the provider ID unchanged. Manifest selects the downstream cloud provider. The actual model is taken from reported response metadata, not assumed from the requested name. Manifest is off by default; cloud-disabled requests preserve local fallback behavior and expose that outcome.
The coupling is minimal: if Manifest is replaced with LiteLLM or OpenRouter, one URL in brainrouter.yaml changes.
POST /v1/chat/completions-- OpenAI format. Covers vibe, opencode, codex, omp, pi (via extension).POST /v1/messages-- Anthropic Messages API format. Covers Claude Code (viaANTHROPIC_BASE_URL) and droid (viaprovider: "anthropic"in custom_models).
Internally the request is immediately translated to OpenAI format and routed through the same Bonsai / Manifest / llama-swap pipeline. Response is translated back to Anthropic SSE events at the edge. Proxy responses are streaming SSE; non-streaming provider completion APIs and OpenAI Responses endpoints are not implemented.
The brainrouter mcp process does not load Bonsai and does not run the review loop. It is a JSON-RPC stdio server that maps four tool calls to HTTP requests against the daemon's UDS. This keeps harness cold-start fast (no model load) and keeps all state -- sessions, health tracker, circuit breakers -- in one place. Both mcp and the CLI share one DaemonClient (src/daemon_client.rs) so there is exactly one HTTP client path into the daemon.
brainrouter cli uses DaemonClient over UDS by default or TCP with --url. It covers core management, routing profiles, catalogs, review configuration, and review lifecycle. It does not start a second inference service. Review requests poll every five seconds with a 30-minute cap, or return immediately with --async. Benchmark and observability workflows use the documented HTTP API or their pages; dedicated CLI subcommands for those workflows are not shipped.
When an agent requests a review, the service snapshots its Reviewer choice and iteration settings. The loop uses Router::route_with_choice, independently of the Main and subagent choices. This means:
- Explicit local/cloud reviewer IDs are honored; an Auto reviewer follows classifier behavior, not the current Main override.
- If Manifest is down, review calls fall back to llama-swap automatically.
- The same provider clients, circuit breakers, and streaming protections are reused.
- Changing global preferences affects new reviews, not the reviewer of an existing session or continuation.
src/project_pins.rs persists optional {main, reviewer, subagent} overrides keyed by the canonical absolute Git repository root of a request cwd. The store is loaded from project_pins.json beside the default config state, writes with owner-only atomic replacement, and treats missing/corrupt/unreadable files as empty so every project inherits the global profile by default. A lock-free has_pins flag keeps the proxy hot path from doing filesystem work when the store is empty or when the client sent a selector that never consults a pin.
Role precedence is deliberately role-specific: Main uses explicit request model > project pin > global profile > auto for auto/empty selectors; Reviewer injects a reviewer pin into a fresh review's config snapshot before admission; Subagent consults the pin only for subs/brainrouter/subs. GET /api/project-pin?path=... normalizes a raw path for the dashboard, while POST is a strict full replacement (path, main, reviewer, and subagent all present; null means inherit) and DELETE removes by normalized key.
Two independent circuit breakers: one for manifest, one for llama-swap. Three failures within a window open the breaker; 60-second cooldown before retry. This prevents hammering a degraded provider while allowing automatic recovery.
The adapter translates content/tool events and protocol termination, handles split lines, and reports stream failures. It drains final usage, [DONE], and clean EOF before final Anthropic terminal events so measurement capture is not abandoned at finish_reason. Tail waiting is bounded to two seconds from the first finish/DONE marker, with additional 256 KiB total, 64 KiB line, and 1,024 subsequent-chunk budgets. Content continues streaming; invalid, cancelled, errored, or incomplete tails do not produce successful performance samples.
Destructive operations (upgrade, restart) are restricted to loopback interfaces (127.0.0.1, ::1) or Unix Domain Sockets. Browser-originated requests to management endpoints are validated against Origin/Referer headers. Working directory tracking for sessions includes absolute-path enforcement, null-byte rejection, and path-traversal component blocking.
When enabled on a managed local route, the prompt rewriter replaces verbose harness instructions with a lean local system prompt and anti-loop directives. It is off at daemon startup. Direct local IDs, including explicit local role models, bypass rewriting; not every auto alias necessarily reaches a managed local route.
The rewrite is coupled to the classifier server-side, not just in the UI: turning Bonsai off via the toggle/CLI forces the rewrite flag off with it (the invariant "rewrite on ⇒ Bonsai on" holds in the daemon), and POST /api/prompt-rewrite refuses to enable while Bonsai is down. Direct model picks are never rewritten — only managed local routes.
The Discord and Signal transports ship as the bridge plugin (v2 L2): a sidecar binary (brainrouter-bridge) supervised by the daemon and spoken to over a per-plugin Unix-domain socket (newline-delimited JSON-RPC 2.0). They do not call LLMs directly. They shell out to the omp CLI as a subprocess, which connects back to brainrouter through the normal proxy path. This means bridge conversations get the same routing, fallback, and review capabilities as direct harness usage. Session state, working directories, and model aliases are persisted to disk per-transport in the plugin state dir.
The MCP server needs to know the caller's working directory. On Linux, brainrouter resolves this by mapping the peer's socket connection (TCP or UDS) back to a PID via /proc, then reading /proc/<pid>/cwd. This is a Linux-native approach that avoids requiring the caller to pass CWD explicitly, though callers may override it.
src/escalation/templates/main_dashboard.html organizes the main dashboard as four primary destinations: Overview, Models, Benchmarks, and Config. Models has internal tabs for Local models, Downloads, and Serving plus a link to /models. The command bar contains the frequent controls: project/global scope selector, Main model controls, reviewer/subagent expander, Flush Models, Restart Local Stack, Sync Models → OMP, a /api/service-health health pill, and a read-only posture line for nudge/review/HankNDory/PR state. Set-and-forget quality toggles, plugins (status, config, logs), toolbox inventory, service status/restarts, and raw config editors live in Config. The Plugins destination (v2 L2) lists installed plugins with state and set_status lines, enables/disables them, edits plugin config, and shows HTML-escaped plugin logs.
The registry gives the dashboard a live view of every in-flight request — the "monitor LLM activity without opening llama-swap" goal. Each request registers an entry at handler entry, before route_tagged, so the row is visible during model load (when the registry is otherwise idle). The entry's identity (InflightHandle) is an Arc carried by the spawned routing task and, once routing resolves, by a SniffStream that wraps the provider's SSE stream. The row is removed when the last handle drops — i.e. at stream end, on a routing error (no stream is created, so the task-side handle is the last one), or on an explicit dashboard Cancel (a watch channel flips a flag; the SniffStream ends the stream immediately and the handles drop). Stale entries (older than a grace window) are swept on read.
The activity label is derived by sniffing the head window of each SSE chunk for provider-specific markers (tool_calls / toolUse → tool calling, reasoning_content / thinking → reasoning, multiple choice → asking, otherwise generating). The sniff is sticky with a priority order — tool calling > reasoning > asking > generating — so a strong signal is never downgraded by a later weaker one.
model is an interior-mutable Mutex<String>: it starts as the requested model and is updated to the resolved llama-swap model key after routing.
A stream wrapper records successful measurements only after [DONE] and clean EOF, keyed to the unique ID returned by RoutingEvents::emit. It never assigns data to the latest event merely because it shares a conversation. Measured TTFT ends at the first nonempty content/reasoning/tool-argument frame; role-only frames and heartbeats do not count. Generation TPS uses (completion_tokens - 1) / (last output time - first output time) only when reported usage and distinct observations support it. It is an estimate affected by buffering/backpressure, not engine decode instrumentation. PP throughput is not fabricated. See Model Observability and Regression Alerts for retention, attribution, and comparability requirements.
| Component | File | Responsibility |
|---|---|---|
| Entry point | src/main.rs |
Clap subcommand dispatch |
| Daemon startup | src/daemon.rs |
Load config, create all services, validate environment, start server |
| HTTP server | src/server.rs |
Route /v1/*, /review/*, /api/*, and /dashboard |
| Classifier | src/classifier.rs |
Bonsai-based cloud/local decision |
| Bonsai server lifecycle | src/bonsai_server.rs |
BonsaiServer: spawn, 60s health poll, SIGTERM→SIGKILL stop. BonsaiControl: runtime start/stop from the dashboard, shared enabled flag for the classifier |
| Router | src/router.rs |
Dispatch to Manifest or llama-swap; fallback; timeout |
| Routing preferences | src/routing_profile.rs |
Typed role choices/presets, strict writes, atomic per-user state, legacy migration |
| Project pins | src/project_pins.rs |
Per-Git-repository role pins, canonical key resolution, atomic owner-only project_pins.json |
| Prompt rewriter | src/prompt_rewriter.rs |
System prompt rewriting for local mode (strips OMP bloat, injects anti-loop prompt) |
| Anthropic shim | src/anthropic.rs |
/v1/messages to /v1/chat/completions translation; strict SSE state machine |
| MCP server | src/mcp_server.rs |
JSON-RPC stdio; forwards to daemon over UDS |
| Installer | src/install.rs |
Idempotent harness config merger for 7 harnesses |
| Session store | src/session.rs |
In-memory HashMap<id, Session> behind Mutex |
| Config | src/config.rs |
YAML parsing + validation |
| Types | src/types.rs |
OpenAI request/response structs |
| Health tracker | src/health.rs |
Per-provider circuit breaker |
| Stream wrapper | src/stream.rs |
TimeoutStream: lazy-armed 180s inter-chunk stall detection; SafeStream: error-to-SSE converter |
| Routing events | src/routing_events.rs |
In-memory event buffer for dashboard live feed |
| Benchmark store/domain | src/benchmark.rs |
Validation, canonical identities, SQLite ingestion/query/detail/export, shared admission, SpiderQuery/spider_chart() for the radar chart |
| Benchmark HTTP/imports | src/benchmark/http.rs, src/benchmark/imports.rs |
Bounded workers/bodies/responses, rollback previews, reusable templates and examples |
| Benchmark schema | migrations/0001_benchmark_explorer.sql, migrations/0002_benchmark_lab.sql, migrations/0003_toolbox_serving_dimension.sql |
Versioned normalized STRICT tables, indexes, summary view; serving_runtimes/toolbox_catalog_snapshot for backend/engine dimensions. Pre-existing gap (recorded, out of patch scope): the frozen 0003 serving-runtime CHECK list omits gufo; no gufo serving-runtime rows are written by current code |
| Toolbox catalog | src/toolbox_catalog.rs, src/toolbox_catalog/{types,models,schema_validate,gufo_overlay}.rs |
Typed, vendored ai-toolbox-cockpit catalog plus gufo overlay (CatalogBackendId/SupportedServingBackend/ModelPayload) + structural validator |
| Vendored catalog data | assets/cockpit-catalog/{toolboxes,models}.json, SOURCE |
Pinned upstream catalog snapshot, refreshed by weekly CI sync |
| Catalog sync tooling | scripts/sync-toolbox-catalog.sh, .github/workflows/sync-toolbox-catalog.yml, src/bin/toolbox_catalog_check.rs |
Weekly PR-based catalog refresh + standalone structural-validation binary |
| Cockpit config bridge | src/cockpit_config.rs |
Read (and explicit single-write "apply") of cockpit's own ~/.config/ai-toolbox-cockpit/config.json |
| Toolbox ownership sidecar | src/managed_toolboxes.rs |
Owner-only managed_toolboxes.json with podman container IDs for brainrouter-created/adopted toolboxes |
| Model downloads | src/model_downloads.rs |
Native download job registry for ds4/gufo/llama_cpp catalog models plus HF CLI preflight (v2 L1: halogen/r9v payload assembly and the Prepare-PLE job kind removed) |
| Server Mode | src/server_mode.rs |
Headless detached podman run servers for ds4/gufo: create/start/stop/status (v2 L1 purge: halogen/vllm/r9v builders/routes removed) |
| Serving-identity registry | src/serving_identity.rs |
Read-only {toolbox_backend, compute_api, runtime_profile_id, endpoint} bookkeeping for running Server Mode containers |
| Model observability | src/observability.rs |
Read-only polling, completed measurements, rolling alerts, explicit reference settings |
| Explorer pages | src/escalation/templates/benchmarks.html, src/escalation/templates/model_observability.html |
Embedded benchmark workflows and local model activity |
| Inference state | src/inference_state.rs |
Track active inference status per provider |
| Peer CWD | src/peer_cwd.rs |
Linux-native PID/inode mapping for directory discovery (IPv4/IPv6/UDS) |
| Lib | src/lib.rs |
Crate-level re-exports |
| Review service | src/review/mod.rs |
start_review, per-project reviewer-pin injection, runtime enable/disable, resolve_session |
| Review loop | src/review/review_loop.rs |
Iterative LLM review, admission, token budget, design-aware review, robust JSON parsing |
| Context gatherer | src/review/context.rs |
PRD auto-detect, git diff HEAD, safe UTF-8 truncation |
| Prompt builder | src/review/prompt.rs, src/review/tokens.rs |
Review prompt template and conservative token-aware budget planner |
| Escalation UI | src/escalation/mod.rs |
/review/* HTTP handlers + embedded HTML templates; CWD sanitization |
| Escalation templates | src/escalation/templates/ |
Embedded HTML for review session UI |
| In-flight registry | src/inflight.rs |
InflightRegistry + InflightHandle + SniffStream: per-request live rows (elapsed, model, UA, address, session, bytes, PP, activity) with dashboard cancel; sticky head-window activity sniff |
| Provider adapter | src/provider/mod.rs |
Provider trait and common types |
| OpenAI provider | src/provider/openai.rs |
OpenAI-compatible HTTP client with fault-aware circuit breaking (429/5xx) |
| Plugin core | src/plugins/ |
Manifest, registry, supervisor, per-plugin UDS JSON-RPC (v2 L2) |
| Credentials store | src/credentials.rs |
credentials.json (0600), resolve(), grant model (v2 L2) |
| Migration framework | src/migration.rs |
Snapshot-first boot migrations + named restore (v2 L2) |
| Bridge plugin core | plugins/bridge/src/core.rs |
Shared transport logic: OMP subprocess management, message chunking, aliases |
| Bridge plugin persistence | plugins/bridge/src/persist.rs |
Session, model alias, and working directory persistence to JSON files |
| Discord transport | plugins/bridge/src/discord/mod.rs |
Serenity-based Discord bot with channel-scoped sessions |
| Signal transport | plugins/bridge/src/signal/mod.rs |
Signal CLI polling-based transport with group support |
| Integration tests | tests/anthropic_shim_test.rs |
Anthropic protocol compliance tests |
| Integration tests | tests/failover_test.rs |
Circuit breaker and fallback behavior tests |
| Integration tests | tests/install_test.rs |
Harness installer tests |
| Integration tests | tests/review_test.rs |
Review loop and session lifecycle tests |
| Role/upgrade tests | tests/routing_profiles_test.rs, tests/daemon_availability_test.rs |
Exact roles, protocols, persistence, legacy upgrades, failure isolation, complete import/reference workflow |
| Browser-logic tests | scripts/test-benchmark-ui.cjs |
Synthetic DOM/import/inspector/precision and asynchronous-state regressions |
| System installer | install.sh |
One-script Fedora system-wide installer: packages, bun, oh-my-pi, Bonsai, Manifest, llama-swap, brainrouter, toolbox container, shared /etc/brainrouter/ config, per-user services with linger |
When installed via install.sh on a shared Fedora machine:
| Path | Purpose |
|---|---|
/usr/local/bin/brainrouter |
System-wide brainrouter binary |
/usr/local/bin/llama-server-toolbox |
Wrapper: runs llama-server inside the llama-vulkan-radv toolbox container |
/etc/brainrouter/brainrouter.yaml |
Shared base config (readable by all aistack group members) |
/etc/brainrouter/env |
Shared env file holding MANIFEST_API_KEY (root:aistack, mode 640) |
/etc/skel/.config/systemd/user/brainrouter.service |
Template brainrouter service for new user accounts |
/etc/profile.d/ai-stack.sh |
Sets PATH, ANTHROPIC_BASE_URL, OPENAI_BASE_URL for all login shells |
/opt/models/bonsai/ |
Shared GGUF model storage (root:aistack, setgid 2775) |
/opt/ai/llama-swap/ |
llama-swap Docker compose stack |
/opt/ai/manifest/ |
Manifest Docker compose stack |
/etc/systemd/system/llama-swap.service |
System service: starts llama-swap Docker stack at boot |
/etc/systemd/system/manifest.service |
System service: starts Manifest Docker stack at boot |
| Field | Type | Default | Description |
|---|---|---|---|
enabled |
bool |
false |
Cloud routing is off by default. When false, route_cloud skips Manifest and falls back to llama-swap |
base_url |
String |
(required) | URL of the Manifest instance. Validated: must start with http:// or https:// |
api_key_env |
String? |
None |
Name of the environment variable holding the mnfst_* API key (NOT the key itself) |
| Field | Type | Default | Description |
|---|---|---|---|
base_url |
String |
(required) | URL of the llama-swap instance |
fallback_model |
String |
(required) | Model key to use when Manifest fails or Bonsai picks local. Must match a key in llama-swap config |
local_models |
Vec<String>? |
[] |
Explicit llama-swap model keys; a request using one of them as model= routes straight to llama-swap, bypassing Bonsai |
subs_model |
String? |
None |
Subs-pool model key; model=subs / brainrouter/subs routes here, bypassing Bonsai. Absent → subs falls back to auto |
local_system_prompt |
String? |
None |
Path to a custom system prompt file for model=local mode. Built-in lean prompt used if absent |
nudge.enabled |
bool |
false |
Thinking-budget nudge: inject reasoning_budget_tokens into local routes when the client didn't set one |
nudge.model_key |
String? |
None |
Local model key that receives the nudge; when None, nudge applies to the routing target |
nudge.budgets.light |
u32 |
10240 |
Budget injected when the tier is light. Legacy configs may use local (accepted via serde alias) |
nudge.budgets.deep |
u32 |
12288 |
Budget injected when the tier is deep |
| Field | Type | Required | Description |
|---|---|---|---|
enabled |
bool |
false |
Classifier is off by default. When false, auto routing goes straight to local (no cloud hop) |
model_path |
PathBuf? |
(required when enabled) | Configured Bonsai GGUF path; ${models_path} is expanded. Existence is checked only when enabled, so missing classifier assets do not prevent a local-only startup |
fork_path |
PathBuf |
$HOME/.local/share/brainrouter/llama-prism/llama-server |
Configured PrismML fork binary; do not assume a particular host's build or quantization |
server_port |
u16 |
9200 |
Port for the external Bonsai llama-server process |
| Field | Type | Default | Description |
|---|---|---|---|
path |
PathBuf |
/opt/models |
Shared GGUF model directory; ${models_path} inside bonsai.model_path expands to this value |
shared_write |
bool |
false |
When true, all aistack group members may add/delete models (dir mode 770); when false, only the owner can write (mode 750) |
| Field | Type | Default | Description |
|---|---|---|---|
max_iterations |
u32 |
5 |
Positive maximum LLM review iterations before escalation to human |
forced_mode |
String |
"local" |
Legacy YAML/default reviewer choice: auto, cloud, or local. Role profiles and saved preferences can override it |
forced_model |
String? |
None |
Exact local OR cloud provider ID; null selects that backend's default. New auto+model combinations are invalid |
hankndory_integration |
bool |
false |
YAML seed for design-aware review. Runtime /api/review/hankndory can override it without editing YAML |
design_doc_path |
String? |
None |
Repo-relative design doc under docs/design/ used when design-aware review is enabled |
Optional typed YAML defaults: {preset, main, reviewer, subagent_model}.
Main/Reviewer are {backend: auto} or {backend: local|cloud, model: null|"<explicit-id>"}.
Subagent is a local model ID or null. Presets are auto, cloud,
local_main_sub, local_custom, cloud_main_local_review,
local_main_cloud_review, and custom; role backends must match the preset.
local_custom requires an explicit local Main model. Applying a preset preserves
the independently selected subagent pool.
Without routing, Main defaults to local, Reviewer derives from review,
and the pool derives from llama_swap.subs_model. Per-user
$XDG_CONFIG_HOME/brainrouter/routing_state.json (otherwise under ~/.config)
takes precedence over YAML role defaults. Writes validate before committing
runtime state and use atomic owner-only files. review.max_iterations is
YAML-owned across restarts.
When the new state file is absent, valid legacy review_state.json overrides
migrate and persist once; the legacy file is not deleted. File reads alone
normalize the old, ignored forced_mode: auto + forced_model combination,
warning with the source path and removal guidance. New write APIs remain strict;
local/cloud choices and other errors are not silently normalized. Migration
errors identify their source/destination and cause.
Resetting to YAML requires moving aside both new routing state and any retained
legacy state while the daemon is stopped; removing only the new file can
trigger legacy migration again.
database_path is optional and defaults to
$XDG_DATA_HOME/brainrouter/benchmarks.sqlite3, or
~/.local/share/brainrouter/benchmarks.sqlite3. Use an explicit absolute path for
an override; no shell ~ expansion is performed on a configured path. Initialization
failure produces a logged explorer-only HTTP 503, not a core daemon startup failure.
Repair the path/database and restart to retry; newer schemas are never downgraded.
Alert policy and explicit baseline mappings live beside the selected config:
brainrouter.yaml -> brainrouter.observability.json (Path::with_extension).
The page/API performs revision-checked writes; copy the latest revision and full
policy from GET /api/observability/settings before changing it. Corrupt/unreadable
settings expose errors and disable saves until repaired/restarted, but leave
core routing and live observations available. Revision counters reset on restart;
clients must reload settings before writing.
The v1 bridge.* YAML keys are gone. Discord/Signal run as the bridge
plugin: install it with brainrouter plugin install plugins/bridge (from a
built dist: plugin.toml + brainrouter-bridge), enable with
brainrouter plugin enable bridge. The plugin reads its config — the v1
bridge.* keys minus the Discord token — via the config.get RPC
(namespaced per plugin, edited with brainrouter plugin config set bridge <key>=<value> or the dashboard Plugins tab; the per-transport pause is
discord.enabled / signal.enabled). The Discord token lives only in the
credentials store (migrated from v1 YAML at boot; id discord_token,
granted to plugin bridge) and reaches the plugin exclusively through the
grant-enforced secrets.get RPC. Signal keeps its direct llama-swap
listing via the live llama_swap_url config key (v1 parity). Persistence:
on first start the six v1 JSON files under ~/.local/share/omp-bridge/
are copied into the plugin state dir ($XDG_STATE_HOME/brainrouter/plugins/ bridge/, 0700); v1 originals are kept for revert. Restore the v1 config
state with brainrouter migration restore v2.0-p2.
| Argument | Default | Description |
|---|---|---|
--config |
$XDG_CONFIG_HOME/brainrouter/brainrouter.yaml, otherwise ~/.config/brainrouter/brainrouter.yaml |
Path to YAML configuration file |
--tcp-addr |
127.0.0.1:9099 |
TCP bind address |
--socket |
$XDG_RUNTIME_DIR/brainrouter.sock, otherwise /run/brainrouter.sock |
Unix domain socket path |
| Argument | Default | Description |
|---|---|---|
--socket |
Same runtime-directory default as serve |
UDS path to connect to the daemon |
Thin client for core management, roles, and reviews. --socket <path> selects the daemon UDS, --url http://… talks TCP instead. Output is JSON except raw YAML config commands. Explorer/observability workflows additionally use HTTP directly.
| Command | Description |
|---|---|
status |
Overall health: llama-swap, manifest, llama.cpp, toolbox, cloud-fallback |
versions |
Installed versions for all components + latest available |
inference |
Live inference status: loaded model, state, slot progress |
events |
Recent routing events feed |
stats |
Aggregated routing statistics |
models [--llama-swap] |
Model list (/v1/models view, or raw llama-swap keys with --llama-swap) |
bonsai status / enable / disable / toggle |
Classifier server control (idempotent: enable/disable are no-ops when already in that state) |
nudge status / enable / disable / toggle / tier <auto|light|deep> |
Thinking-budget nudge control (light = tight budget; legacy API spelling local still accepted) |
prompt-rewrite status / enable / disable |
Local prompt-rewrite toggle |
context status / set <tokens|auto> |
Legacy CLI only: validates 2048..262144 (auto = 131072), but /api/context has no server handler at this baseline and returns 404 |
routing-mode status / set <auto|cloud|local> |
Routing-mode override |
routing-profile status / models / preset / choose / pool / set |
Inspect/replace typed preferences, choose exact Main/Reviewer models, and configure the independent local pool |
plugin install <dir> / list / enable / disable / remove / status [id] |
Sidecar plugin management (v2 L2) |
plugin config set <id> k=v [...] / plugin config show <id> |
Plugin-namespaced config (v2 L2) |
migration restore <phase> |
Restore a leg snapshot (e.g. v2.0-p2) |
toolboxes |
List llama-* toolbox containers |
restart <llama-swap|llama-cpp|manifest|brainrouter> |
Restart a service. llama-cpp is the dashboard's llama.cpp row; toolbox is the toolboxes image |
upgrade <llama-swap|manifest|toolbox> |
Upgrade a component |
flush-models |
Unload every model from llama-swap (frees VRAM) |
sync-omp |
Push llama-swap models into OMP's models.yml (one-way) |
config-files |
List config files the daemon manages |
brainrouter-config show / set <path|-> |
Read brainrouter.yaml; write one from a file or stdin (-). Alias: config |
llama-swap-config show / set <path|-> |
Same for llama-swap's config.yaml |
review-config status / update [--max-iterations N] [--forced-mode auto|cloud|local] [--forced-model KEY|--clear-model] |
Current/new-session reviewer configuration; continuations keep the initial snapshot |
review list |
All review sessions |
review get <sessionId> |
One session's details |
review request <taskId> <summary> [--details TEXT] [--cwd DIR] [--async] |
Request a review; blocks with progress (polls every 5 s, 30-min cap) unless --async |
review continue <sessionId> |
Additional LLM review rounds (seeds from the persisted turn history) |
review lgtm <sessionId> |
Quick-approve a session |
review resolve <sessionId> <feedback> |
Resolve a session with feedback |
review get/continue/lgtm/resolve validate sessionId (alphanumeric + -/_) before URL use — path-traversal guard.
| Argument | Default | Description |
|---|---|---|
<harness> |
(required) | One of: omp, vibe, opencode, codex, droid, claude, pi |
--yes |
false |
Skip confirmation prompts |
--shell-rc |
false |
Boolean flag to append Claude environment variables to the shell RC |
--bin |
(none) | Path to brainrouter binary |
| Argument | Default | Description |
|---|---|---|
--log-level |
info |
Log verbosity. Overridden by RUST_LOG env var |
| Variable | Description |
|---|---|
RUST_LOG |
Log filter directive. Overrides --log-level |
HOME |
User home directory. Used for default paths |
XDG_RUNTIME_DIR |
Runtime directory for UDS (e.g., /run/user/$UID) |
XDG_CONFIG_HOME |
User YAML/config directory and routing_state.json location |
XDG_DATA_HOME |
Default benchmark database base directory; falls back to ~/.local/share |
BRAINROUTER_MANIFEST_DIR |
Override directory for Manifest configuration |
| (dynamic) | Whatever manifest.api_key_env names (e.g., MANIFEST_API_KEY) |
| Constant | Value | Description |
|---|---|---|
FAILURE_THRESHOLD |
3 |
Failures before circuit breaker opens |
COOLDOWN_PERIOD |
60s |
Circuit breaker cooldown before retry |
STREAM_STALL_TIMEOUT |
180s |
Maximum time between SSE chunks before timeout |
TTFT_TIMEOUT / deferred routing wait |
600s |
Provider/routing wait budget; not a promised first-output latency |
MAX_EVENTS |
500 |
Maximum routing events kept in memory for dashboard feed |
CLASSIFY_MAX_TOKENS |
10 |
Maximum tokens for Bonsai classification response |
USER_MSG_TRUNCATE |
800 chars |
Truncation limit for user message sent to classifier |
Discord chunk size |
1500 chars |
Maximum message length per Discord message |
Signal chunk size |
4000 chars |
Maximum message length per Signal message |
Signal poll interval |
3s |
Polling interval for Signal CLI message retrieval |
| Feature | Dependencies | Description |
|---|---|---|
default |
— | Core proxy + plugin subsystem only (v2 L2; the bridges left the core binary) |
benchmarks |
— | Benchmark explorer + native lab (v2 R2) |
The bridge-only dependencies (serenity, async-trait, serde_yml) now live in
the standalone plugins/bridge workspace, not the core.
Incoming request (OpenAI or Anthropic format)
|
v server.rs: deserialize, translate if Anthropic; extract_session_id()
| reads a client conversation header (x-omp-session / x-session-id /
| x-conv-id / x-conversation-id / x-client-session / x-request-conv)
| so the dashboard can group events per conversation
|
v router.rs: default/auto aliases resolve Main with explicit > project pin > global > auto
| review calls instead use the session's Reviewer snapshot; fresh reviews
| first inject a project reviewer pin when one exists
|
v match on the resolved selector — only managed routing tokens get
| Bonsai/nudge/subs treatment; every other key is a DIRECT model
| selection and is authoritative (bypasses Bonsai entirely)
|
+-- resolved Auto role --> classifier.rs (skipped if Bonsai disabled)
| +-- Cloud --> manifest (if enabled + healthy) --> llama-swap fallback
| +-- Local --> prompt_rewriter.rs (when rewrite on) --> llama-swap (Bonsai-chosen model)
|
+-- model="local" --> prompt_rewriter.rs only when rewriting is enabled
| --> llama-swap (fallback_model)
|
+-- model="cloud" --> manifest (if enabled + healthy) --> llama-swap fallback
+-- model="cloud/<id>" --> exact Manifest model --> same cloud fallback policy
|
+-- model="subs" / "brainrouter/subs" --> project subagent pin if present, else
| current local pool, bypassing Bonsai; unconfigured -> classifier/auto,
| NOT the Main profile override
|
+-- model="brainrouter/<key>" or any bare key --> llama-swap (that key), no rewrite,
| no classifier hop; works even when Bonsai is off
|
+-- nudge (if enabled): inject reasoning_budget_tokens on local routes
| unless the client already supplied one
|
v provider/openai.rs: HTTP request to chosen backend
|
v stream.rs: timeout/keepalive protection; event-correlated measurement wrapper
|
v health.rs: record success/failure for circuit breaker
|
v If Anthropic client: anthropic.rs SSE state machine translates back
|
v Response streamed to caller
route_local takes an allow_fallback gate:
- Managed routing (
auto/localtokens) passesallow_fallback: true— if the chosen local model fails, the router retries once withfallback_model. - Direct model picks (any named/bare key,
subs, specific-key routes) passallow_fallback: false— a failed explicit selection surfaces the error to the client instead of silently switching models. This is the "memory compaction" fix: a failingds4-*pick used to jump to the subs pool behind the agent's back; direct picks are authoritative.
Every failed hop is recorded in RouteInfo.failed_attempts (stage, provider, model_key, error) and the router emits one RouteEvent per failed hop plus one for the winner. The dashboard therefore renders multi-hop routes as a hop chain (e.g. manifest ✗ → local ✓) inside the card body; the Sankey ROUTING column embeds the chain in the routing node key itself.
Both Bonsai and Manifest disabled (the default): auto → local, cloud → local — every request is a single hop to llama-swap.
Agent calls mcp_brainrouter_request_review (or: brainrouter cli review request <taskId> <summary>)
|
v mcp_server.rs / cli.rs: POST /review/api/request-async over UDS
|
v escalation/mod.rs: parse request (invalid cwd → HTTP 400, no silent fallback),
call ReviewService::start_review_async
|
v review/context.rs: gather context
| - Auto-detect PRD in project
| - git diff HEAD
| - Read AGENTS.md if present
| - Safe UTF-8 truncation
|
v review/review_loop.rs:
for i in 1..max_iterations (default 5):
review/prompt.rs + review/tokens.rs: build a token-budgeted prompt
if protected content overflows or required evidence is truncated, escalate safely
router.route_with_choice(snapshot) --> selected local/cloud/default reviewer
parse JSON response (STATUS: approved | needs_revision)
update session + llm_turns in the in-memory session store
if approved --> return success
if max_iterations reached --> escalate to human UI
|
v response: {sessionId} (async) — CLI/MCP poll GET /review/api/sessions/{id} every 5s
|
v Dashboard: GET /review/ redirects to /dashboard; JSON/session detail APIs remain available
|
v Human resolve: POST /review/api/resolve (or cli review resolve / lgtm)
|
v Continue: POST /review/api/continue — registers a notifier and waits for
human resolution if the continuation escalates; seeds from llm_turns
Runtime switches are GET/POST /api/review/enabled, /api/review/pr-guidelines, and /api/review/hankndory; GET /api/review/status exposes the HankNDory/review ledger. /review/api/* POST routes pass the same is_destructive gate as the other
management endpoints (loopback-only + Origin check), closing the browser-CSRF
hole. The legacy blocking POST /review/api/request endpoint is still routed
but no client uses it; CLI/MCP use request-async + polling.
brainrouter is a native Rust webui for kyuz0's ai-toolbox-cockpit —
a Python Textual TUI with no CLI or API mode that otherwise is the only
way to manage podman toolbox containers for these backends. brainrouter reads
cockpit's own catalog/config formats and reimplements its Toolboxes/Server
Mode/Models tabs natively, so managing these backends never requires cockpit
to be installed or running. Design rationale, alternatives, and the full
phased rollout are recorded in
docs/design/ai-toolbox-cockpit-integration.md.
Backends covered: llama_cpp (Vulkan/ROCm), ds4 (DeepSeek4 GGUF), and
gufo (Strix Halo ROCm/OpenAI-compatible GGUF serving). v2 L1 purge:
halogen, vllm, and r9v were v1-scope serving backends; their code
(builders, routes, downloads, panels) is removed and their catalog feed ids
now parse as browse-only data (no actionable payload) so the vendored
cockpit feed keeps loading unchanged. comfyui is explicitly excluded from every user-facing surface
(image-gen, not coding-relevant) — the catalog parser still tolerates it
structurally so a shared upstream catalog file never fails to load.
toolboxes.json/models.json are vendored, not fetched at runtime: a
pinned copy lives in the repo (assets/cockpit-catalog/SOURCE records the
upstream commit), refreshed by a weekly sync-toolbox-catalog CI workflow
(.github/workflows/sync-toolbox-catalog.yml + scripts/sync-toolbox-catalog.sh)
that opens a PR rather than mutating main directly, so drift from upstream
is visible and reviewed, not silent. src/toolbox_catalog/schema_validate.rs
structurally validates candidate catalog files before vendoring; unlike
upstream's own catalog/schema.py, it is permissive on unrecognized
backend ids (warnings, not hard failures), so brainrouter keeps loading the
catalog if cockpit adds a new backend before brainrouter has code for it.
Two backend-identity types exist deliberately (src/toolbox_catalog/types.rs):
CatalogBackendId is open and lossless (parses every id the catalog can ever
contain, including comfyui and the brainrouter-owned gufo overlay); SupportedServingBackend is the closed enum
(ds4/gufo/llama_cpp after the v2 L1 purge) that every user-facing surface
actually switches on. CatalogBackendId deliberately keeps parsing the
removed backends' feed strings (halogen/vllm/r9v) — feed vocabulary,
not actionable backends. ModelPayload is a per-backend enum, not one generic
struct, because the feed's backends' model entries are structurally
incompatible with each other.
gufo is not in the vendored upstream cockpit catalog. src/toolbox_catalog/gufo_overlay.rs embeds assets/gufo-catalog/{toolboxes,models}.json, merges it into clones of the vendored JSON, validates the merged whole with the normal structural validator plus gufo-specific checks, and fails closed if the overlay is malformed or conflicts with upstream IDs. GUFO_BACKEND_ID is "gufo"; the runtime image is ghcr.io/gufo-org/toolboxes/gufo-runtime:latest, and the injected runtime is a Strix Halo ROCm engine.
GET /api/toolbox-catalog and /api/toolbox-models expose the effective typed catalog;
GET /api/toolbox-containers lists podman containers across all six
backends (generalizing the old llama-only /api/toolboxes, which remains for
compatibility). POST /api/toolbox-containers creates a new toolbox
container from a catalog entry; /{name}/update, /{name}/delete, and
/{name}/adopt cover the rest of the lifecycle, each serialized by a
per-container operation lock so concurrent create/update/delete against the
same container can't race.
Fedora Toolbx has no toolbox create --label, so brainrouter now tracks
ownership in managed_toolboxes.json: a local sidecar mapping container name
to the exact live podman container ID, image, catalog ID/revision and timestamp.
A container is managed only if it has the legacy label or its sidecar ID
matches the live inspect ID. Create pulls the image first and writes the
sidecar after inspect; adopt records the current ID without recreating;
delete/update/legacy toolbox upgrade are ownership-gated and remove by ID, not
by name. The sidecar is atomic-written with mode 0600; corrupt/unreadable
state fails safe toward "unmanaged."
src/model_downloads.rs implements native download orchestration for
ds4/gufo/llama_cpp (mirroring each backend's own upstream
mechanism; the v2 L1 purge removed the halogen/r9v payload assembly and
the R9V Prepare-PLE job kind) via a job-registry pattern (queued/downloading/verifying/complete/
failed/cancelled), with OS-level cancel and a global download concurrency
limit of 1. vllm is deliberately excluded — its server pulls from Hugging
Face on demand at container start, so there is no separate download step to
orchestrate, matching upstream's own design. Progress is raw-log-tail only
(no byte-level progress bar) because the underlying hf CLI has no
machine-readable progress output to consume.
GET/POST /api/model-downloads* exposes this. (v1 also carried
POST /api/model-downloads/r9v/prepare-ple + ModelPresence.ple_ready as
R9V's second readiness gate; removed with the backend by the v2 L1 purge.)
GET /api/model-downloads/status response schema. Returns { "models": [ModelPresence…], "hf": { "found_on_path": bool, "binary": string, "message": string|null, "install_command": string|null } }. The hf object
is an advisory preflight: the ds4/halogen/r9v/gufo/llama_cpp download path
runs the hf (Hugging Face) CLI on the host, so found_on_path reports
whether an executable hf is visible on the brainrouter service's PATH (a
best-effort scan; the authoritative check is still the per-job download
spawn). When found_on_path is false, message/install_command are
populated (python3 -m pip install --user -U "huggingface_hub[cli]") and the
dashboard shows a warning banner on the Models/Downloads and Server Mode
panels; when true they are null. Install hf on the service PATH as a
prerequisite for model downloads (vllm pulls at server start and needs no
host hf).
src/server_mode.rs starts/stops/checks-status of headless, detached
podman run servers for ds4 and gufo — distinct
from the Toolboxes tab's interactive dev-shell containers. GET/POST /api/server-mode/{ds4,gufo}/{status,start,stop} cover the
lifecycle. v1 also shipped halogen, vllm (with a cache-paths action),
and r9v (with a paths action); the v2 L1 purge removed those builders,
their routes, and their panels — requests to the removed paths answer 404. R9V server mode is
native Rust but shipped with an explicit "unverified" badge: no 2×AMD
R9700 hardware was reachable from this environment to functionally validate
ROCm/HSA device passthrough end-to-end (see Open Questions).
Gufo is a Strix Halo OpenAI-compatible ROCm engine. Its server command is
podman run ... ghcr.io/gufo-org/toolboxes/gufo-runtime:latest gufo serve --host 0.0.0.0 --port <port> --sessions <n> llm --model /models/<main>.gguf --served-model-name <model_id> --context <ctx>, with optional DFlash2
speculative decoding appended as --speculative dflash2 --dflash-model /models/<draft>.gguf. src/serving_identity.rs reports
openai_compatible=true for gufo and derives compute_api="rocm", but this
remains bookkeeping only — the request router still does not dispatch to gufo.
src/cockpit_config.rs reads cockpit's own JSON config file directly, so
both tools share one source of truth for active_platform and per-backend
default toolboxes — instead of a second, divergent brainrouter-only config.
This is Phase 1 only: brainrouter never performs continuous, automatic
read-modify-write against this file; the only write path is an explicit,
user-triggered "apply" action scoped to the specific keys cockpit itself
owns (GET /api/cockpit-config, POST /api/cockpit-config/default-toolbox,
POST /api/cockpit-config/active-platform). A continuously-synced
bidirectional merge protocol is explicitly out of scope for this rollout
(user-confirmed — see the design doc's Decision log), since no
brainrouter/cockpit locking protocol exists upstream to make concurrent
writes safe. If the config directory doesn't exist on a host, brainrouter
surfaces that rather than silently auto-creating it.
src/serving_identity.rs records a {toolbox_backend, compute_api, runtime_profile_id, endpoint} fact per currently-running, brainrouter-managed
Server Mode container (GET /api/serving-identities). This is read-only
bookkeeping, not a routing mechanism — it does not make any backend
reachable from brainrouter's actual /v1/chat/completions request router.
Only llama_cpp (via Manifest/llama-swap) is genuinely routable traffic
today; every other backend renders as an explicit status_only entry in the
Sankey/benchmark dashboards below, never faked as routed traffic. Making
status_only backends actually dispatchable is unshipped future work — see
Planned and Unshipped Work.
The dashboard's routing Sankey (GET /dashboard, described under
Dashboard above) gains two additional columns: toolbox
backend (which of the six families served the request) and compute
API (vulkan/rocm/etc.). Both are nullable/additive on RouteEvent; the
existing 4-column behavior is a strict subset of the extended layout, and
click-to-filter works identically on the new columns. Traffic from backends
that aren't yet wired into the router (everything except llama_cpp today)
renders in an explicit status_only bucket rather than being omitted or
faked as routed.
- Not a CLI-parity surface.
brainrouter cli toolboxes/upgrade toolboxremain llama.cpp-only; the new backends are dashboard/API-only today (see Planned and Unshipped Work). - Not a router-integration. No Server Mode container is dispatchable as real LLM traffic yet — see Serving-identity registry above.
- Not a TUI-fidelity clone. Feature parity is scoped to capabilities (create/update/delete/list/launch/configure), not cockpit's exact widgets, keybindings, or ASCII banner.
v2 R2 feature gate: the explorer (
/benchmarks,/api/benchmarks/*, the native lab) compiles only with--features benchmarks; default builds answer the documented 503 degradation envelope ("Benchmark explorer unavailable: … Core routing is still available.") from the same routes, and the dashboard hides the explorer nav link via a server-injected flag. The registry core (store schema, ingestion types, degradation contract) is always compiled.
This is an import/planning/inspection subsystem inside the existing Rust/Hyper daemon, with embedded HTML/JavaScript and SVG charts. It is not a Python/FastAPI sidecar, a model runner, or a benchmark scheduler. Opening pages, discovering metadata, planning matrices, and previewing imports must not start inference.
| Entity / table | Shipped responsibility |
|---|---|
models |
Family, architecture, checkpoint/revision, tokenizer, parameter counts, native context, metadata |
artifacts |
Model reference, format/quantization, disk size/BPW, declared SHA-256/source and conversion provenance |
runtimes |
Repository/fork/commit, compiler/backend, build flags, executable/container identities, feature capabilities |
hardware_profiles |
CPU/cores/RAM, per-GPU definitions, unified-memory flag, OS/kernel/drivers, capture time |
workloads |
Versioned manifest identity, workload kind, input/output/corpus counts and license |
experiment_specs, experiments |
Canonical configuration and its SHA-256 identity; registry references, context/token/batch/thread/optimization/sampling settings |
runs |
Zero-based repetition, status, exact command, timestamps, seed/warmup, environment, paths, raw result and failure reason |
performance_metrics, speculative_metrics |
Optional one-to-one measurements per run |
quality_results, telemetry_samples |
Per-task quality and ordered host/per-GPU observations |
entity_fingerprints |
Reject payload changes under reused registry/experiment identities |
schema_migrations, indexes, run_summary |
Schema/version bookkeeping and joined query support |
exclusions |
Reserved schema table; the current planner returns exclusions in its response rather than persisting them |
The initial migration creates normalized SQLite STRICT tables and initializes WAL. Each connection enables foreign keys and a five-second busy timeout. Startup checks the migration identity/version and required tables. A future, corrupt, or unusable database disables only the explorer; it is not reset or downgraded. This is a new dedicated database, not a conversion of ephemeral review sessions or the live routing buffer.
Rust boundary models reject unknown structural fields and invalid enums,
negative/out-of-range metrics, malformed lowercase SHA-256 values, invalid
timestamp order, invalid parameter/test/proposal relationships, and integers
too large for normalized SQLite integer columns. Backend-specific metadata remains JSON. Feature
states distinguish unsupported, disabled, enabled, and
requested_unavailable; speculation kinds are recorded, not executed.
An experiment hash covers the canonical serialized registry IDs and complete experiment settings but excludes its application ID. Registry fingerprints protect those referenced identities from later mutation. Artifact, executable, manifest, and build metadata are declarations supplied by the importer; the daemon does not hash/download files or attest what a runtime actually loaded. Declared draft-artifact references must exist.
Runs are uniquely identified by ID and (experiment_id, repetition). Those
dimensions cannot move. A succeeded run is immutable; a correction requires a
new repetition or experiment. Other statuses (planned, running, failed,
oom, timeout, cancelled, skipped) may receive a replacement full bundle
under the same attempt identity. This is not an append-only live telemetry feed.
- A complete bundle includes
model,artifact,runtime,hardware,workload,experiment, andrun, with optional performance, speculation, quality, and telemetry sections. Cross-references must agree. - Validation/prepare endpoints exercise transactional ingestion checks and roll back. Only explicit ingestion commits; previews neither reserve IDs nor guarantee a later concurrent import will succeed.
- Reusable templates contain registry definitions plus an experiment, not a
run payload.
/preparecombines a template, explicit repetition/status/command, optional timestamps/llama-bench output, and optional selected plan candidate. Existing matching custom IDs are reused. - The llama-bench adapter accepts one object, an array, or a
resultsarray. It rejects ambiguous/repeated/contradictory phases, invalid metric types, conflicting bundle metrics, and incompatible reported token counts. Genericavg_ts/tokens_per_second/tpsmust resolve to one phase; combined tests require separate prompt and generation rates. Raw uploaded output is retained underrun.raw_result.llama_bench. - JSON/YAML matrices deterministically expand registry IDs, contexts, token counts, and optimization combinations. Duplicate dimensions/configurations are invalid; prompts exceeding context are returned as exclusions. Planning does not validate artifact existence on disk, runtime feasibility, or actual capability support. Repetition/randomization/telemetry fields are execution hints for external tooling, not actions performed by this daemon.
- The UI supports file upload, validation preview, explicit confirmation, success links, opt-in browser-local template reuse, and plan-candidate selection. Supplied examples are clearly labelled synthetic, with matching IDs and 8K/32K/128K contexts; examples never auto-populate the database.
GET /api/benchmarks/runs returns
{items,page,per_page,total,total_pages}. Defaults are page 1, 25 rows, and
sort=started_at&order=desc. page must be positive; per_page is 1..100.
Exact filters are status, family, backend, workload, quant_name, and
speculator_type. q is literal substring search across model/runtime/workload/run
labels with SQLite's ASCII case-insensitive LIKE behavior; %/_ are escaped.
Blank filters are omitted. Repeated parameters take their last value; unknown
parameters, sort fields, and orders are rejected. Unknown exact filter values
simply match no rows.
Sort fields are started_at, family, workload, context_tokens,
prompt_tps, generation_tps, and ttft_ms, with asc/desc. Timestamp sorting
uses creation time when start time is absent. Missing numeric metrics sort last;
ties use ascending run ID. A page beyond the result count is empty.
The inspector presents all configuration/provenance, performance/latency,
speculation and task-quality data, with ordered host/per-GPU sparklines that keep
null gaps and never sum duplicated host readings. Commands, URLs, and paths are
inert text. Raw JSON view/download preserves original response text; the browser
warns on unsafe JavaScript numeric precision and rejects unsafe import rewrites.
quality_score in summaries is the mean of pass@1 results only, not an average
of heterogeneous metrics.
The page's scatter/Pareto, context, runtime, quantization, and pass@1-per-GiB views use the current page only (at most 25 runs). The total matching count is separate. Changing page, order, or status selection changes the displayed sample; these views are not controlled cross-workload experiments.
CSV/JSONL export ignores UI pagination and uses one consistent SQLite snapshot. It returns all matching summaries within the documented budgets or an explicit limit error, not partial data. CSV escapes delimiters/newlines and neutralizes spreadsheet formula prefixes. It is not a full database backup, bundle archive, telemetry export, or Parquet implementation.
| Resource | Contract |
|---|---|
| Shared admission | Two slots across store clones and run_blocking; saturation returns 503 with Retry-After: 1, no waiting queue |
| Benchmark POST body | 16 MiB; 15-second collection deadline (413 / 408) |
| Benchmark URI | 8,192 bytes |
| HTTP SQLite row / JSON expression | 4 MiB |
| Serialized response / export / expanded plan | 8 MiB; explicit 413 on excess |
| Run detail | 10,000 telemetry rows and 1,000 quality rows, additionally byte-limited |
| Filter choices | 1,000 distinct values per field |
| Matrix | 10,000 candidate combinations, additionally response-budgeted |
| Filtered export | 10,000 summaries, additionally byte-limited |
| SQLite lock waiting | Five seconds; contention returns 503 |
HTTP parsing/validation/planning/SQLite/serialization executes on bounded
blocking workers. Admission survives cancellation until actual blocking work
exits, and survives Hyper body drop while response Bytes remain queued; output
frames are capped at 16 KiB. A disconnected ingestion may already have committed:
inspect its run ID before retrying. Larger records remain in SQLite for offline
or synchronous consumers; HTTP does not silently sample/truncate them. Async
internal consumers must reuse BenchmarkStore::run_blocking.
runtimes.backend (CHECK(backend IN ('cpu','cuda','rocm','vulkan','metal','sycl','rpc','other')))
already existed and is the "engine" axis surfaced as Engine everywhere in
the UI. Migration 0003_toolbox_serving_dimension.sql adds the genuinely new
dimension: which toolbox family (ds4/halogen/vllm/r9v/llama_cpp
in the frozen migration vocabulary) served a run, via a new serving_runtimes
table plus a
toolbox_catalog_snapshot table (both nullable/additive; existing
llama.cpp-only rows are unaffected). GET /api/benchmarks/filters and
GET /api/benchmarks/runs gained toolbox_backend/compute_api as
queryable/filterable fields alongside the existing family/workload/etc.
Spider/radar chart (GET /api/benchmarks/spider, rendered on
/benchmarks): compares runs across a user-picked pair of axes drawn from
models, harnesses, engine, backend, toolbox, speed, and TEPR ("token
efficiency to positive result" — see below), with a "compare by"
family/workload/family×workload picker, an entity picker capped at 5 series,
and an SVG radar renderer with min-max normalization and greyed-out vertices
for insufficient data.
TEPR is the confirmed metric definition (signed off by the user). As implemented:
SUM(quality_results.generated_tokens)across every attempt in the cohort, divided by the count of positive runs (status='succeeded'and no associatedquality_resultsrow withpassed=0); runs with zeroquality_resultsrows are excluded entirely. Comparing by model requires a single pinned workload (token cost is workload-dependent); comparing by harness, or by model×harness pairs, needs no pin.
/models (including ?model=<encoded-key>) combines exact local model keys
from backend status and observed routes with active requests, reported metadata,
completed measurements, and explicit historical references. Read-only polls use
/running, /v1/models, and /props for at most four already-ready proxies.
No poll loads a model. Backend requests have two-second deadlines, 1 MiB bodies,
and a 256-model catalog limit; there is a five-second pause between poll cycles.
Unavailable metadata is represented explicitly. Runtime/build/quantization
values are shown only when reported; disk size is never used as resident memory.
Unattributed/cloud activity is not silently attached to a local model.
| Measurement | Meaning / exclusion |
|---|---|
| Routing latency | Time until the routing body returns a provider result; not full response latency or TTFT |
| Measured TTFT | Routing-body entry to first complete nonempty content/reasoning/tool-argument SSE frame; includes classification/retries, excludes outer role selection and client-side timing |
| Generation TPS | (reported completion_tokens - 1) / observed first-to-last output interval; requires >1 token and distinct output times; buffering/hidden tokens/backpressure limit precision |
| Usage | Provider-reported prompt/completion counts; missing/conflicting counts stay unknown |
| Completion validity | [DONE] plus clean EOF; errors, cancellation, malformed/oversized frames, dropped/incomplete streams do not become successful measurements |
| Live TPS / RAM / VRAM | Unavailable unless a reliable live source reports them; no inference from elapsed time or benchmark peaks |
The global in-memory limits are 500 route events and 500 completed samples; per-model detail exposes up to 25 recent samples. Restart clears history and alert latches. High traffic can evict a comparison window early.
Alerts compare two disjoint windows from the same local provider/model and measurement source. They are indicative live trends, not statistically controlled benchmark regressions or causal thermal-throttling diagnoses.
| Policy field | Default | Allowed range |
|---|---|---|
window_seconds |
300 | 30..3600 |
min_samples per window/metric |
5 | 3..100 |
generation_drop_percent |
20 | 5..95 |
ttft_rise_percent |
30 | 5..500 |
rate_increase_points |
20 | 1..100 percentage points |
repeated_count |
3 | 2..100 |
debounce_samples, recovery_samples |
2 each | 1..10 |
baseline_max_age_days |
30 | 1..3650 |
Performance comparisons use medians; route-error/fallback warnings use rates and minimum occurrence counts. OOM/timeout labels require corresponding recorded error text. New evidence is required to advance warning/recovery debounce; repeated polls and insufficient data are not recovery. Late/out-of-order completed event IDs remain deduplicated. Router errors are not an exhaustive count of stream failures. Policy/mapping changes reset latches.
Operators preview a succeeded run with a completion timestamp and positive generation/TTFT measurement, then select it for a known exact local model key with the expected experiment hash and a nonempty rationale. Family names are not matched automatically. Configuration/artifact/runtime/hardware/workload provenance is retained independently of live observations.
Reference status is not_selected, unavailable, stale, mismatched, or
not_comparable. Even a valid reference remains not comparable until per-sample
artifact/build/hardware/context/workload/concurrency identity and metric
equivalence can be established. Benchmark failure affects reference lookup, not
the live page or core routing; mappings can be cleared while that database is down.
At most 32 baseline mappings are stored, alongside policy, in the config-adjacent observability JSON file. Writes require the latest revision, 128 KiB maximum body/settings size, and a five-second body deadline. Stale revision/hash confirmation returns 409; validation, read, and write errors are visible. Unreadable settings disable saving until repair/restart, not daemon startup. No request history, automated remediation, scheduling, or outbound notifications are persisted or performed by this subsystem.
The Bridge Architecture below ships as the bridge sidecar plugin (v2 L2): the daemon supervises the brainrouter-bridge binary and serves it a grant-gated JSON-RPC surface over a per-plugin Unix socket. Discord and Signal transports bring LLM access to messaging apps. They do not call LLMs directly -- they shell out to the omp CLI as a subprocess, which connects back to brainrouter through the normal proxy path. This means bridge conversations get the same routing, fallback, and review capabilities as direct harness usage.
Each user message triggers an omp subprocess invocation:
- Transport receives a message (Discord event / Signal CLI poll)
- Bridge core resolves the session context: working directory, active model, conversation history
ompCLI is spawned with the resolved context, pointed at brainrouter as its backend- Response is chunked to fit platform limits (Discord: 1500 chars, Signal: 4000 chars)
- Session state is updated and persisted
Runtime persistence moved to the plugin state dir
$XDG_STATE_HOME/brainrouter/plugins/bridge/ (0700). On the plugin's first
start, the six v1 files are copied from ~/.local/share/omp-bridge/ if
absent; v1 originals are left in place for revert:
| File | Contents |
|---|---|
discord-sessions.json |
Per-channel Discord session state |
discord-channel-models.json |
Per-channel model alias mappings |
discord-work-dirs.json |
Per-channel working directory |
signal-sessions.json |
Per-user/group Signal session state |
signal-channel-models.json |
Per-user/group model alias mappings |
signal-work-dirs.json |
Per-user/group working directory |
| Command | Description |
|---|---|
!br ping |
Health check |
!br reset |
Clear the current channel's session |
!br status |
Show current model |
!br auto / local / cloud |
Set routing mode |
!br <model-name> |
Set a specific llama-swap model (names containing - or .) |
!br list |
List all models (routing modes + llama-swap) |
!br review |
Show current review mode |
!br review auto|local|cloud |
Set review mode |
!br model <name> <query> |
One-off query with a specific model |
!br ls |
List files in current working directory |
!br cd <dir> |
Change working directory |
!br .. |
Move up one directory |
!br mkdir <name> |
Create a directory |
!br help / !br ? |
Show command help |
@bot <query> |
Send a query via mention |
| bare text | Send a query to the LLM (no prefix needed) |
| Command | Description |
|---|---|
!br ping |
Health check |
!br reset |
Clear the current session |
!br status |
Show current model |
!br auto / local / cloud |
Set routing mode |
!br <model-name> |
Set specific llama-swap model (names containing - or .) |
!br model <name> |
Set model (legacy form) |
!br list |
List available models |
!br review |
Show current review mode |
!br review auto|local|cloud |
Set review mode |
!br help / !br ? |
Show command help |
!br <query> |
Send a query to the LLM |
| bare text | Send a query (no prefix needed) |
Each transport maintains per-channel (Discord) or per-user/group (Signal) state:
- Model selection: Users can set a specific llama-swap model with
!br <model-name>(Discord) or!br model <name>(Signal), or switch routing modes with!br auto,!br local,!br cloud. The selection persists across messages until explicitly changed. - Working directories: Tracked per-channel/user. Commands like
!br cd,!br .., and!br mkdirmanipulate the working directory, which is passed toompas the CWD for file-aware operations. - Sessions: Conversation context is maintained per-channel/user and can be reset with
!br reset. Bare text (without any prefix) is treated as a query.
| Method | Path | Description |
|---|---|---|
GET |
/health |
Health check. Returns daemon status |
GET |
/v1/models |
List available models |
POST |
/v1/chat/completions |
OpenAI-compatible chat completion proxy |
POST |
/v1/messages |
Anthropic Messages API compatible proxy |
| Method | Path | Description |
|---|---|---|
GET |
/ |
Redirect to /dashboard |
GET |
/dashboard |
Embedded HTML dashboard: live routing feed grouped per conversation (client session header, or a stable hash of the conversation prefix when no header is present — each conversation renders as one card), hop-chain rendering for multi-hop fallbacks, in-flight request tracker (with cancel), a four-column Sankey flow diagram — HARNESS (Pi / OMP / Opencode / Claude / Droid, from User-Agent) → SESSION (OMP session title) → ROUTING (auto / cloud / local / fallback chain) → MODEL (resolved llama-swap key) — where clicking any node highlights its start-to-end path and clicking a card highlights the matching Sankey path, version display, one-click upgrades, review sessions |
GET |
/benchmarks |
Imported benchmark explorer, inspector and import/planning workflows; 503 when storage initialization failed, and in default builds — the explorer ships behind the non-default benchmarks cargo feature (v2 R2) and answers the same 503 degradation envelope when the feature is off |
GET |
/models |
Read-only local model activity, measurements, rolling alerts and reference controls |
GET |
(favicon/logo assets) | Static assets for dashboard UI |
| Method | Path | Description |
|---|---|---|
GET |
/api/routing-events |
JSON polling feed, not an SSE event subscription |
GET |
/api/routing-stats |
Aggregate routing statistics |
GET |
/api/inference-status |
Current inference state, model, elapsed time, token progress. n_tokens = total tokens decoded on the llama-server slot (monotonic within a request) — the progress bar is n_tokens / max_tokens. Slot data comes from the llama-server /slots endpoint (served as a bare JSON array on this build; both shapes are tolerated), with the live per-request count read from next_token.n_decoded (object or single-element array) |
GET |
/api/service-health |
Service reachability probes and fallback overview, not an exhaustive stream/circuit history |
GET |
/api/versions |
Version information for brainrouter and dependencies |
GET |
/api/review-config |
Current review configuration (mode, model) |
POST |
/api/review-config |
Update review configuration |
GET/POST |
/api/routing-profile |
Read/replace {preset,main,reviewer,subagent_model}; strict role/preset validation, 16 KiB JSON body limit |
GET/POST/DELETE |
/api/project-pin |
Per-project role pins; ?path= resolves an absolute path to a canonical Git-root key |
GET |
/api/routing-models |
Authenticated local/cloud metadata catalogs, visible per-provider errors; disabled cloud skips discovery |
GET |
/api/models/llama-swap |
List models available in llama-swap |
GET |
/api/plugins |
Plugin registry + live status + notification toasts (v2 L2) |
POST |
/api/plugins/install|enable|disable|remove|config |
Plugin management (v2 L2) |
GET |
/api/plugins/logs?id=<id>&tail=<n> |
Captured plugin log (rendered HTML-escaped by the dashboard) |
GET |
/api/bonsai |
Bonsai classifier server state (enabled, healthy, url) |
GET/POST |
/api/nudge |
Thinking-budget nudge state/tier |
GET/POST |
/api/prompt-rewrite |
Local prompt rewrite state |
GET/POST |
/api/review/enabled |
Runtime code-review master switch |
GET/POST |
/api/review/pr-guidelines |
Runtime PR-structuring guideline injection toggle |
GET/POST |
/api/review/hankndory |
Runtime design-aware review toggle |
GET |
/api/review/status |
Design-aware review state and recent verdict ledger |
| Method | Path | Description |
|---|---|---|
POST |
/api/restart/llama-swap |
Restart llama-swap service |
POST |
/api/restart/llama-cpp |
Restart llama.cpp toolbox |
POST |
/api/restart/manifest |
Restart Manifest service |
POST |
/api/restart/brainrouter |
Restart brainrouter daemon |
POST |
/api/upgrade/llama-swap |
One-click upgrade llama-swap |
POST |
/api/upgrade/manifest |
One-click upgrade Manifest |
POST |
/api/upgrade/toolbox |
One-click upgrade llama.cpp toolbox |
POST |
/api/bonsai/toggle |
Stop/start the Bonsai classifier llama-server to free or reclaim VRAM; while stopped, auto routing defaults to Local. Toggling Bonsai off also turns prompt rewrite off (server-side coupling — the invariant "rewrite on ⇒ Bonsai on" is enforced in the daemon, not just the UI) |
POST |
/api/models/flush |
Unload all models from llama-swap memory (frees VRAM) without restarting the service |
Most POST routes require the loopback/CSRF gate (see Security Model). Covers
ds4/gufo/llama_cpp after the v2 L1 purge — never comfyui, and never
the browse-only feed ids (halogen/vllm/r9v).
| Method | Path | Description |
|---|---|---|
GET |
/api/toolboxes |
Legacy llama-only container listing (kept for compatibility) |
GET |
/api/toolbox-catalog |
Effective typed catalog for all 6 backends (vendored cockpit catalog plus gufo overlay) |
GET |
/api/toolbox-models |
Typed models.json catalog (per-backend ModelPayload) |
GET |
/api/toolbox-containers |
List podman containers across all 6 backends, with ownership/adoption status |
POST |
/api/toolbox-containers |
Create a toolbox container from a catalog entry |
POST |
/api/toolbox-containers/:name/update |
Pull + recreate a toolbox container |
POST |
/api/toolbox-containers/:name/delete |
Remove a toolbox container |
POST |
/api/toolbox-containers/:name/adopt |
Record a pre-existing container's live podman ID as brainrouter-managed |
GET |
/api/model-downloads / /status |
List/summarize download jobs (ds4/gufo/llama_cpp) |
POST |
/api/model-downloads |
Start a model download job |
GET |
/api/model-downloads/:id |
Job detail (raw-log-tail progress, no byte-level progress) |
POST |
/api/model-downloads/:id/cancel |
Cancel a download job |
POST |
/api/model-downloads/verify |
Verify a downloaded model's integrity |
GET/POST |
/api/server-mode/{ds4,gufo}/status|start|stop |
Detached Server Mode lifecycle (v2 L1: the removed halogen/vllm/r9v paths, incl. vllm/cache-paths and r9v/paths, answer 404) |
GET |
/api/serving-identities |
Read-only registry of currently-running Server Mode containers' {toolbox_backend, compute_api, runtime_profile_id, endpoint} |
GET |
/api/cockpit-config |
Read cockpit's ~/.config/ai-toolbox-cockpit/config.json |
POST |
/api/cockpit-config/default-toolbox |
Write the default-toolbox-per-platform key back to cockpit's config |
POST |
/api/cockpit-config/active-platform |
Write the active-platform key back to cockpit's config |
All POST routes require the loopback/CSRF gate. Bodies and result limits apply to previews as well as writes. Responses are JSON unless downloading examples, CSV/JSONL, or HTML.
| Method | Path | Contract |
|---|---|---|
GET |
/api/benchmarks/runs |
Paginated summaries and counts; filter/order rules above |
GET |
/api/benchmarks/runs/:id |
{run,configuration,run_record,performance_metrics,speculative_metrics,quality_results,telemetry_samples} |
GET |
/api/benchmarks/filters |
Sorted observed status/family/backend/workload/quant/speculator/toolbox_backend/compute_api values |
GET |
/api/benchmarks/spider |
Radar-chart series for a picked axis pair (models/harnesses/engine/backend/toolbox/speed/TEPR) — see Backend/engine dimensions and the spider/radar chart |
GET |
/api/benchmarks/export?format=csv |
Full filtered summaries within budgets; jsonl is also supported and is the default |
GET |
/api/benchmarks/examples/:name |
bundle.json, template.json, matrix.yaml, matrix.json, llama-bench.json; labelled synthetic downloads |
POST |
/api/benchmarks/prepare |
Template + run fields + optional selected experiment/llama output -> {valid:true,persisted:false,bundle,warnings} |
POST |
/api/benchmarks/validate |
Complete bundle -> transactional rollback preview |
POST |
/api/benchmarks/validate/llama-bench |
{bundle,llama_bench} -> adapter + rollback preview |
POST |
/api/benchmarks/ingest |
Complete bundle -> 201 {run_id} |
POST |
/api/benchmarks/ingest/llama-bench |
{bundle,llama_bench} -> adapter + committed ingestion |
POST |
/api/benchmarks/plan |
Matrix JSON, or YAML with YAML content type -> {name,experiments,exclusions,repetitions,run_count,randomize_order,capture_telemetry} |
| Method | Path | Contract |
|---|---|---|
GET |
/api/observability/models |
Cached local models, active/recent data, policy/retention/measurement definitions and visible backend/attribution errors |
GET |
/api/observability/settings |
{settings:{policy,baselines},revision,read_error,write_error,path} |
POST |
/api/observability/settings |
{revision,policy} using the complete validated policy |
GET |
/api/observability/reference?run_id=... |
Validated successful reference projection; errors are not empty references |
GET |
/api/observability/baseline?model_key=... |
Selected-reference/comparability status, including lookup failure reason |
POST |
/api/observability/baseline |
{revision,model_key,run_id,expected_experiment_hash,note}; null run ID clears without a benchmark read |
| Method | Path | Description |
|---|---|---|
GET |
/api/inflight |
Active in-flight requests: {requests:[{id, elapsed_ms, method_path, model, user_agent, peer_addr, conv_id, session_id, bytes_received, activity, pp_progress}]}. The panel is hidden in the dashboard when this is empty |
POST |
/api/inflight/cancel |
Cancel one in-flight request. Body: {id}. Returns {"cancelled":true} on success, HTTP 404 for an unknown id. Loopback-only + CSRF-protected (in the is_destructive gate list) |
GET |
/api/omp-sessions |
OMP session titles for the Sankey "SESSION" column: {home, sessions:[{slug, title, updated_ms}]} — each session is a directory under ~/.omp/agent/sessions/<slug> whose first JSONL line carries {"type":"title",...} |
| Method | Path | Description |
|---|---|---|
GET |
/review/ |
Redirect to /dashboard |
GET |
/review/session/:id |
View a specific review session (HTML) |
POST |
/review/session/:id/resolve |
Resolve a review session (HTML form) |
GET |
/review/api/sessions |
List all review sessions (JSON) |
GET |
/review/api/sessions/:id |
Get review session details (JSON) |
POST |
/review/api/request |
Start a new review (JSON) |
POST |
/review/api/request-async |
Start and return {sessionId,status} immediately; does execute the configured reviewer asynchronously |
POST |
/review/api/resolve |
Resolve a session programmatically (JSON) |
POST |
/review/api/continue |
Continue a review iteration (JSON) |
POST |
/review/api/lgtm |
Mark a session as approved (JSON) |
The MCP server exposes 4 tools via JSON-RPC stdio, forwarded to the daemon over UDS.
Start a new code review session.
| Parameter | Type | Required | Description |
|---|---|---|---|
taskId |
String |
Yes | Identifier for the task being reviewed |
summary |
String |
Yes | Brief summary of the changes |
cwd |
String |
No | Working directory for context gathering. Falls back to peer CWD resolution |
details |
String |
No | Additional context about the changes |
conversationHistory |
String[] |
No | Conversation history for context |
List all review sessions. No parameters.
Get details for a specific review session.
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId |
String |
Yes | The session ID to retrieve |
Resolve a review session with feedback.
| Parameter | Type | Required | Description |
|---|---|---|---|
sessionId |
String |
Yes | The session ID to resolve |
feedback |
String |
Yes | Resolution feedback |
| Harness | Protocol | MCP | Install Command | Notes |
|---|---|---|---|---|
| omp | OpenAI | stdio | brainrouter install omp |
Updates models.yml and mcp.json |
| vibe | OpenAI | stdio | brainrouter install vibe |
Appends to config.toml |
| opencode | OpenAI | local | brainrouter install opencode |
Merges into config.json |
| codex | OpenAI | stdio | brainrouter install codex |
Writes ~/.codex/config.toml entries |
| droid | Anthropic | stdio | brainrouter install droid |
provider: "anthropic" required; hits /v1/messages |
| claude | Anthropic | stdio | brainrouter install claude |
Sets ANTHROPIC_BASE_URL=http://127.0.0.1:9099 |
| pi | HTTP (extension) | N/A | brainrouter install pi |
Calls /review/api/* directly from a pi extension |
| Failure | Detection | Mitigation |
|---|---|---|
| Manifest returns 429 / 5xx | HTTP status code | Report failure to health tracker; try llama-swap fallback |
| Manifest stream stalls after output begins | TimeoutStream (180s per chunk) |
Error surfaces in SSE; no output replay or successful completed measurement |
| Manifest circuit open | Health tracker (3 failures) | Skip Manifest, route directly to llama-swap |
| llama-swap returns error / 404 | HTTP status code | Error returned to caller; next request may load a different model |
| llama-swap circuit open | Health tracker (3 failures) | Error: no backend available. 60s cooldown before retry |
| Both circuits open | Health tracker | Error returned to caller; both providers on 60s cooldown |
| Enabled Bonsai classification fails | HTTP/parse error or timeout | Auto roles choose the cloud path, still respecting Manifest's enabled flag and local fallback policy |
| Bonsai server OOM / crash | Process exits; next health probe fails | Default to Cloud until the server is started again from the dashboard |
| Bonsai server stopped from dashboard | Shared enabled flag cleared before kill |
Auto roles skip classification and choose local; pinned roles remain pinned |
| Daemon not running when MCP connects | UDS connect error | brainrouter mcp exits with a clear error message |
| Review LLM returns unparseable JSON | JSON parse failure in review_loop | Retry within iteration; count as iteration attempt |
| Review hits max iterations | Iteration counter | Escalate to human via dashboard UI |
| Review disabled | review_runtime_state.json / /api/review/enabled |
New review requests return disabled without contacting a model |
| Review prompt would exceed safe budget or required evidence is truncated | review/tokens.rs budget planner |
Terminal escalation rather than a confident approval |
| Project pin path is non-absolute or not in a Git repo | ProjectPinStore::set / resolver |
Reject pin write or inherit global profile |
| Peer CWD resolution fails | /proc read failure |
Use a valid explicit review CWD; invalid/absent resolved review directories are rejected, not silently substituted |
| Config validation failure at startup | Missing required fields, invalid URLs, missing GGUF | Daemon refuses to start with descriptive error |
| Legacy auto reviewer retains an ignored model | YAML/legacy-state read normalization | Discard only that ignored field, warn with source/action, persist state migration once; keep new API writes strict |
| Invalid routing migration | Validation/read/write failure | Report source/destination and cause; do not silently change local/cloud semantics |
| Corrupt/unwritable/future benchmark database | Store initialization | Log cause; benchmark routes 503; core routing/listeners/Bonsai continue |
| Benchmark workers saturated or SQLite busy | Shared admission/lock timeout | 503 and Retry-After; no unbounded job queue |
| Benchmark result/export exceeds budget | Row/count/serialization checks | Explicit 413; narrow request or read offline, never silent truncation |
Missing hf CLI for toolbox downloads |
/api/model-downloads/status.hf advisory preflight |
Dashboard banner shows python3 -m pip install --user -U "huggingface_hub[cli]"; download spawn remains authoritative |
| Toolbox sidecar missing/corrupt or ID mismatch | managed_toolboxes.json + podman inspect |
Treat as unmanaged and refuse destructive operations |
| Missing measurements or insufficient retained history | Source validation and sample minima | Unknown/insufficient-data state, no fabricated metric or recovery |
| Observability settings/reference unavailable | File read/CAS/reference validation | Visible error or reference status; core routing/live page remain available |
| Bridge OMP subprocess timeout | Configurable timeout (default 600s) | Kill subprocess, return timeout error to chat |
| Signal CLI unavailable | Subprocess spawn failure | Signal transport disabled; logged |
| Discord token invalid | Serenity connection failure | Discord transport disabled; logged |
Main, Reviewer and Subagent are model-selection roles, not authorization roles.
Brainrouter binds to 127.0.0.1:9099 by default. Its Unix socket defaults to
$XDG_RUNTIME_DIR/brainrouter.sock, falling back to /run/brainrouter.sock.
There is no general proxy/read-API authentication layer. Do not expose a
non-loopback listener without a separate trusted access-control layer.
Protected mutations include routing profiles, /review/api/*, benchmark and
observability POSTs, and the listed management operations. They require a
loopback peer or the local Unix socket.
The protected gate checks Origin first, otherwise Referer. HTTP localhost,
IPv4 loopback, and IPv6 loopback hosts are accepted on any port to support local
tunnel/container port remapping. null, non-loopback, malformed, and
credential-bearing origins are rejected. Headerless local CLI clients are
allowed. The peer restriction still applies; a public HTTPS/Tailscale hostname
is not implicitly trusted by this loopback policy. This is not a configurable
general reverse-proxy origin allowlist.
The legacy /review/session/:id/resolve form route is not in the
/review/api/* prefix gate. Keep the listener local and use the protected JSON
review APIs for automation; this gate is not general authentication for every route.
Imported commands/paths/URLs are never executed or fetched. Browser rendering uses inert text, CSV exports neutralize spreadsheet formula prefixes, and benchmark/observability bodies and workloads are explicitly bounded. Raw imported environment/result metadata can still be sensitive; operators must redact it before importing or sharing exports.
Working directory tracking (for bridge sessions and review context gathering) enforces:
- Absolute path requirement
- Null-byte rejection
- Path-traversal component blocking (no
..escapes in session-tracked paths)
Required configuration is validated at daemon startup:
manifest.base_urlmust start withhttp://orhttps://bonsai.model_pathmust exist on disk — only whenbonsai.enabled: true; a missing file with Bonsai off does not crash startup (Bonsai can be enabled later at runtime once the model is in place)llama_swap.fallback_modelis required
The Manifest API key (mnfst_*) is never committed: it lives in /etc/brainrouter/env (shared) or each user's ~/.config/brainrouter/.env, and configs reference it by variable name via manifest.api_key_env. Per-user repair scripts in deploy/*.sh are machine-local and gitignored — they never embed the key; they source it from an existing .env on the machine or prompt the admin for it at deploy time.
/etc/brainrouter/env is owned root:aistack with mode 640. Only the root user and members of
the aistack group can read it. install.sh adds every human system user to aistack automatically.
Individual users never have write access to this file -- only root can update the API key.
The file is not world-readable; a user not in aistack cannot extract the Manifest API key.
Required core configuration/routing-state failures remain explicit startup errors. Optional benchmark storage and observability-settings failures are isolated as described above; legacy auto-mode compatibility is a read-only normalization, not weakened validation for new writes.
- Not a multi-provider cloud backend. Brainrouter implements OpenAI-compatible proxying and an Anthropic wire shim; Manifest handles downstream cloud providers and pricing.
- Not a general request-time model runner. brainrouter proxies chat traffic to Manifest/llama-swap and manages detached Server Mode containers for status/operations; the request router does not dispatch to Server Mode containers. It only spawns the single classifier llama-server for Bonsai.
- Not an auth layer. Loopback is the default trust boundary, not user authentication. Credentials live in provider configuration/environment.
- Not a conversation store. Chat history is managed by the harness. brainrouter is stateless for proxy calls; review sessions are in-memory and lost on daemon restart.
- Not a chat application. The Discord/Signal bridges are thin wrappers around the
ompCLI, not standalone chat bots with their own reasoning. - Not a benchmark executor or runtime attestation service. Registered capabilities/hashes and imported metrics do not prove artifact integrity, runtime feature support, or controlled comparability.
- Not a persistent live-traffic monitor. Samples and alert latches reset on restart; role preferences, project pins, benchmark records, toolbox ownership sidecars and explicit observation settings persist.
The implementation baseline was verified with 192 Rust tests and 31 Node browser-logic tests using fixtures/stub providers, without real inference. Coverage includes both wire protocols, event-ID/terminal-frame measurements, hybrid roles and continuation snapshots, legacy migrations/strict write APIs, actual daemon availability with invalid benchmark storage, bounded database/ response backpressure, synthetic import-to-reference flows, and alert threshold/dedup/recovery behavior. HTML script syntax and scoped Rust formatting were checked; a debug binary built successfully. This is not a performance benchmark, a real-browser/accessibility certification, or proof of deployment on a particular host.
The ai-toolbox-cockpit integration rollout (docs/design/ai-toolbox-cockpit-integration.md)
extended this to 391 Rust tests and 38 Node browser-logic tests, covering
catalog structural validation (including deliberately-permissive handling of
unrecognized/comfyui backend ids), the toolbox-container lifecycle,
cockpit-config read/apply, model-download job state machines, Server Mode
command construction for ds4/gufo, the migration-runner's version-3
gate (serving_runtimes/toolbox_catalog_snapshot), and the spider chart's
TEPR formula/cohort-pinning/insufficient-data behavior (v2 L1 purge: the
halogen/vllm/r9v Server Mode suites were deleted with their backends; the
explorer suites additionally run under --features benchmarks). None of ds4/
gufo's command builders have been run against real target hardware from
this environment — they are verified against upstream documentation/catalog
data and unit-tested in isolation, not integration-tested end-to-end on a real
podman/GPU host. Ship behind an explicit "unverified" badge until a real host
confirms them.
cargo test --locked
cargo clippy --locked --all-targets
node --test scripts/test-benchmark-ui.cjs
bash scripts/check-html-js.sh
cargo build --locked --bin brainrouterExisting unrelated Clippy warnings remain; strict CI must distinguish baseline warnings from new diagnostics. Transferring source/Git commits does not rebuild, install, or restart a running binary. Plan any runtime rollout separately.
These are ideas requiring separate approval, not acceptance claims for this release.
- Bonsai as a Strategic Context Expert -- real-time synthesis of cloud agent output
- Interrupt-and-redirect: user types a correction mid-stream; brainrouter cancels and re-prompts
- Persistent review sessions (SQLite) so they survive daemon restarts
- Persistent request history, per-request token costs and budget enforcement
- Non-model per-project settings, optional strict cloud fallback consent, reviewer diversity rules, and escalation ladders
- Quant Lab: actual llama-perplexity/PPL/KLD execution, compatible same-family reference checking, queued/sequential runs with cancellation/progress, per-token distributions and GGUF parsing
- Model Compare: complete common-suite execution across unrelated families, token-space compatibility checks, and blind human evaluation
- Automated benchmark scheduling, controlled regression campaigns, outbound notifications, or automatic model switching
- Parquet export and an integrated analytical warehouse
- Runtime-loaded component attestation (PLE/n-gram tables, adapters, caches, retrieval assets), beyond declared registry provenance
- Runtime context-selection controls: the retained
cli contextcommand has no implemented/api/contextserver route brainrouter cliparity for ds4/gufo. The toolbox backends are dashboard/API-only;cli toolboxes/cli upgrade toolboxremain llama.cpp-only. (v1's halogen/vllm/r9v scope in this item was removed by the v2 L1 purge.)- Register downloaded GGUFs into llama-swap. Designed in
docs/design/llama-swap-register-downloaded-gguf.md, not implemented. - Router integration for Server Mode backends ("PR 11b" / Feature B). Server Mode containers are not dispatchable as real request traffic;
Router/route()still only knows Manifest + llama-swap. This is the prerequisite forstatus_onlySankey/benchmark entries to ever become real routed traffic, and needs its own pre-implementation design note plus a live deployment to validate against before it touches request dispatch. R9V functional validation on real 2×AMD R9700 hardware.v1 scope; the r9v backend was removed by the v2 L1 purge (seedocs/design/v2-streamlining.mdL1) — re-validating this hardware path requires a new backend per the v2 engine-adapters plan (R6), not a revival of v1 code.- Bidirectional/continuous config.json sync with ai-toolbox-cockpit. Only an explicit, user-triggered single-write "apply" path exists; no locking/conflict protocol is defined for concurrent brainrouter+cockpit writes.
- ROCm/AMD-GPU auto-detection. brainrouter has no
rocminfo/sysfs-based hardware detection; R9V is offered as a selectable option and fails loudly on unsupported hardware rather than being auto-hidden.
The notes motivating Quant Lab/Model Compare are not fully implemented by the current explorer. Ember/Flash environment, cache-building, or provisioning work is not a shipped Brainrouter feature. Existing tables, chart labels, and imported measurements must not be presented as an execution pipeline or attestation.