Part of #5. This is a DRAFT to send upstream to stablyai/orca (as a Discussion or issue) once we're ready — not yet posted there. Tracking it here so it lives with the rest of the work. Edit freely before it goes out.
Orchestration: unify the coordinator DAG with the Orcastrator director
Status: design proposal + a standalone bugfix, prototyping on a fork (zaridan/orca) behind experimentalOrchestrators. Seeking direction before investing in the larger change.
TL;DR
Orca currently has two orchestration systems that share a name but don't compose:
- Orcastrator director (
orchestrator-launch.ts, OrchestratorMissionControl.tsx): a hidden worktree runs a Claude session that plans work and creates child worktrees per PR-track. Mission Control discovers workers purely by worktree lineage and reads .orcastrate/log.jsonl. It never calls the orchestration backend.
- Coordinator / task DAG (
src/main/runtime/orchestration/): a main-process coordinator runs a SQLite task DAG and dispatches tasks to terminals inside a single worktree. It never creates worktrees, and nothing in the Orcastrator UI reads its DB (OrchestratorMissionControl.tsx even comments "no orchestration RPC yet").
We'd like to unify them so the coordinator can drive worktree-based workers that Mission Control can see — which unlocks a live DAG "Control Panel" and an optional token-free "recipe" director. Before we build that, two questions for maintainers (below), and one bugfix we're sending regardless.
Ship-now bugfix (independent of the proposal)
Coordinator clash / multi-coordinator DB poaching (the issue the coordinator skill calls bug stablyai#4389). The orchestration DB is a single global namespace: tasks, dispatch_contexts, messages, decision_gates have no run scoping, the coordinator handle defaults to the literal 'coordinator', and run-start (getActiveCoordinatorRun → createCoordinatorRun in orchestration-gates.ts) is a TOCTOU race. Two coordinators consume each other's worker_done messages (silent hang) and trip the per-assignee dispatch guard ("already working on the same thing").
Fix: per-run isolation (per-run DB file or coordinator_run_id on all run-scoped tables) + per-run coordinator handle + atomic run-start. PR incoming on its own; no design buy-in needed.
The proposal (the part we want direction on)
Make the Orcastrator a director with two backends over the same coordinator/DAG:
- LLM director — today's behavior.
- Recipe director — token-free: compiles a fixed workflow into the DAG and runs it with no LLM in the director worktree. (Justified by determinism/reproducibility for known-shape jobs, not token cost — the director is a small fraction of a run's tokens.)
Both feed one Control Panel that renders the live DAG using Orca's existing visual vocabulary (WorkspaceCard/AgentRow/AgentStateDot).
Required changes, all verified additive/targeted:
- Coordinator creates worktrees. Add
createWorktree to CoordinatorRuntime, implemented on OrcaRuntimeService by wrapping the existing addWorktree (src/main/git/worktree.ts). Workers run in their own worktrees with lineage parent = director, so Mission Control sees them.
- Mission Control reads the DAG. Consume
listTasksWithDispatch (already exists in db.ts) instead of pure lineage.
- Coordinator resume-on-boot. A startup hook that resumes or reconciles
coordinator_runs WHERE status='running' — today the loop is in-memory with no resume (fine while a human-driven agent re-issues run, but not for a headless director).
- Renderer→main
orchestration.run binding (today CLI-only).
- Per-run isolation (the bugfix above) — prerequisite.
Questions for maintainers
- Is the terminal-dispatch-in-one-worktree model of the coordinator intentional, or an artifact? Any reason not to have it create worktrees per track?
- Do you want these two systems unified, or kept deliberately separate (LLM director = worktree/lineage; coordinator = terminal DAG for a different use case)? If separate, we'll scope the Control Panel narrowly.
- Appetite for a token-free "recipe" director at all, or is the LLM director the only intended planner?
- Anything in flight upstream that would collide with per-run DB isolation or a
CoordinatorRuntime.createWorktree?
We're happy to build it on the fork and bring a working demo; we'd just rather align on (1)–(3) before investing in the bridge.
Part of #5. This is a DRAFT to send upstream to
stablyai/orca(as a Discussion or issue) once we're ready — not yet posted there. Tracking it here so it lives with the rest of the work. Edit freely before it goes out.Orchestration: unify the coordinator DAG with the Orcastrator director
Status: design proposal + a standalone bugfix, prototyping on a fork (
zaridan/orca) behindexperimentalOrchestrators. Seeking direction before investing in the larger change.TL;DR
Orca currently has two orchestration systems that share a name but don't compose:
orchestrator-launch.ts,OrchestratorMissionControl.tsx): a hidden worktree runs a Claude session that plans work and creates child worktrees per PR-track. Mission Control discovers workers purely by worktree lineage and reads.orcastrate/log.jsonl. It never calls the orchestration backend.src/main/runtime/orchestration/): a main-process coordinator runs a SQLite task DAG and dispatches tasks to terminals inside a single worktree. It never creates worktrees, and nothing in the Orcastrator UI reads its DB (OrchestratorMissionControl.tsxeven comments "no orchestration RPC yet").We'd like to unify them so the coordinator can drive worktree-based workers that Mission Control can see — which unlocks a live DAG "Control Panel" and an optional token-free "recipe" director. Before we build that, two questions for maintainers (below), and one bugfix we're sending regardless.
Ship-now bugfix (independent of the proposal)
Coordinator clash / multi-coordinator DB poaching (the issue the coordinator skill calls bug stablyai#4389). The orchestration DB is a single global namespace:
tasks,dispatch_contexts,messages,decision_gateshave no run scoping, the coordinator handle defaults to the literal'coordinator', and run-start (getActiveCoordinatorRun→createCoordinatorRuninorchestration-gates.ts) is a TOCTOU race. Two coordinators consume each other'sworker_donemessages (silent hang) and trip the per-assignee dispatch guard ("already working on the same thing").Fix: per-run isolation (per-run DB file or
coordinator_run_idon all run-scoped tables) + per-run coordinator handle + atomic run-start. PR incoming on its own; no design buy-in needed.The proposal (the part we want direction on)
Make the Orcastrator a director with two backends over the same coordinator/DAG:
Both feed one Control Panel that renders the live DAG using Orca's existing visual vocabulary (
WorkspaceCard/AgentRow/AgentStateDot).Required changes, all verified additive/targeted:
createWorktreetoCoordinatorRuntime, implemented onOrcaRuntimeServiceby wrapping the existingaddWorktree(src/main/git/worktree.ts). Workers run in their own worktrees with lineage parent = director, so Mission Control sees them.listTasksWithDispatch(already exists indb.ts) instead of pure lineage.coordinator_runs WHERE status='running'— today the loop is in-memory with no resume (fine while a human-driven agent re-issuesrun, but not for a headless director).orchestration.runbinding (today CLI-only).Questions for maintainers
CoordinatorRuntime.createWorktree?We're happy to build it on the fork and bring a working demo; we'd just rather align on (1)–(3) before investing in the bridge.