Skip to content

Latest commit

 

History

History
400 lines (306 loc) · 11.3 KB

File metadata and controls

400 lines (306 loc) · 11.3 KB

High-Level Design (HLD)

System overview, container views, and primary flows

1. System Overview

AgentStack is a multi-agent orchestration framework that enables Claude Code to manage specialized AI agents, persistent memory, and complex workflows through the Model Context Protocol (MCP).

1.1 System Goals

  1. Agent Management: Spawn and coordinate specialized agents (coder, tester, reviewer, etc.)
  2. Agent Identity: Persistent lifecycle management for agents (create, activate, deactivate, retire)
  3. Persistent Memory: Store and retrieve context with full-text and semantic search
  4. Task Coordination: Queue, prioritize, and distribute tasks to agents
  5. Semantic Drift Detection: Detect when task descriptions are too similar to ancestors
  6. Resource Exhaustion Monitoring: Track and prevent runaway agents consuming excessive resources
  7. Consensus Checkpoints: Require validation before high-risk tasks can spawn subtasks
  8. Workflow Automation: Execute multi-phase workflows with validation
  9. Extensibility: Support plugins for custom agents, tools, and hooks

1.2 Key Stakeholders

Stakeholder Interest
Claude Code Users Access to specialized agents via MCP tools
Developers Programmatic API for agent orchestration
Plugin Authors Extension points for customization

2. System Context

flowchart TB
    subgraph External
        CC[Claude Code]
        ANT[Anthropic API]
        OAI[OpenAI API]
        OLL[Ollama]
        GH[GitHub]
    end

    subgraph AgentStack
        MCP[MCP Server]
        CORE[Core Services]
        DB[(SQLite)]
    end

    CC <-->|stdio| MCP
    MCP --> CORE
    CORE --> DB
    CORE --> ANT
    CORE --> OAI
    CORE --> OLL
    CORE --> GH
Loading

2.1 External Dependencies

System Protocol Purpose
Claude Code MCP over stdio Primary client interface
Anthropic API HTTPS Claude chat completions
OpenAI API HTTPS Chat and embeddings
Ollama HTTP (localhost) Local LLM inference
GitHub gh CLI Issue/PR operations

3. Container Architecture

3.1 MCP Server Container

Responsibility: Expose all capabilities as MCP tools for Claude Code.

Components:

  • Request handler for ListTools and CallTool
  • 46 registered tools across 8 categories
  • JSON-RPC style response formatting

Interfaces:

  • Input: MCP protocol messages via stdin
  • Output: Tool results via stdout

3.2 Agent Manager Container

Responsibility: Define, register, and manage agent instances.

Components:

  • Registry: Maps agent types to definitions
  • Spawner: Creates and tracks active agents
  • Definitions: 11 built-in agent types

Agent Types:

Type Capabilities
coder write-code, edit-code, refactor, debug
tester write-tests, run-tests, coverage-analysis
reviewer code-review, security-review, best-practices
researcher search-code, analyze-patterns, gather-requirements
adversarial attack-surface-analysis, security-testing, vulnerability-detection
architect system-design, technical-decisions, documentation
coordinator task-decomposition, agent-coordination
analyst data-analysis, performance-profiling, metrics
devops deployment, infrastructure, monitoring
documentation docs-writing, api-docs, guides
security-auditor security-audit, compliance-check, threat-modeling

3.3 Memory Manager Container

Responsibility: Persist and search key-value data with metadata.

Components:

  • SQLiteStore: Core persistence layer
  • FTSSearch: BM25-based full-text search
  • VectorSearch: Optional embedding-based semantic search

Storage Schema:

memory (key, namespace, content, embedding, metadata)
sessions (id, status, timestamps, metadata)
tasks (id, session_id, agent_type, status, input, output)

3.4 Coordination Container

Responsibility: Manage task execution and inter-agent communication.

Components:

  • TaskQueue: Priority-based task queue with events
  • MessageBus: Pub/sub for agent-to-agent messages
  • HierarchicalCoordinator: One coordinator managing workers

3.5 Workflow Runner Container

Responsibility: Execute multi-phase workflows with validation.

Phases:

  1. Inventory: Discover documents/resources
  2. Analysis: Analyze current state
  3. Sync: Apply updates
  4. Consistency: Verify cross-document consistency
  5. Adversarial: Red-team validation
  6. Reconciliation: Fix failures and retry

4. Primary Data Flows

4.1 Agent Spawn Flow

sequenceDiagram
    participant CC as Claude Code
    participant MCP as MCP Server
    participant REG as Agent Registry
    participant SP as Agent Spawner

    CC->>MCP: agent_spawn("coder", {name: "my-coder"})
    MCP->>REG: getAgentDefinition("coder")
    REG-->>MCP: AgentDefinition
    MCP->>SP: spawnAgent("coder", options)
    SP->>SP: Generate UUID
    SP->>SP: Create SpawnedAgent
    SP-->>MCP: SpawnedAgent
    MCP-->>CC: {success: true, agent, prompt}
Loading

4.2 Memory Search Flow

sequenceDiagram
    participant CC as Claude Code
    participant MCP as MCP Server
    participant MM as Memory Manager
    participant VS as Vector Search
    participant FTS as FTS Search

    CC->>MCP: memory_search({query: "pattern"})
    MCP->>MM: search(query, options)

    alt Vector Search Enabled
        MM->>VS: search(query)
        VS->>VS: Generate embedding
        VS->>VS: Cosine similarity
        VS-->>MM: Vector results
    end

    MM->>FTS: search(query)
    FTS->>FTS: BM25 ranking
    FTS-->>MM: FTS results

    MM->>MM: Merge results
    MM-->>MCP: SearchResults
    MCP-->>CC: {count, results}
Loading

4.3 Task Coordination Flow

sequenceDiagram
    participant COORD as Coordinator
    participant TQ as Task Queue
    participant MB as Message Bus
    participant WORKER as Worker Agent

    COORD->>TQ: enqueue(task, priority)
    TQ-->>COORD: task:added event

    COORD->>COORD: getAvailableWorker()

    alt No idle worker
        COORD->>COORD: spawnAgent(task.type)
    end

    COORD->>TQ: dequeue(worker.type)
    COORD->>TQ: assign(taskId, workerId)
    COORD->>MB: send(coordinator, worker, "task:assign", task)
    MB-->>WORKER: Message

    Note over WORKER: Execute task

    WORKER->>MB: send(worker, coordinator, "task:completed")
    MB-->>COORD: Message
    COORD->>TQ: complete(taskId)
Loading

4.4 Workflow Execution Flow

sequenceDiagram
    participant CLI as CLI/API
    participant WR as Workflow Runner
    participant PE as Phase Executor

    CLI->>WR: run(config)
    WR->>WR: Initialize context
    WR-->>CLI: workflow:start event

    loop For each phase
        WR->>PE: executePhase(phase, context)
        PE->>PE: Phase-specific logic
        PE-->>WR: PhaseResult
        WR-->>CLI: phase:complete event
    end

    alt Adversarial Failed
        loop Reconciliation (max 3)
            WR->>PE: executePhase("sync", context)
            WR->>PE: executePhase("adversarial", context)
        end
    end

    WR->>WR: generateReport()
    WR-->>CLI: WorkflowReport
Loading

4.5 Consensus Checkpoint Flow

sequenceDiagram
    participant Agent as Parent Agent
    participant CS as ConsensusService
    participant DB as SQLite
    participant REV as Reviewer

    Agent->>CS: checkConsensusRequired(agentType, input, parentTaskId)
    CS->>CS: estimateRiskLevel(agentType, input)

    alt Risk requires consensus
        CS->>DB: createCheckpoint(taskId, subtasks, riskLevel)
        DB-->>CS: Checkpoint (pending)
        CS-->>Agent: {required: true, checkpointId}

        Note over REV: Reviewer evaluates subtasks
        REV->>CS: approveCheckpoint(checkpointId, feedback)
        CS->>DB: Update status to 'approved'
        CS->>DB: Log 'approved' event
        CS-->>Agent: Proceed with subtasks
    else Risk below threshold
        CS-->>Agent: {required: false}
        Agent->>Agent: Spawn subtasks directly
    end
Loading

5. Integration Points

5.1 MCP Tool Categories

Category Tool Count Purpose
Agent 6 Spawn, list, stop, status, types, update
Identity 8 Create, get, list, update, activate, deactivate, retire, audit
Memory 5 Store, search, get, list, delete
Task 8 Create, assign, complete, list, get, check_drift, get_relationships, drift_metrics
Consensus 5 Check, list_pending, get, approve, reject
Session 4 Start, end, status, active
System 3 Status, health, config
GitHub 7 Issues and PRs

Total: 46 tools

Note: Review loop functionality is available via programmatic API (createReviewLoop) and CLI, but not exposed as MCP tools.

5.2 Provider Interface

All LLM providers implement:

interface LLMProvider {
  name: string;
  chat(messages: ChatMessage[], options?: ChatOptions): Promise<ChatResponse>;
  embed?(text: string): Promise<number[]>;  // Optional - OpenAI and Ollama only
}

API Providers:

Provider Embeddings Model
Anthropic No claude-sonnet-4-20250514
OpenAI Yes gpt-4o / text-embedding-3-small
Ollama Yes llama3.2 / nomic-embed-text

CLI Providers:

Provider CLI Tool Default Model
Claude Code claude sonnet
Gemini CLI gemini gemini-2.0-flash
Codex codex -

CLI providers execute tasks through external command-line tools and are useful for interactive agent workflows or when using pre-authenticated CLI sessions.

5.3 Plugin Interface

Plugins can extend:

interface AgentStackPlugin {
  name: string;
  version: string;
  agents?: AgentDefinition[];    // Custom agent types
  tools?: MCPToolDefinition[];    // Additional MCP tools
  hooks?: HookDefinition[];       // Lifecycle hooks
  providers?: ProviderDefinition[]; // Custom LLM providers
  init?(config): Promise<void>;
  cleanup?(): Promise<void>;
}

6. Cross-Cutting Concerns

6.1 Configuration

  • JSON config file with environment variable interpolation
  • Zod schema validation with defaults
  • Singleton pattern for cached access

6.2 Logging

  • Hierarchical logger with child contexts
  • Levels: debug, info, warn, error
  • JSON metadata support

6.3 Error Handling

  • Consistent try-catch patterns
  • Graceful degradation for optional features
  • Descriptive error messages in MCP responses

6.4 State Management

State Scope Persistence
Config Global singleton File-based
Memory Global singleton SQLite
Agents Module-level maps In-memory
Tasks Per-coordinator In-memory
Sessions Memory Manager SQLite

7. Non-Functional Requirements

7.1 Performance

  • Synchronous SQLite operations for reliability
  • In-memory agent and task tracking
  • Batch embedding support

7.2 Scalability

  • Configurable max concurrent agents (1-20)
  • Priority-based task queue
  • Worker pooling in hierarchical coordinator

7.3 Reliability

  • SQLite transactions for data integrity
  • Task requeue on failure
  • Graceful shutdown with cleanup

8. Related Documents