A multi-agent LLM system that analyzes GitHub issues proposing new AI quickstarts against an existing catalog. Built with LangGraph for agent orchestration and designed for deployment on OpenShift AI.

flowchart TD
    A[Data Loading] --> B[Portfolio Analyst]
    B -->|portfolio_gaps| C

    subgraph C [Parallel Issue Analysis - up to 6 concurrent]
        direction LR
        I1[Issue #1] ~~~ I2[Issue #2] ~~~ IN[Issue #N]
    end

    C --> D[PDF Report]

The tool runs in four stages: load data from GitHub and the local catalog, analyze the portfolio for blind spots, evaluate each issue in parallel, and generate a PDF report.

Each issue is analyzed by three specialist agents running in parallel, then synthesized by the Coordinator:

flowchart TD
    START([GitHub Issue]) --> TA & PP & PS

    TA[Technical Analyst<br/><br/>Overlap check<br/>Dev stage<br/>Similar stack<br/>RAG tool calls]
    PP[Persona Panel<br/><br/>6 non-technical<br/>personas evaluate<br/>broad appeal]
    PS[Platform Specialist<br/><br/>Feature identification<br/>Platform fit<br/>New vs. reused]

    TA & PP & PS --> CO

    CO[Coordinator<br/><br/>Synthesize results<br/>Priority scoring<br/>Gap detection]

    CO --> END([Final Analysis])

The Technical Analyst uses LLM tool-calling to search a vector index of published quickstart content (semantic search, README lookups, code inspection) over multiple rounds of research. The Platform Specialist and Persona Panel use direct prompting without tools.

flowchart TD
    subgraph S1 [Step 1: Data Loading]
        GH[Fetch GitHub issues] --> CAT[Load quickstart catalog]
        CAT --> REPO[Fetch org repositories]
        REPO --> FEAT[Load features + coverage]
    end

    subgraph S2 [Step 2: Portfolio Analysis]
        PA[Build catalog context] --> LLM1[LLM: strategic gap analysis]
        LLM1 --> GAPS[Extract portfolio_gaps dict]
    end

    subgraph S3 [Step 3: Parallel Issue Analysis]
        direction TB
        CHK{Cached?}
        CHK -->|Yes| USE[Return cached result]
        CHK -->|No| GRAPH[Run per-issue LangGraph]
        GRAPH --> CACHE[Cache result]
    end

    subgraph S4 [Step 4: Report Generation]
        COLLECT[Collect all analyses] --> PDF[Generate PDF report]
    end

    S1 --> S2 --> S3 --> S4

Step 1 — Data Loading. GitHub issues are fetched from the rh-ai-quickstart organization. The published quickstart catalog is loaded from data/catalog.yaml (synced from ai-quickstart-pub). Organization repositories, the feature catalog (data/features.yaml), and the coverage matrix (data/coverage.yaml) are also loaded. All GitHub data is cached locally.

Step 2 — Portfolio Analysis. The Portfolio Analyst receives a description of every published quickstart and produces a strategic gap analysis: underserved industries, missing use cases, undemonstrated capabilities, and expected adjacent quickstarts. The result is cached and a simplified portfolio_gaps dict is passed into every issue analysis.

Step 3 — Parallel Issue Analysis. Each issue is checked against the cache first. Uncached issues enter the per-issue LangGraph where all three specialist agents run in parallel. Once they finish, the Coordinator merges their outputs, applies a consistency check (e.g., if use case overlaps are listed, overlap level cannot be "Unique"), matches the issue against portfolio gaps, and calculates a priority score. The final result is cached.

Step 4 — Report Generation. If --report is passed, all analyses are collected and rendered into a PDF containing a portfolio blind spots section, a summary table, and detailed per-issue breakdowns.

The refresh command prepares the data pipeline that powers the Technical Analyst's RAG tools:

flowchart LR
    SYNC[sync<br/>Fetch catalog from<br/>ai-quickstart-pub] --> INDEX[index<br/>Clone repos, extract<br/>content, embed chunks]
    INDEX --> COV[sync-coverage<br/>Detect features in<br/>indexed quickstarts]
    COV --> READY([Ready for analysis])

The indexer extracts content from each quickstart repository (READMEs, Helm charts, notebooks, dependency files, config templates), chunks it with markdown-aware splitting, and embeds it into PostgreSQL with pgvector for similarity search.

Each issue analysis produces the following fields, which appear in both the PDF report and the cached JSON:

Priority is scored from factors including overlap level, development stage, audience appeal, new platform feature demonstrations, and portfolio gap coverage, then mapped to Low/Medium/High.

git clone <repo-url>
cd issue-review
python -m venv venv
source venv/bin/activate
pip install -e .

Copy the example environment file and fill in your values:

cp .env.example .env

Required:

The tool works with any OpenAI-compatible endpoint (Anthropic, OpenAI, vLLM, Ollama). See .env.example for provider-specific examples.

Optional:

# 1. Start PostgreSQL
docker-compose up -d

# 2. Initialize data (sync catalog, index repos, detect coverage)
issue-review refresh

# 3. Analyze all issues and generate a PDF report
issue-review analyze --report

# Analyze a specific issue
issue-review analyze -i 42

# Force re-analysis (ignore cache)
issue-review analyze --reanalyze --report

issue-review/
├── main.py                  # CLI entry point
├── config.py                # Environment variable loading
├── prompts.py               # All LLM prompt templates
├── report_generator.py      # PDF report rendering
├── comment_generator.py     # GitHub comment formatting
├── analysis_store.py        # Analysis caching (thread-safe)
├── data_sources.py          # GitHub API data fetching
├── agents/
│   ├── state.py             # Shared state types and dataclasses
│   ├── graph.py             # Per-issue LangGraph (parallel agents → coordinator)
│   ├── report_graph.py      # Top-level orchestrator (portfolio → parallel issues)
│   ├── technical.py         # Technical Analyst agent
│   ├── personas.py          # Persona Panel agent
│   ├── platform.py          # Platform Specialist agent
│   ├── portfolio.py         # Portfolio Analyst agent
│   └── coordinator.py       # Coordinator (synthesis + scoring)
├── llm/
│   ├── client.py            # OpenAI-compatible LLM client
│   ├── tools.py             # Tool-calling infrastructure
│   ├── parsing.py           # JSON extraction from LLM responses
│   ├── callbacks.py         # Langfuse tracing integration
│   └── guardrails.py        # Content safety validation
├── tools/
│   ├── research.py          # RAG tools (semantic search, README, code)
│   └── features.py          # Feature catalog tools
├── indexer/
│   ├── content.py           # Content extraction and chunking
│   ├── embeddings.py        # Embedding generation and vector storage
│   ├── sync.py              # Catalog sync from GitHub
│   └── coverage.py          # Feature coverage detection
├── data/
│   ├── catalog.yaml         # Published quickstart catalog
│   ├── features.yaml        # OpenShift AI feature definitions
│   ├── coverage.yaml        # Feature coverage matrix
│   └── personas.yaml        # Persona definitions
├── cache/                   # Runtime caches (gitignored)
├── reports/                 # Generated PDFs (gitignored)
├── .env.example             # Environment template
├── pyproject.toml           # Dependencies and project metadata
└── docker-compose.yml       # PostgreSQL + pgvector

See CONTRIBUTING.md for guidance on adding personas, features, agents, and tools.