A lightweight, general-purpose framework for evaluating GPU kernel correctness and performance.

• Three Evaluation Modes: Analyze, Compare, Benchmark
• Heterogeneous Hardware: AMD (HIP) and NVIDIA (CUDA) GPUs
• Execution Environments: Local, Sandbox Container, and Remote Ray Cluster
• Hardware Control: Hardware-aware kernel evaluation under controlled execution settings
• Trace Analysis: TraceLens integration for performance profiling analysis
• MCP Server: Model Context Protocol integration for AI agents
• Structured Reports: JSON output for pipeline integration

• Python 3.10+
• AMD ROCm (HIP) or NVIDIA CUDA toolchain (for kernel compilation/profiling)
• rocprof-compute (AMD) or ncu (NVIDIA) if you enable performance profiling
• Docker (required for Benchmark mode)

# Basic installation
pip install git+https://github.com/AMD-AGI/Magpie.git

git clone https://github.com/AMD-AGI/Magpie.git
cd Magpie

# Editable install (recommended for development)
pip install -e .

# Or use make
make install

# Analyze a kernel using a config file
magpie analyze --kernel-config Magpie/kernel_config.yaml.example

# Compare kernels directly
magpie compare kernel_v1.hip kernel_v2.hip

# Benchmark vLLM with torch profiling
magpie benchmark --benchmark-config examples/benchmark_vllm.yaml

# Run MCP server
python -m Magpie.mcp

Note: You can also use python -m Magpie instead of magpie command.

📖 See Benchmark Mode Documentation for detailed usage.

Key categories:

• gpu: force device selection and hardware control (power/frequency).
• scheduler: local/container/remote execution and scheduling behavior.
• performance: profiling and profiler configuration.
• logging: log levels and output formatting.

See Magpie/kernel_config.yaml.example for full examples.

Example configs live in examples/:

MCP configuration example: Magpie/mcp/config.json

Available tools:

• analyze - Analyze kernel correctness and performance
• compare - Compare multiple kernel implementations
• hardware_spec - Query GPU hardware specifications
• configure_gpu - Configure GPU power and frequency
• discover_kernels - Scan a project and suggest analyzable kernels/configs
• suggest_optimizations - Suggest performance optimizations from analyze output
• create_kernel_config - Generate a kernel config YAML for analyze

make install-dev
make lint
make format

├── README.md
├── LICENSE
├── .gitignore
├── pyproject.toml       # Package configuration (pip install)
├── requirements.txt
├── Makefile
├── examples/            # Example configurations
├── docs/                # Documentation
│   └── benchmark.md     # Benchmark mode documentation
└── Magpie/
    ├── __init__.py          # Package initialization
    ├── __main__.py          # Entry point for python -m Magpie
    ├── main.py              # CLI implementation
    ├── config.yaml          # Framework configuration
    ├── kernel_config.yaml.example
    ├── config/              # Configuration classes
    ├── core/                # Core engine components
    ├── eval/                # Evaluation pipeline
    ├── modes/               # Evaluation modes
    │   ├── analyze_eval/    # Single kernel analysis
    │   ├── compare_eval/    # Multi-kernel comparison
    │   └── benchmark/       # Framework-level benchmarking
    │       ├── benchmarker.py   # Benchmark orchestration
    │       ├── config.py        # Benchmark configuration
    │       ├── tracelens.py     # TraceLens integration
    │       └── result.py        # Result data structures
    ├── mcp/                 # MCP Server
    │   ├── __init__.py
    │   ├── __main__.py      # Entry point for python -m Magpie.mcp
    │   ├── server.py        # MCP server implementation
    │   └── config.json      # MCP client configuration
    └── utils/               # Utility functions

MIT License. See LICENSE.