AI-powered code review bot for GitHub pull requests, deployed as a GitHub App with inline review comments.
- Inline review comments: Posts GitHub PR reviews with comments on specific lines
- Mathlib4-aware: Reviews check style, naming conventions, and mathlib-specific patterns
- Smart caching: Reviews stored and reused for identical PR state + prompt combinations
- Feedback collection: Users can provide feedback on review quality
- Polling daemon: Watches for
@botbakitriggers every 2 minutes
Botbaki runs as a GitHub App: botbaki-review[bot]
This provides:
- Separate identity (not impersonating a user)
- Higher rate limits (5000+ requests/hour)
- No abuse detection issues
Trigger botbaki by commenting on a PR:
| Command | Description |
|---|---|
@botbaki review |
Generate review with inline comments |
@botbaki review --single |
Post as single comment (no inline) |
@botbaki review --diff-only |
Review without timeline context |
@botbaki feedback <text> |
Provide feedback on the last review |
@botbaki help |
Show available commands |
- Python 3.10+
- GitHub App credentials (for GitHub API access)
- Anthropic API key (for Claude reviews)
git clone https://github.com/kim-em/botbaki.git
cd botbaki
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt- Go to https://github.com/settings/apps/new (or org settings for org app)
- Set permissions:
- Contents: Read-only
- Issues: Read and write
- Pull requests: Read and write
- Disable webhooks (we use polling)
- Generate and download private key
- Install the app on your repository
mkdir -p ~/.config/botbaki
# GitHub App credentials
echo "YOUR_APP_ID" > ~/.config/botbaki/github-app-id
cp path/to/private-key.pem ~/.config/botbaki/github-app-key.pem
echo "YOUR_INSTALLATION_ID" > ~/.config/botbaki/github-installation-id
chmod 600 ~/.config/botbaki/github-app-key.pem
# Anthropic API key
echo "ANTHROPIC_API_KEY=sk-ant-..." > ~/.config/botbaki/env
chmod 600 ~/.config/botbaki/env# Manual run
source .venv/bin/activate
python3 -m src.cli daemon
# Or install as systemd service
cd service && ./install.sh# Generate a review (outputs to stdout)
botbaki review leanprover-community/mathlib4 32904
botbaki review leanprover-community/mathlib4 32904 --diff-only # No timeline
botbaki review leanprover-community/mathlib4 32904 --force # Bypass cache
# Sync PR data from GitHub
botbaki sync leanprover-community/mathlib4 --pr 32904 # Single PR
botbaki sync leanprover-community/mathlib4 --incremental # Changed PRs only
botbaki sync leanprover-community/mathlib4 --since 2026-01-01 # Full sync since date
# List and view PRs
botbaki list leanprover-community/mathlib4 --state open --limit 20
botbaki show leanprover-community/mathlib4 32904
# Search comments (FTS5 syntax)
botbaki search "omega tactic"
# Repository statistics
botbaki stats leanprover-community/mathlib4
botbaki stats leanprover-community/mathlib4 --author kim-em
# Run daemon
botbaki daemon
botbaki daemon --repo leanprover-community/mathlib4 --interval 120
# Check GitHub API rate limit
botbaki rate-limit# Show feedback report (unhandled only, with emoji reactions)
botbaki feedback-report
botbaki feedback-report --all # Include handled feedback
botbaki feedback-report --since 2026-01-01 # Filter by date
botbaki feedback-report --no-reactions # Skip fetching reactions from GitHub
# Analyze feedback and generate improved prompt
botbaki analyze-feedback # Generate prompts/mathlib4/YYYY-MM-DD.md
botbaki analyze-feedback --dry-run # Preview without API calls
botbaki analyze-feedback --verbose # Show progress
# Mark feedback as handled after incorporating into prompts
botbaki mark-feedback-handled
botbaki mark-feedback-handled --dry-runThe daemon polls for @botbaki comments every 2 minutes:
botbaki daemonManagement (when installed as systemd service):
sudo systemctl status botbaki # Check status
sudo systemctl restart botbaki # Restart
sudo journalctl -u botbaki -f # Follow logsAll credentials stored in ~/.config/botbaki/:
| File | Description |
|---|---|
env |
Anthropic API key (ANTHROPIC_API_KEY=...) |
github-app-id |
GitHub App ID |
github-app-key.pem |
GitHub App private key |
github-installation-id |
Installation ID for target repo |
github-app-slug |
App slug for bot login (optional, defaults to "botbaki") |
Review prompts are in prompts/{repo}/YYYY-MM-DD.md:
- Use
{diff},{timeline},{title},{author}, etc. as placeholders - Reference files in
prompts/{repo}/references/for style guides - Latest prompt (by filename) is used by default
SQLite database (data/github_prs.db) contains:
- pull_requests - PR metadata
- commits - Commits on PR branches
- review_comments - Inline code comments
- issue_comments - General PR discussion
- reviews - Review submissions
- pr_reviews - Generated AI reviews (cached)
- review_feedback - User feedback on reviews
- processed_triggers - Tracked bot triggers
┌─────────────────┐ ┌──────────────┐
│ GitHub │────▶│ Polling │
│ (PRs/Comments) │ │ Daemon │
└─────────────────┘ └──────┬───────┘
│
▼
┌──────────────┐
│ Trigger │
│ Processing │
└──────┬───────┘
│
┌────────────────┼────────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ SQLite │ │ Claude │ │ GitHub │
│ Database │ │ API │ │ API │
└──────────┘ └──────────┘ └──────────┘
(PyGithub)
Key modules:
src/cli.py- Command-line interfacesrc/daemon.py- Polling daemonsrc/review.py- AI review generation with structured outputsrc/analysis.py- Feedback analysis and prompt improvementsrc/github_client.py- GitHub App authentication via PyGithubsrc/sync.py- PR data synchronizationsrc/database.py- SQLite operations
Currently deployed to chonk.lean-fro.org:
# SSH to chonk
ssh chonk
# Check status
sudo systemctl status botbaki
sudo journalctl -u botbaki -f
# Deploy updates
cd ~/projects/botbaki
git pull
sudo systemctl restart botbakiMIT