MCRAG is an intelligent code generation system that uses multiple Large Language Models to generate, review, and iteratively refine code based on user prompts. The system employs a collaborative AI approach where one LLM generates code while two critic LLMs provide feedback for continuous improvement.

👉 Click here for the project demo !

The system consists of three main components:

• FastAPI Server: RESTful API endpoints for code generation requests
• Redis Database: Session storage and data persistence
• Multi-LLM Integration: Gemini 2.5 Flash (generator), GPT-4o (critic1), DeepSeek R1 (critic2)
• Background Processing: Asynchronous workflow execution
• Rate Limiting: Built-in API rate limiting with exponential backoff

• Modern React Interface: Clean, responsive user interface
• Real-time Status Updates: Live progress tracking during generation
• Code Display: Syntax-highlighted code output with copy functionality
• Iteration Tracking: Complete history of code evolution and reviews

• Iterative Refinement: Up to 3 cycles of generation and improvement
• Multi-LLM Collaboration: Generator creates code, critics provide feedback
• Intelligent Stopping: Automatic termination based on feedback quality
• Error Recovery: Comprehensive error handling and recovery mechanisms

• FastAPI: Modern Python web framework
• Redis: In-memory data structure store
• Pydantic: Data validation and serialization
• AsyncIO: Asynchronous programming support
• Uvicorn: ASGI server implementation

• React 18: Modern React with hooks
• Tailwind CSS: Utility-first CSS framework
• Heroicons: Beautiful SVG icons
• Axios: HTTP client for API communication
• CRACO: Create React App Configuration Override

• OpenRouter API: Unified access to multiple LLM providers
• Multi-Model Support: Gemini 2.0 Flash, GPT-4o, DeepSeek R1
• Rate Limiting: Intelligent API usage management
• Model Flexibility: Easy model switching and configuration

mcrag/
├── .env                        
├── .env.example               # Environment template
├── .gitignore
├── README.md
├── start-dev.sh              # main development startup script
├── backend/
│   ├── llm_services.py       # LLM integration and services
│   ├── models.py             # Pydantic data models
│   ├── requirements.txt      # Python dependencies
│   ├── review_workflow.py    # Code generation workflow engine
│   ├── server.py            # FastAPI server and routes
│   └── start-dev.sh         # Backend startup script
├── evaluation/
│   ├── evaluate_mcrag.py     # Comprehensive evaluation framework
│   ├── quality_evaluator.py  # Code quality assessment
│   ├── quick_eval.py         # Quick single-case testing
│   ├── requirements.txt      # Evaluation dependencies
│   └── test_cases.py         # Test case definitions
└── frontend/
    ├── craco.config.js       # Create React App configuration
    ├── package.json          # Node.js dependencies and scripts
    ├── postcss.config.js     # PostCSS configuration
    ├── tailwind.config.js    # Tailwind CSS configuration
    ├── start-dev.sh          # Frontend startup script
    ├── public/
    │   └── index.html        # HTML template
    └── src/
        ├── App.css          # Application styles
        ├── App.js           # Main React application
        ├── index.css        # Global styles
        ├── index.js         # React entry point
        └── components/
            ├── CodeSubmission.js  # Code generation form
            ├── ReviewProgress.js  # Progress tracking component
            └── ReviewResult.js    # Results display component

• Python 3.8+ with pip or uv
• Node.js 16+ with npm
• Redis server
• OpenRouter API key (provides access to multiple LLM providers)

1. Clone and navigate to the project:

git clone https://www.github.com/sohv/mcrag.git  
cd mcrag

2. Set up environment variables:

# Copy the example environment file
cp .env.example .env

# Edit .env with your API keys and configuration
nano .env

3. Start the development environment:

# This starts both backend and frontend automatically
./start-dev.sh

The application will be available at:

• Frontend: http://localhost:3000
• Backend API: http://127.0.0.1:8001
• API Documentation: http://127.0.0.1:8001/docs

This project uses a centralized .env file in the root directory. Both backend and frontend reference this shared configuration.

Server Configuration:

• BACKEND_HOST=127.0.0.1 - Backend server host
• BACKEND_PORT=8001 - Backend server port
• FRONTEND_PORT=3000 - Frontend development server port
• REACT_APP_BACKEND_URL=http://127.0.0.1:8001 - Frontend API endpoint

Database Configuration:

• REDIS_URL=redis://localhost:6379 - Redis connection
• MONGO_URL=mongodb://localhost:27017 - MongoDB connection (if used)
• DB_NAME=test_database - Database name

LLM Configuration (OpenRouter):

• OPENROUTER_API_KEY - OpenRouter API key for unified LLM access
• OPENROUTER_URL - OpenRouter API endpoint (default: https://openrouter.ai/api/v1)

Model Configuration (OpenRouter model names):

• MODEL_GENERATOR=google/gemini-2.5-flash - Code generator model
• MODEL_CRITIC1=openai/gpt-4o - First critic model
• MODEL_CRITIC2=deepseek/deepseek-r1 - Second critic model

Start services individually:

# Backend only
cd backend && ./start-dev.sh

# Frontend only (in another terminal)
cd frontend && ./start-dev.sh

Manual setup:

# Backend
cd backend
pip install -r requirements.txt  # or: uv pip install -r requirements.txt
source ../.env
uvicorn server:app --reload --host $BACKEND_HOST --port $BACKEND_PORT

# Frontend (in another terminal)
cd frontend
npm install
source ../.env
npm start

1. Navigate to backend directory:

cd backend

2. Install Python dependencies:

# Using pip
pip install -r requirements.txt

# Or using uv (recommended)
uv pip install -r requirements.txt

3. Start Redis server (if not running):

redis-server

4. Run the FastAPI server:

# Using environment from root .env
source ../.env
uvicorn server:app --reload --host $BACKEND_HOST --port $BACKEND_PORT

# Or use the startup script
./start-dev.sh

1. Navigate to frontend directory:

cd frontend

2. Install Node.js dependencies:

npm install

3. Start the development server:

# Using environment from root .env
source ../.env
npm start

# Or use the startup script
./start-dev.sh

• User submits prompt with language and requirements
• Generator LLM creates initial code version
• System saves code with version 1

• Critic 1 (GPT-4o) analyzes code for quality and improvements
• Critic 2 (DeepSeek R1) provides alternative perspective
• Both critics provide scores, suggestions, and severity ratings

• Generator LLM evaluates both critic reviews
• Ranks feedback quality and usefulness
• Creates incorporation plan for improvements

• System decides whether to continue or stop refinement
• Continues if critic feedback is valuable (high scores)
• Stops if max iterations reached or poor feedback quality

• Generator implements improvements based on critic feedback
• Creates new code version incorporating suggestions
• Process repeats until completion criteria met

• Complete generation history preserved
• Final code version marked as result
• All iterations and reviews available for analysis

MCRAG includes a comprehensive evaluation system to quantify performance across multiple dimensions:

Test the system with a single case:

cd evaluation
python quick_eval.py

Run comprehensive testing across all languages:

cd evaluation  
python evaluate_mcrag.py

Test specific programming languages:

python evaluate_mcrag.py --languages python javascript

• Functionality (25%) - Syntax correctness, logic implementation
• Code Quality (20%) - Style, structure, best practices
• Completeness (20%) - Required features, edge cases
• Efficiency (15%) - Algorithm optimality, performance
• Error Handling (10%) - Exception handling, validation
• Documentation (10%) - Comments, clarity, maintainability

MCRAG EVALUATION SUMMARY
============================================================
Total Tests: 15
Successful: 14  
Success Rate: 93.3%
Avg Processing Time: 45.2s

QUALITY METRICS
------------------------------
Overall         0.812 (±0.082)
Functionality   0.892 (±0.067)
Code Quality    0.834 (±0.089)
Completeness    0.876 (±0.074)

{
  "id": "uuid",
  "user_prompt": "string",
  "language": "enum",
  "requirements": "optional string",
  "status": "enum",
  "session_id": "uuid"
}

{
  "id": "uuid",
  "request_id": "uuid",
  "session_id": "uuid",
  "generated_code": "string",
  "explanation": "string",
  "version": "integer",
  "created_at": "datetime"
}

{
  "id": "uuid",
  "session_id": "uuid",
  "code_id": "uuid",
  "critic_type": "CRITIC1|CRITIC2",
  "llm_model": "string",
  "review_text": "string",
  "suggestions": "list[string]",
  "severity_rating": "float",
  "confidence_score": "float",
  "processing_time": "float"
}

All configuration is managed through the centralized .env file in the project root. Copy .env.example to .env and configure the following:

• OPENROUTER_API_KEY: OpenRouter API for unified LLM access

• BACKEND_HOST: Backend server host (default: 127.0.0.1)
• BACKEND_PORT: Backend server port (default: 8001)
• FRONTEND_PORT: Frontend development server port (default: 3000)
• REACT_APP_BACKEND_URL: Frontend API endpoint (default: http://127.0.0.1:8001)

• REDIS_URL: Redis connection string (default: redis://localhost:6379)
• MONGO_URL: MongoDB connection string (optional)
• DB_NAME: Database name (optional)

• MODEL_GENERATOR: Generator model (default: google/gemini-2.5-flash)
• MODEL_CRITIC1: First critic model (default: openai/gpt-4o)
• MODEL_CRITIC2: Second critic model (default: deepseek/deepseek-r1)

• OPENROUTER_URL: OpenRouter API endpoint (default: https://openrouter.ai/api/v1)
• MAX_ITERATIONS: Maximum refinement cycles (default: 3)
• DEBUG: Enable debug mode (default: true)
• LOG_LEVEL: Logging level (default: INFO)

The system implements intelligent rate limiting for OpenRouter API:

• Configurable rate limiting per model provider
• Automatic retry mechanisms with proper delays
• Unified rate limiting across all models

1. System environment variables
2. Root .env file
3. Default values in code

• Minimal, clean code generation with brief comments
• Modern language features and best practices
• No multi-line comments or verbose documentation in generated code
• Separation of code and explanations

1. Follow existing code style and patterns
2. Add comprehensive tests for new features
3. Update documentation for API changes
4. Ensure rate limiting compliance for new LLM integrations

• Support for additional programming languages
• Custom critic model configuration
• Advanced code analysis and quality metrics
• Integration with popular development tools

This project is licensed under the MIT License. See the LICENSE file for details.

For questions, issues, or contributions, please refer to the project's issue tracker and documentation (to be updated).