Prism is a lightweight, dual-layer intent routing middleware designed for LLM Agents. It acts as a "Triage Nurse" before your expensive "Doctor" (LLM), reducing latency by 90% and token costs by 80% for high-frequency queries.
中文文档 | Architecture | Quick Start | Dify Integration
Building a production-ready Agent often faces the "Cost vs. Speed" dilemma:
- Directly using GPT-4/DeepSeek: Too slow (1-3s) and expensive for simple queries like "Check attendance" or "Hi".
- Keyword Matching: Too dumb to understand variations like "Show me the logs".
Prism solves this by using a Cascade Architecture:
- ⚡ Level 1 (Fast Path): Uses local Embeddings (BGE-Small) to intercept simple intents in <20ms.
- 🧠 Level 2 (Slow Path): Routes complex/ambiguous queries to LLMs (DeepSeek V3) for reasoning and Parameter Extraction.
Prism employs a "Funnel" strategy to process user queries:
graph TD
User([👤 User Input]) --> BGE
subgraph "⚡ Level 1: Fast Path"
direction TB
BGE[🔍 BGE Embedding]
Gate{Confidence > 0.85?}
BGE --> Gate
end
subgraph "🧠 Level 2: Smart Path"
direction TB
LLM[🤖 DeepSeek V3]
Tools[🛠️ Function Calling]
LLM --> Tools
end
Gate -->|Yes| Action_Simple[🚀 Direct Execution]
Gate -->|No| LLM
Tools --> Action_Complex[🧩 Complex Execution]
- Hybrid Routing: Combines the speed of Vector Search with the intelligence of LLMs.
- Zero-Cost Interception: 80% of daily queries (FAQ, Navigations) are handled locally without consuming LLM tokens.
- Smart Parameter Extraction: Automatically extracts dates, metrics, and entities (e.g., "Last week's DAU" ->
date="2023-10-20", metric="DAU"). - Dify Ready: Built-in FastAPI wrapper, ready to be added as a Custom Tool in Dify.
- Private Deployment: Supports Docker deployment; data stays within your VPC.
| Scenario | Traditional Agent (LLM Only) | Prism (Cascade) | Improvement |
|---|---|---|---|
| Simple Query |
(e.g., "Search Docs") | ~1.2s | ~0.015s | 80x Faster 🚀 |
| Complex Query
(e.g., "Show me last week's sales") | ~2.5s | ~2.6s | Same Accuracy |
| Cost (1M reqs) | $500 | **$80** | 84% Cheaper 💰 |
git clone https://github.com/the1sky/prism-router.git
cd prism-router
# Create and activate conda environment
conda create -n prism python=3.13 -y
conda activate prism
# Install dependencies
pip install -r requirements.txtCreate a .env file or set environment variables:
export DEEPSEEK_API_KEY="sk-your-api-key"# Activate environment (if not already activated)
conda activate prism
# Start the server
python main.py
# Server running at http://0.0.0.0:8000# Test Fast Path (No Token Cost)
curl -X POST "http://localhost:8000/v1/route" \
-H "Content-Type: application/json" \
-d '{"query": "Please check the attendance policy"}'
# Test Slow Path (DeepSeek with Args)
curl -X POST "http://localhost:8000/v1/route" \
-H "Content-Type: application/json" \
-d '{"query": "Help me restart the test server"}'
Prism is designed to work seamlessly with Dify.
- Deploy Prism: Run this project on a server accessible by Dify (e.g., expose via Ngrok or deploy to Render/Vercel).
- Add Custom Tool: In Dify, go to
Tools->Custom Tool->Create. - Import Schema: Use the
openapi.jsonprovided athttp://your-server/openapi.json. - Use in Workflow:
- Add the Prism Tool node at the beginning of your Chatflow.
- Use
Conditional Branching(If/Else) based on the outputintent.
Pro Tip: See docs/dify_integration.md for a step-by-step screenshot guide.
prism/
├── main.py # Application entry point
├── api/
│ ├── __init__.py
│ └── server.py # FastAPI server and route definitions
├── config/
│ ├── kb_data.json # Knowledge base data (intent examples)
│ └── tools_schema.json # Function calling schemas for LLM
├── core/
│ ├── __init__.py
│ └── router.py # Core routing logic (Fast/Slow path)
├── requirements.txt
├── Dockerfile
├── README.md # English documentation
└── README_CN.md # Chinese documentation
We welcome contributions! Please see CONTRIBUTING.md for details.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE for more information.