Micro-RAG is a lightweight, framework-agnostic memory management system for LLM agents. It decouples memory formation, storage, retrieval, and context-window management into independent components that integrate with any LLM harness or orchestrator.

Micro-RAG uses a two-layer memory architecture: raw conversation text is stored verbatim as Layer 1 chunks, and structured beliefs are derived from those chunks as Layer 2. This means ingestion is instant (zero LLM calls — just chunking and hashing), and no information is ever lost to extraction judgment. Beliefs are formed asynchronously by the consolidator and serve as a derived index over the permanent record.

At retrieval time, the injector searches across both layers, assembles the most relevant facts within a fixed token budget, and injects them into the prompt — no system instructions, no coaching, just context.

flowchart LR
    subgraph INGESTION ["Ingestion (zero LLM calls)"]
        A["Conversation\nTurns"] --> B["MemoryIngestor"]
        B --> C["Layer 1\n~100-token verbatim chunks\nwith timestamps & metadata"]
    end

    subgraph CONSOLIDATION ["Consolidation (async / batch)"]
        C --> D["BeliefConsolidator"]
        D --> E["Layer 2\nStructured beliefs\n(premises, preferences,\npropositions, skills)"]
        D -->|"tag-based\nrollups"| E
        D -->|"HDBSCAN\nclustering"| E
        D -->|"session\nmerges"| E
    end

    subgraph RETRIEVAL ["Retrieval (per turn)"]
        F["User Input"] --> G["PreGenerativeInjector"]
        C --> G
        E --> G
        G -->|"multi-head search\n+ token budget"| H["Injected\nContext\n~400-700 tokens"]
    end

    H --> I["LLM\n(no system prompt)"]

Clone the repository and install it locally in editable mode:

git clone https://github.com/munch2u-a11y/mRAG.git
cd mRAG
pip install -e .

Or install with specific vector database extras:

pip install -e .[chromadb]  # For local ChromaDB support
pip install -e .[pinecone]  # For cloud Pinecone support

You can also install directly from GitHub:

pip install git+https://github.com/munch2u-a11y/mRAG.git

from mrag import BeliefStore, create_vector_store, PreGenerativeInjector

# 1. Setup Data Store
belief_store = BeliefStore(data_dir="./mrag_data")

# 2. Select Vector Database Backend (e.g. 'chromadb', 'pinecone', or 'dummy' for sandboxed testing)
vector_store = create_vector_store("chromadb")

# 3. Setup Pre-generative Injector Pipeline
injector = PreGenerativeInjector(belief_store=belief_store, vector_store=vector_store)

user_input = "Hello, what do you know about me?"

# 4. Inject Beliefs (Run this BEFORE calling your LLM)
injected_context = injector.inject(trigger_text=user_input)
print(injected_context)
# --- Injected Context ---
# • User prefers Python [0.95]
# ------------------------

See the examples/ directory for a full simulation of LangGraph integration with compression and belief consolidation.

All numbers below were produced by the scripts in tests/ on the code in this repository; the full reports (JSON + Markdown, with methodology notes) live in benchmarks/. Retrieval always uses real embeddings (Chroma's default all-MiniLM-L6-v2); LLM-dependent steps in the token-efficiency protocol use deterministic extractive mocks so every run is reproducible without API keys. Prompt and injection token counts use a tokenizer-backed counter when available (tiktoken by default; override with MRAG_TOKENIZER_MODEL or MRAG_TOKENIZER_ENCODING) instead of a chars-per-token heuristic.

A simulated 200-turn agent session (facts introduced every 5 turns, rolling compression + belief consolidation + injection) versus the same session with naive full-history prompting:

• Cumulative prompt tokens over 200 turns: 65.4% fewer (631k vs 1.82M) — and the saving keeps growing with session length.
• Long-range fact recall via compressed context + injection: 0.86 overall, 0.88 for facts more than 100 turns old (full-history baseline is 1.0 by construction, at full token cost).
• Skill retrieval: 20/20 natural-language task queries surfaced the correct imported tool in the top-5 injection.

Reproduce with:

python tests/run_token_efficiency_benchmark.py
python tests/run_niah_benchmark.py
python tests/run_associative_benchmark.py
python tests/run_100k_benchmark.py

No vendor comparison numbers are published here on purpose: other systems' published results are not normalized to these protocols. The scripts are self-contained, so you can run the same protocol against any system you want to compare with.

The benchmarks above use deterministic extractive mocks for the LLM-dependent steps. The results below run the full pipeline end to end against real long- form conversation datasets, with gemini-2.5-flash doing fact extraction, answer generation, and grading. Full per-question logs are in benchmarks/.

Grading methodology: LoCoMo uses strict PASS/MISS grading (no lenient semantic matching). ConvoMem uses LLM-judge grading via the memorybench harness. All miss audits are documented in the per-question markdown reports in benchmarks/.

No answering prompt: The answering model receives no system prompt, no role instructions, and no coaching (e.g., "answer only from context" or "say I don't know if unsure"). It gets the retrieved context and the question — nothing else. Strict context → question → answer.

Where misses come from: Across all runs, remaining misses fall into two categories: (1) retrieval gaps — the relevant belief was never surfaced (e.g., a fact was consolidated away during ingestion, or a knowledge-update superseded the wrong version), and (2) question design issues — questions requiring implicit inference from context that was never stated in the conversation. Model-side "I don't know" errors were eliminated by removing coaching instructions from the answering prompt.

ConvoMem category breakdown (run 2, 24Q):

Token efficiency: The PreGenerativeInjector caps injected context at a fixed token budget regardless of how large the underlying belief store grows. A LongMemEval context block with ~155k raw conversation tokens is reduced to ~496 injected tokens at retrieval time — a >300x reduction in latency-critical prompt tokens while maintaining high retrieval accuracy.

Reproduce with:

python tests/run_locomo_benchmark.py

Requires a GEMINI_API_KEY (see the script header for how credentials are loaded). ConvoMem runs use the external memorybench harness — see benchmarks/ for the full JSON reports.

python -m unittest discover tests

The suite covers the belief store (decay, pruning, cache/index consistency), the injector (retrieval, anti-repetition fallback, index sync), the context compressor, the vector store factory, and all skill/soul adapters.

If you already have existing agents with defined tools/skills, you can import them directly into Micro-RAG's BeliefStore as skills using the mrag.adapters module:

from mrag import adapters, BeliefStore

belief_store = BeliefStore(data_dir="./mrag_data")

openai_tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather info."
        }
    }
]
adapters.import_openai_tools(openai_tools, belief_store)

mcp_tools = {
    "tools": [
        {
            "name": "calculate_tax",
            "description": "Calculate tax rate based on zip code."
        }
    ]
}
adapters.import_mcp_tools(mcp_tools, belief_store)

You can import a directory of skill files:

adapters.import_from_directory("./my_skills_dir", belief_store)

You can also pass a custom_parser to extract name/description mapping from any proprietary schema structure.