Headroom

The Context Optimization Layer for LLM Applications

Tool outputs are 70-95% redundant boilerplate. Headroom compresses that away.

The setup: 100 production log entries. One critical error buried at position 67.

[
  {"timestamp": "2024-12-15T00:00:00Z", "level": "INFO", "service": "api-gateway", "message": "Request processed successfully - latency=50ms", "request_id": "req-000000", "status_code": 200},
  {"timestamp": "2024-12-15T01:01:00Z", "level": "INFO", "service": "user-service", "message": "Request processed successfully - latency=51ms", "request_id": "req-000001", "status_code": 200},
  {"timestamp": "2024-12-15T02:02:00Z", "level": "INFO", "service": "inventory", "message": "Request processed successfully - latency=52ms", "request_id": "req-000002", "status_code": 200},
  // ... 64 more INFO entries ...
  {"timestamp": "2024-12-15T03:47:23Z", "level": "FATAL", "service": "payment-gateway", "message": "Connection pool exhausted", "error_code": "PG-5523", "resolution": "Increase max_connections to 500 in config/database.yml", "affected_transactions": 1847},
  // ... 32 more INFO entries ...
]

AFTER: Headroom compresses to 6 entries (1,155 chars):

[
  {"timestamp": "2024-12-15T00:00:00Z", "level": "INFO", "service": "api-gateway", ...},
  {"timestamp": "2024-12-15T01:01:00Z", "level": "INFO", "service": "user-service", ...},
  {"timestamp": "2024-12-15T02:02:00Z", "level": "INFO", "service": "inventory", ...},
  {"timestamp": "2024-12-15T03:47:23Z", "level": "FATAL", "service": "payment-gateway", "error_code": "PG-5523", "resolution": "Increase max_connections to 500 in config/database.yml", "affected_transactions": 1847},
  {"timestamp": "2024-12-15T02:38:00Z", "level": "INFO", "service": "inventory", ...},
  {"timestamp": "2024-12-15T03:39:00Z", "level": "INFO", "service": "auth", ...}
]

What happened: First 3 items + the FATAL error + last 2 items. The critical error at position 67 was automatically preserved.

The question we asked Claude: "What caused the outage? What's the error code? What's the fix?"

Both responses: "payment-gateway service, error PG-5523, fix: Increase max_connections to 500, 1,847 transactions affected"

87.6% fewer tokens. Same answer.

Run it yourself: python examples/needle_in_haystack_test.py

Headroom's guarantee: compress without losing accuracy.

We validate against established open-source benchmarks. Full methodology and reproducible tests: Benchmarks Documentation

Why recall matters most: For LLM applications, capturing all relevant information is critical. 98.2% recall means nearly all content is preserved — LLMs can answer questions accurately from compressed context.

# Install with benchmark dependencies
pip install "headroom-ai[evals,html]" datasets

# Run HTML extraction benchmark (no API key needed)
pytest tests/test_evals/test_html_oss_benchmarks.py::TestExtractionBenchmark -v -s

# Run QA accuracy tests (requires OPENAI_API_KEY)
pytest tests/test_evals/test_html_oss_benchmarks.py::TestQAAccuracyPreservation -v -s

The setup: An Agno agent with 4 tools (GitHub Issues, ArXiv Papers, Code Search, Database Logs) investigating a memory leak. Total tool output: 62,323 chars (~15,580 tokens).

from agno.agent import Agent
from agno.models.anthropic import Claude
from headroom.integrations.agno import HeadroomAgnoModel

# Wrap your model - that's it!
base_model = Claude(id="claude-sonnet-4-20250514")
model = HeadroomAgnoModel(wrapped_model=base_model)

agent = Agent(model=model, tools=[search_github, search_arxiv, search_code, query_db])
response = agent.run("Investigate the memory leak and recommend a fix")

Results with Claude Sonnet:

76.3% fewer tokens. Same comprehensive answer.

Both found: Issue #42 (memory leak), the cleanup_worker() fix, OutOfMemoryError logs (7.8GB/8GB, 847 threads), and relevant research papers.

Run it yourself: python examples/multi_tool_agent_test.py

Headroom optimizes LLM context before it hits the provider — without changing your agent logic or tools.

flowchart LR
  User["Your App"]
  Entry["Headroom"]
  Transform["Context<br/>Optimization"]
  LLM["LLM Provider"]
  Response["Response"]

  User --> Entry --> Transform --> LLM --> Response

flowchart TB

subgraph Pipeline["Transform Pipeline"]
  CA["Cache Aligner<br/><i>Stabilizes dynamic tokens</i>"]
  SC["Smart Crusher<br/><i>Removes redundant tool output</i>"]
  CM["Intelligent Context<br/><i>Score-based token fitting</i>"]
  CA --> SC --> CM
end

subgraph CCR["CCR: Compress-Cache-Retrieve"]
  Store[("Compressed<br/>Store")]
  Tool["Retrieve Tool"]
  Tool <--> Store
end

LLM["LLM Provider"]

CM --> LLM
SC -. "Stores originals" .-> Store
LLM -. "Requests full context<br/>if needed" .-> Tool

Headroom never throws data away. It compresses aggressively and retrieves precisely.

1. Headroom intercepts context — Tool outputs, logs, search results, and intermediate agent steps.

2. Dynamic content is stabilized — Timestamps, UUIDs, request IDs are normalized so prompts cache cleanly.

3. Low-signal content is removed — Repetitive or redundant data is crushed, not truncated.

4. Original data is preserved — Full content is stored separately and retrieved only if the LLM asks.

5. Provider caches finally work — Headroom aligns prompts so OpenAI, Anthropic, and Google caches actually hit.

For deep technical details, see Architecture Documentation.

• Zero code changes - works as a transparent proxy
• 47-92% savings - depends on your workload (tool-heavy = more savings)
• Image compression - 40-90% reduction via trained ML router (OpenAI, Anthropic, Google)
• Reversible compression - LLM retrieves original data via CCR
• Content-aware - code, logs, JSON, images each handled optimally
• Provider caching - automatic prefix optimization for cache hits
• Framework native - LangChain, Agno, MCP, agents supported

pip install "headroom-ai[all]"  # Recommended for best performance
headroom proxy --port 8787

Note: First startup downloads ML models (~500MB) for optimal compression. This is a one-time download.

Dashboard: Open http://localhost:8787/dashboard to see real-time stats, token savings, and request history.

Point your tools at the proxy:

# Claude Code
ANTHROPIC_BASE_URL=http://localhost:8787 claude

# Any OpenAI-compatible client
OPENAI_BASE_URL=http://localhost:8787/v1 cursor

Enable Persistent Memory - Claude remembers across conversations:

headroom proxy --memory

Memory auto-detects your provider (Anthropic, OpenAI, Gemini) and uses the appropriate format:

• Anthropic: Uses native memory tool (memory_20250818) - works with Claude Code subscriptions
• OpenAI/Gemini/Others: Uses function calling format
• All providers share the same semantic vector store for search

Set x-headroom-user-id header for per-user memory isolation (defaults to 'default').

Claude Code Subscription Users - Use MCP for CCR (Compress-Cache-Retrieve):

If you use Claude Code with a subscription (not API key), you need MCP to enable the headroom_retrieve tool:

# One-time setup
pip install "headroom-ai[mcp]"
headroom mcp install

# Every time you code
headroom proxy          # Terminal 1
claude                  # Terminal 2 - now has headroom_retrieve!

What this does:

• Configures Claude Code to use Headroom's MCP server (~/.claude/mcp.json)
• When the proxy compresses large tool outputs, Claude sees markers like [47 items compressed... hash=abc123]
• Claude can call headroom_retrieve to get the full original content when needed

Check your setup:

headroom mcp status

Using AWS Bedrock, Google Vertex, or Azure? Route through Headroom:

# AWS Bedrock - Terminal 1: Start proxy
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="us-east-1"
headroom proxy --backend bedrock --region us-east-1

# AWS Bedrock - Terminal 2: Run Claude Code
export ANTHROPIC_API_KEY="sk-ant-dummy"  # Any value works! Headroom ignores it.
export ANTHROPIC_BASE_URL="http://localhost:8787"
# IMPORTANT: Do NOT set CLAUDE_CODE_USE_BEDROCK=1 (Headroom handles Bedrock routing)
claude

{
  "claudeCode.environmentVariables": [
    { "name": "ANTHROPIC_API_KEY", "value": "sk-ant-dummy" },
    { "name": "ANTHROPIC_BASE_URL", "value": "http://localhost:8787" },
    { "name": "AWS_ACCESS_KEY_ID", "value": "AKIA..." },
    { "name": "AWS_SECRET_ACCESS_KEY", "value": "..." },
    { "name": "AWS_REGION", "value": "us-east-1" }
  ]
}

Using OpenRouter? Access 400+ models through a single API:

# OpenRouter - Terminal 1: Start proxy
export OPENROUTER_API_KEY="sk-or-v1-..."
headroom proxy --backend openrouter

# OpenRouter - Terminal 2: Run your client
export ANTHROPIC_API_KEY="sk-ant-dummy"  # Any value works! Headroom ignores it.
export ANTHROPIC_BASE_URL="http://localhost:8787"
# Use OpenRouter model names in your requests:
# - anthropic/claude-3.5-sonnet
# - openai/gpt-4o
# - google/gemini-pro
# - meta-llama/llama-3-70b-instruct
# See all models: https://openrouter.ai/models

# Google Vertex AI
headroom proxy --backend vertex_ai --region us-central1

# Azure OpenAI
headroom proxy --backend azure --region eastus

pip install "headroom-ai[langchain]"

from langchain_openai import ChatOpenAI
from headroom.integrations import HeadroomChatModel

# Wrap your model - that's it!
llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))

# Use exactly like before
response = llm.invoke("Hello!")

See the full LangChain Integration Guide for memory, retrievers, agents, and more.

pip install "headroom-ai[agno]"

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from headroom.integrations.agno import HeadroomAgnoModel

# Wrap your model - that's it!
model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
agent = Agent(model=model)

# Use exactly like before
response = agent.run("Hello!")

# Check savings
print(f"Tokens saved: {model.total_tokens_saved}")

See the full Agno Integration Guide for hooks, multi-provider support, and more.

Skeptical? Good. We built a comprehensive evaluation framework to prove compression preserves accuracy.

# Install evals
pip install "headroom-ai[evals]"

# Quick sanity check (5 samples)
python -m headroom.evals quick

# Run on real datasets
python -m headroom.evals benchmark --dataset hotpotqa -n 100

Original Context ───► LLM ───► Response A
                                   │
Compressed Context ─► LLM ───► Response B
                                   │
                    Compare A vs B │
                    ─────────────────
                    F1 Score: 0.95
                    Semantic Similarity: 0.97
                    Ground Truth Match: ✓
                    ─────────────────
                    PASS: Accuracy preserved

# GitHub Actions
- name: Run Compression Evals
  run: python -m headroom.evals quick -n 20
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

Exit code 0 if accuracy ≥ 90%, 1 otherwise.

See the full Evals Documentation for datasets, metrics, and programmatic API.

These numbers are from actual API calls, not estimates:

Overhead: ~1-5ms compression latency

When savings are highest: Tool-heavy workloads (search, logs, database queries) When savings are lowest: Conversation-heavy workloads with minimal tool use

New models auto-supported via naming pattern detection.

• Never removes human content - user/assistant messages preserved
• Never breaks tool ordering - tool calls and responses stay paired
• Parse failures are no-ops - malformed content passes through unchanged
• Compression is reversible - LLM retrieves original data via CCR

# Recommended: Install everything for best compression performance
pip install "headroom-ai[all]"

# Or install specific components
pip install headroom-ai              # SDK only
pip install "headroom-ai[proxy]"     # Proxy server
pip install "headroom-ai[mcp]"       # MCP server for Claude Code subscriptions
pip install "headroom-ai[langchain]" # LangChain integration
pip install "headroom-ai[agno]"      # Agno agent framework
pip install "headroom-ai[evals]"     # Evaluation framework
pip install "headroom-ai[code]"      # AST-based code compression
pip install "headroom-ai[llmlingua]" # ML-based compression

Requirements: Python 3.10+

First-time startup: Headroom downloads ML models (~500MB) on first run for optimal compression. This is cached locally and only happens once.

Add your project here! Open a PR or start a discussion.

git clone https://github.com/chopratejas/headroom.git
cd headroom
pip install -e ".[dev]"
pytest

See CONTRIBUTING.md for details.

Apache License 2.0 - see LICENSE.

_{Built for the AI developer community}