Recursive Language Model (RLM) Framework

A Python framework for handling long contexts using recursive decomposition and sub-calls to language models. Tested with contexts up to 128k tokens.

Recursive Language Models (RLMs) enable LLMs to handle long contexts by recursively decomposing them:

• Breaking down complex tasks into smaller, manageable sub-problems
• Invoking sub-LLMs recursively on targeted snippets
• Aggregating results back up through a tree-like call structure
• Operating in a REPL environment where context is accessed programmatically, not loaded into neural context

This implementation supports multiple LLM providers including OpenAI (GPT-4o, GPT-5), xAI (Grok), and Anthropic (Claude), with an extensible provider architecture.

• Handle long contexts: Tested with contexts up to 128k tokens via recursive decomposition
• Recursive sub-calls: Automatic decomposition with nested LLM invocations
• REPL-based execution: Generate and execute Python code in a persistent environment
• Model flexibility: Use different models for root and sub-calls (e.g., GPT-4o + GPT-4o-mini, Sonnet + Haiku)
• Direct mode: Automatic fast-path for contexts that fit within the model's context window — single LLM call, no REPL overhead
• Comprehensive testing: Built-in test suite for needle-in-haystack, reasoning, and summarization tasks

• Comprehensive tracking: Tokens, costs, recursion depth, call counts
• Real-time budget controls: Set max cost/token limits
• Detailed analytics: Export metrics to JSON for analysis
• Cache statistics: Monitor cache hit rates and efficiency

• LRU caching: Avoid redundant sub-calls (configurable size & TTL)
• Model tiering: Use cheaper models for sub-calls
• Smart chunking: Token-aware and paragraph-preserving strategies
• Parallel processing: Map-reduce patterns with optional parallelization

• Sandboxed execution: Restricted globals prevent dangerous operations
• Code safety checks: Block forbidden patterns (file I/O, network, subprocess)
• Resource limits: Execution timeouts and output size constraints

• Text processing: Smart chunking, token-based splitting, truncation
• Search & filtering: Regex, keyword search, section extraction
• Aggregation: Multiple strategies (sum, join, count, dict)
• Verification: Answer validation, consensus checking
• Recursion patterns: Recursive split, map-reduce

CLI (easiest):

uv run python main.py run --task "Your question" --context-file data.txt

Python API:

from rlm import RecursiveLanguageModel
rlm = RecursiveLanguageModel(api_key="...", model="grok-4-1-fast-reasoning")
result = rlm.run(task="Your question", context="Your long text...")

Anthropic Claude (three-tier):

from rlm import RecursiveLanguageModel
rlm = RecursiveLanguageModel(
    api_key="...",
    model="claude-sonnet-4-5-20250929",
    simple_model="claude-haiku-4-5-20251001",
    provider="anthropic",
)
result = rlm.run(task="Your question", context="Your long text...")

See CLI Usage for detailed examples with different models and options.

rlm-adk/
├── rlm/                          # Main package
│   ├── core.py                   # RecursiveLanguageModel implementation
│   ├── providers.py              # Multi-provider support (OpenAI, xAI, Anthropic)
│   ├── metrics.py                # Token and cost tracking
│   ├── helpers.py                # Advanced utility functions
│   ├── security.py               # Sandboxed execution
│   ├── cache.py                  # LRU caching system
│   └── __init__.py               # Package exports
├── examples/                     # Usage examples
│   ├── quickstart_anthropic.py   # Minimal Anthropic Claude example
│   ├── quickstart_grok.py        # Minimal Grok example
│   ├── quickstart_gpt5.py        # Minimal GPT-5 example
│   ├── basic_usage.py            # Needle-in-haystack pattern
│   ├── classification_example.py # Classification and aggregation
│   ├── verification_example.py   # Verification pattern
│   ├── long_output_example.py    # Long output generation
│   ├── advanced_patterns.py      # Map-reduce pattern
│   ├── grok_basic_example.py     # Grok integration
│   ├── grok_reasoning_example.py # Grok reasoning metrics
│   └── multi_provider_example.py # Cross-provider comparison
├── tests/                        # Comprehensive test suite
│   ├── test_anthropic_provider.py       # Anthropic provider unit tests
│   ├── test_attention_paper_anthropic.py # PDF paper analysis (agentic mode)
│   ├── test_huberman_demo_anthropic.py  # Anthropic integration demo
│   ├── test_huberman_demo.py            # Grok integration demo
│   ├── test_rlm_comprehensive.py        # Long-context integration tests
│   ├── test_retry_streaming_sandbox.py  # Retry, streaming, and sandbox tests
│   ├── test_helpers.py                  # Helper function tests
│   ├── test_cache.py                    # Caching tests
│   ├── test_metrics.py                  # Metrics tests
│   ├── test_mock.py                     # Mock/stub tests
│   ├── test_data_generator.py           # Test data generation
│   └── Attention_is_all_you_need.pdf    # Test PDF (Transformer paper)
├── main.py                       # CLI entry point
├── Makefile                      # Development commands (uv-based)
├── pyproject.toml                # Package metadata
├── requirements.txt              # Python dependencies
├── .env.example                  # Environment configuration template
├── PROJECT_STRUCTURE.md          # Detailed structure documentation
└── README.md                     # This file

• Python 3.8+
• uv - Fast Python package installer (recommended)
• At least one API key: OpenAI, xAI, or Anthropic

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Sync dependencies (recommended - uses uv.lock for reproducibility)
uv sync

# Or install in editable mode
uv pip install -e .

# Install dependencies and package in editable mode
pip install -e .

# Or install from requirements.txt
pip install -r requirements.txt
pip install -e .

Create a .env file or set environment variables:

# For Anthropic Claude models
export ANTHROPIC_API_KEY="your-anthropic-key"

# For OpenAI models
export OPENAI_API_KEY="your-openai-key"

# For xAI Grok models
export XAI_API_KEY="your-xai-key"

from rlm import RecursiveLanguageModel

# Three-tier model setup: orchestrator + smart + fast
rlm = RecursiveLanguageModel(
    api_key="your-anthropic-key",
    model="claude-sonnet-4-5-20250929",          # Orchestrator
    simple_model="claude-haiku-4-5-20251001",    # Fast sub-tasks
    provider="anthropic",
    enable_cache=True,
    max_cost=5.0
)

from rlm import RecursiveLanguageModel

rlm = RecursiveLanguageModel(
    api_key="your-api-key",
    model="gpt-4o",           # Root model
    sub_model="gpt-4o-mini",  # Cheaper model for sub-calls
    enable_cache=True,        # Cache sub-call results
    max_cost=1.0              # Budget limit: $1
)

# Create a long context
context = "..." # Your long document

# Define task
task = "Find the magic number mentioned in the context."

# Run RLM
result = rlm.run(task=task, context=context, verbose=True)
print(f"Result: {result}")

# View metrics
rlm.print_metrics()

The main.py CLI provides an easy way to run RLM tasks without writing Python code.

# Show available models and configuration
uv run python main.py info

# Run a task with a text file
uv run python main.py run --task "Your question" --context-file document.txt

# Run a task with direct text
uv run python main.py run --task "Your question" --context "Your text here"

# Run tests
uv run python main.py test --quick

# Basic usage with Grok (uses grok-4-1-fast-reasoning by default)
uv run python main.py run \
  --task "Find the secret code mentioned in the document" \
  --context-file data.txt

# Specify a different Grok model
uv run python main.py run \
  --task "Summarize the key findings" \
  --context-file research_paper.txt \
  --model grok-4 \
  --provider xai

# With custom settings
uv run python main.py run \
  --task "Extract all dates and events" \
  --context-file historical_records.txt \
  --model grok-4-1-fast-reasoning \
  --max-cost 2.0 \
  --max-iterations 30 \
  --output results.json

# Using Sonnet 4.5 (recommended)
uv run python main.py run \
  --task "Summarize the key findings" \
  --context-file research_paper.txt \
  --model claude-sonnet-4-5-20250929 \
  --provider anthropic

# With three-tier setup (orchestrator + fast sub-tasks)
uv run python main.py run \
  --task "Extract all dates and events" \
  --context-file historical_records.txt \
  --model claude-sonnet-4-5-20250929 \
  --provider anthropic \
  --max-cost 5.0

# Using GPT-4o
uv run python main.py run \
  --task "Analyze sentiment across all reviews" \
  --context-file reviews.txt \
  --model gpt-4o \
  --provider openai

# Using GPT-5 mini (fast and cheap)
uv run python main.py run \
  --task "Find references to project Apollo" \
  --context-file transcripts.txt \
  --model gpt-5-mini \
  --provider openai \
  --max-cost 1.0

# Short context via command line
uv run python main.py run \
  --task "What is the main topic?" \
  --context "Artificial intelligence is transforming how we process information..."

# Multi-line text (using quotes)
uv run python main.py run \
  --task "Count how many people are mentioned" \
  --context "John works in marketing. Sarah leads engineering. Bob manages operations."

# Save results to JSON file
uv run python main.py run \
  --task "Summarize findings" \
  --context-file report.txt \
  --output summary.json

# Quiet mode (less verbose output)
uv run python main.py run \
  --task "Find the error code" \
  --context-file logs.txt \
  --quiet

# Disable caching
uv run python main.py run \
  --task "Process each item uniquely" \
  --context-file items.txt \
  --no-cache

# Set custom API key (instead of env variable)
uv run python main.py run \
  --task "Your task" \
  --context-file data.txt \
  --api-key "your-api-key-here"

• claude-opus-4-6 - Claude Opus 4.6 (200k context, most capable)
• claude-sonnet-4-5-20250929 - Claude Sonnet 4.5 (200k context, recommended orchestrator)
• claude-sonnet-4-5-20250514 - Claude Sonnet 4.5 (earlier release)
• claude-haiku-4-5-20251001 - Claude Haiku 4.5 (200k context, fast/cheap sub-tasks)

• grok-4 - Standard Grok 4 model (128k context)
• grok-4.20-experimental-beta-0304-reasoning - Experimental Grok 4.20 reasoning beta
• grok-4-1-fast-reasoning - Fast reasoning variant (recommended, cheaper)
• grok-4-1-fast-non-reasoning - Non-reasoning variant
• grok-4-fast-reasoning - Fast reasoning
• grok-4-fast-non-reasoning - Fast non-reasoning
• grok-beta - Beta version

• gpt-5-mini - GPT-5 mini (fast and cheap, recommended for testing)
• gpt-5-nano - GPT-5 nano (ultra-fast)
• gpt-4.1 - GPT-4.1
• gpt-4o - GPT-4 optimized

Run Command Options:

--task, -t          Task description/question (required)
--context, -c       Direct text input
--context-file, -f  Path to text file
--provider, -p      LLM provider: 'anthropic', 'xai', or 'openai' (default: xai)
--model, -m         Model name (default: auto-detected)
--api-key           API key (or use environment variable)
--max-cost          Maximum cost in USD (default: 5.0)
--max-iterations    Maximum iterations (default: 50)
--no-cache          Disable caching
--output, -o        Save results to JSON file
--quiet, -q         Reduce output verbosity

# Quick sanity check
uv run python main.py test --quick --model grok-4-1-fast-reasoning

# Run comprehensive test suite
uv run python main.py test \
  --suite comprehensive \
  --model grok-4-1-fast-reasoning \
  --no-256k

# Run unit tests
uv run python main.py test --suite unit

# Save test results
uv run python main.py test \
  --suite comprehensive \
  --model grok-4-1-fast-reasoning \
  --output test_results.json \
  --max-cost 2.0

# Generate all test types
uv run python main.py generate --type all

# Generate needle-in-haystack test
uv run python main.py generate \
  --type needle \
  --tokens 256 \
  --position middle \
  --output-dir tests/data

# Generate multi-needle test
uv run python main.py generate \
  --type multi \
  --num-needles 5

# Generate reasoning test
uv run python main.py generate \
  --type reasoning \
  --complexity medium

Example 1: Process a large log file

uv run python main.py run \
  --task "Find all ERROR entries and summarize the most common issues" \
  --context-file application.log \
  --model grok-4-1-fast-reasoning \
  --max-cost 3.0 \
  --output error_summary.json

Example 2: Analyze research papers

uv run python main.py run \
  --task "Extract methodology, key findings, and limitations" \
  --context-file paper.txt \
  --model gpt-4o \
  --provider openai \
  --max-iterations 40

Example 3: Process customer feedback

uv run python main.py run \
  --task "Categorize feedback by topic and sentiment, then count frequencies" \
  --context-file customer_reviews.txt \
  --model grok-4-1-fast-reasoning \
  --max-cost 2.0

Example 4: Code review

uv run python main.py run \
  --task "Find potential security issues and performance bottlenecks" \
  --context-file codebase_dump.txt \
  --model gpt-4o \
  --provider openai

The RLM operates in one of two modes depending on context size:

When the context fits within half the model's context window, the RLM uses a single LLM call — no REPL, no code generation. This is fast and cheap.

For contexts that exceed the threshold, the RLM enters an agentic loop:

1. Initialization: Context is loaded into REPL as a variable (not into neural context)
2. Code Generation: Root LLM generates Python code to process the task
3. Execution: Code runs in REPL, can inspect/slice context programmatically
4. Sub-calls: Code invokes llm_query() (smart model) or llm_query_fast() (cheap model) on targeted snippets
5. Recursion: Sub-calls can make their own sub-calls (tree structure)
6. Aggregation: Results bubble up and are combined
7. Iteration: Process repeats until FINAL() is called

Root LLM → Generate Code → Execute in REPL
              ↓
          llm_query(snippet1)      → Sub-Model → Result1
          llm_query_fast(snippet2) → Fast-Model → Result2
              ↓
          Aggregate Results → Next Iteration
              ↓
          FINAL(answer)

The framework supports assigning different models to each role:

Main RLM implementation with:

• REPL management
• LLM API calls
• Iteration loop
• Code execution
• Final answer handling

Tracks:

• Token usage (prompt + completion)
• Costs by model
• Recursion depth
• Call counts
• Execution time
• Per-call details

LRU cache for sub-calls:

• Hash-based lookup (prompt + model)
• Configurable size and TTL
• Hit/miss tracking
• Export capabilities

Sandboxing for code execution:

• Restricted builtins (no eval, open, __import__, etc.)
• Blocked dangerous modules (os, sys, subprocess, etc.)
• Code pattern checking
• Execution monitoring

Advanced utilities:

• TextProcessor: Chunking, token-based splitting
• SearchHelper: Regex, keyword filtering, section extraction
• AggregationHelper: Multiple aggregation strategies
• VerificationHelper: Answer validation, consensus
• RecursionHelper: Recursive patterns, map-reduce

The RLM naturally develops these emergent behaviors:

Use case: Needle-in-haystack tasks

task = "Find the magic number in the context."

# RLM will:
# 1. Use regex_search() to find candidates
# 2. Use llm_query() on each to verify
# 3. Return the verified answer

Example: examples/basic_usage.py

Use case: Long lists, classification tasks

task = "Count how many items are fruits vs vegetables."

# RLM will:
# 1. Split context into lines/chunks
# 2. Call llm_query() on each chunk for classification
# 3. Use count_frequencies() to aggregate

Example: examples/classification_example.py

Use case: Critical facts extraction

task = "Extract key facts and verify them."

# RLM will:
# 1. Extract facts from chunks
# 2. Use verify_answer() to cross-check
# 3. Return only verified facts

Example: examples/verification_example.py

Use case: Comprehensive summaries, reports

task = "Generate detailed summary of all topics."

# RLM will:
# 1. Use find_sections() to identify topics
# 2. Call llm_query() for each topic's summary
# 3. Aggregate into final document

Example: examples/long_output_example.py

Use case: Batch processing, sentiment analysis

task = "Analyze sentiment of all reviews."

# RLM will:
# 1. Use map_reduce() to process reviews
# 2. Map: llm_query() classifies each review
# 3. Reduce: Aggregate results

Example: examples/advanced_patterns.py

When generating code, the RLM has access to these helpers:

# Chunk text with overlap
chunks = chunk_text(text, chunk_size=2000, overlap=200, preserve_paragraphs=False)

# Token-based chunking (more accurate for LLM limits)
chunks = chunk_by_tokens(text, max_tokens=1000, overlap_tokens=100)

# Smart truncation at word boundaries
truncated = smart_truncate(text, max_length=100, suffix="...")

# Regex search with limits
matches = regex_search(pattern, text, max_matches=10, return_positions=False)

# Find markdown sections
sections = find_sections(text, section_pattern=r'^#+\s+(.+)$', include_content=True)

# Keyword filtering with context
snippets = keyword_filter(text, keywords=['important', 'critical'], context_chars=200)

# Aggregate results
result = aggregate_results(results, method='join', separator='\n', filter_empty=True)
# Methods: 'join', 'sum', 'count', 'list', 'dict'

# Count frequencies
freq = count_frequencies(['apple', 'banana', 'apple'])  # {'apple': 2, 'banana': 1}

# Merge dictionaries
merged = merge_dicts([dict1, dict2], merge_strategy='sum')
# Strategies: 'sum', 'last', 'first', 'list'

# Verify an answer
is_valid, explanation = verify_answer(answer, verification_prompt)

# Consensus check (multiple attempts)
consensus_answer, confidence = consensus_check(question, num_attempts=3)

# Recursive split until condition met
chunks = recursive_split(
    text,
    condition=lambda t: len(t) < 1000,  # Stop when small enough
    split_fn=lambda t: chunk_text(t, chunk_size=5000),
    max_depth=10
)

# Map-reduce pattern
result = map_reduce(
    items,
    map_fn=lambda item: llm_query(f"Process: {item}"),
    reduce_fn=lambda results: aggregate_results(results, method='join'),
    parallel=False
)

rlm = RecursiveLanguageModel(
    api_key="...",              # Default API key (used if provider-specific key not set)
    model="gpt-4o",             # Root/orchestrator model
    sub_model="gpt-4o-mini",    # Smart sub-call model (defaults to root model)
    simple_model=None,          # Fast sub-call model (defaults to sub_model)
    provider="openai",          # Provider: 'anthropic', 'openai', or 'xai'
    context_window=None,        # Override auto-detected context window
    enable_cache=True,          # Enable caching
    cache_size=1000,            # Max cached entries
    cache_ttl=3600,             # Cache TTL in seconds (None = no expiration)
    max_cost=None,              # Max total cost in USD (None = unlimited)
    max_tokens=None,            # Max total tokens (None = unlimited)
    enable_security=True,       # Enable sandboxing
    log_level="INFO",           # Logging level
    timeout=None,               # API request timeout in seconds
    sub_call_max_tokens=2048,   # Max tokens for sub-call responses
    # Provider-specific API keys (override api_key for cross-provider setups)
    anthropic_api_key=None,
    openai_api_key=None,
    xai_api_key=None,
)

result = rlm.run(
    task="...",                 # Task description
    context="...",              # Long input context
    max_iterations=50,          # Safety limit
    verbose=True                # Print progress
)

# Print summary
rlm.print_metrics()

# Get metrics dict
metrics = rlm.get_metrics_summary()
print(metrics['cost']['total_usd'])
print(metrics['tokens']['total'])
print(metrics['cache']['hit_rate_percent'])

# Export to JSON
rlm.export_metrics("metrics.json")

# Exported data includes:
# - Summary: duration, iterations, calls, tokens, cost, efficiency
# - Call history: per-call details, timestamps, recursion depth
# - Cache stats (if enabled)

• Use a powerful model for the orchestrator (Sonnet 4.5, GPT-4o)
• Use a cheap/fast model for simple sub-tasks (Haiku 4.5, GPT-4o-mini)
• Optionally use a mid-tier model for complex sub-tasks (sub_model)

rlm = RecursiveLanguageModel(
    ...,
    max_cost=5.0,        # Stop if cost exceeds $5
    max_tokens=1_000_000 # Stop if tokens exceed 1M
)

Essential for repeated sub-calls (e.g., classification of duplicate items):

rlm = RecursiveLanguageModel(
    ...,
    enable_cache=True,
    cache_size=1000,
    cache_ttl=3600  # 1 hour
)

More accurate than character-based:

# In generated code:
chunks = chunk_by_tokens(context, max_tokens=1000)

# In generated code:
answer = llm_query("Extract the fact")
is_valid, explanation = verify_answer(answer, "Cross-check this fact")
if is_valid:
    FINAL(answer)

Always check metrics after runs to optimize:

rlm.print_metrics()
# Look for:
# - High sub-call ratio (good for dense tasks)
# - Cache hit rate (should be high for repeated items)
# - Cost per call (optimize model choices)

1. Over-Recursion Risk: Some models may make excessive sub-calls, inflating costs

   • Mitigation: Set max_cost and max_tokens limits

2. Code Generation Dependency: Relies on LLM's coding ability

   • Mitigation: Use stronger root model (GPT-4o)

3. Execution Time: Many sub-calls can be slow

   • Mitigation: Use caching, cheaper sub-models, parallel processing

4. Security: Code execution has inherent risks

   • Mitigation: Sandboxing is enabled by default

For the easiest way to get started without writing code, see the CLI Usage section above. The CLI allows you to run tasks directly from the command line:

# Quick example with CLI
uv run python main.py run \
  --task "Find all mentions of 'quantum computing'" \
  --context-file research.txt \
  --model grok-4-1-fast-reasoning

For the simplest possible Python usage, start with the quickstart examples:

# Anthropic Claude quickstart (recommended)
uv run python examples/quickstart_anthropic.py

# Grok quickstart (minimal example)
uv run python examples/quickstart_grok.py

# GPT-5 quickstart (minimal example)
uv run python examples/quickstart_gpt5.py

Run the comprehensive examples to see different patterns:

# Basic needle-in-haystack
uv run python examples/basic_usage.py

# Classification and aggregation
uv run python examples/classification_example.py

# Verification pattern
uv run python examples/verification_example.py

# Long output generation
uv run python examples/long_output_example.py

# Advanced map-reduce
uv run python examples/advanced_patterns.py

# Grok-specific examples
uv run python examples/grok_basic_example.py
uv run python examples/grok_reasoning_example.py

# Multi-provider comparison
uv run python examples/multi_provider_example.py

You can also use the Makefile for convenience:

# Run all examples
make examples

# Run quickstart examples only
make quickstart

# Run unit tests
make test

# Run with coverage
make test-coverage

# Run comprehensive integration tests
uv run python tests/test_rlm_comprehensive.py

The RLM framework includes a comprehensive test suite to evaluate long-context capabilities across multiple dimensions.

Located in tests/test_rlm_comprehensive.py, the test suite includes:

• Tests retrieval of specific information buried in massive contexts
• Evaluates precision at different positions (start, middle, end)
• Use case: Finding specific facts in huge documents

• Tests extraction of multiple distributed facts
• Evaluates recall and completeness
• Use case: Extracting all key data points from long reports

• Tests multi-hop reasoning over distributed information
• Evaluates ability to connect facts across long contexts
• Complexity levels: Simple, medium, complex
• Use case: Answering questions requiring synthesis

• Tests contradictory information resolution
• Tests handling of repeated vs. unique information
• Tests numeric precision in calculations
• Use case: Robust processing of real-world messy data

• Tests ability to summarize 25k-90k token documents
• Evaluates key point coverage (70% threshold) and compression ratio (≤0.5)
• Generates realistic technical reports with marked key points
• Use case: Distilling long documents into concise summaries

uv run python tests/test_rlm_comprehensive.py --quick

# Needle-in-haystack (256k tokens)
uv run python tests/test_rlm_comprehensive.py --test needle --model grok-4-1-fast-reasoning --provider xai

# Multi-needle retrieval
uv run python tests/test_rlm_comprehensive.py --test multi --model grok-4-1-fast-reasoning --provider xai

# Reasoning test
uv run python tests/test_rlm_comprehensive.py --test reasoning --model grok-4-1-fast-reasoning --provider xai

# Summarization test (50k-90k tokens recommended)
uv run python tests/test_rlm_comprehensive.py --test summarization --model grok-4-1-fast-reasoning --provider xai

# Edge cases
uv run python tests/test_rlm_comprehensive.py --test edge --model grok-4-1-fast-reasoning --provider xai

# With all tests (expensive, ~$1-2)
uv run python tests/test_rlm_comprehensive.py --model grok-4-1-fast-reasoning --provider xai

# Skip expensive 256k test
uv run python tests/test_rlm_comprehensive.py --model grok-4-1-fast-reasoning --provider xai --no-256k

# Skip summarization test
uv run python tests/test_rlm_comprehensive.py --model grok-4-1-fast-reasoning --provider xai --no-summarization

# Minimal test suite (skip both)
uv run python tests/test_rlm_comprehensive.py --model grok-4-1-fast-reasoning --provider xai --no-256k --no-summarization

uv run python tests/test_rlm_comprehensive.py \
  --model grok-4-1-fast-reasoning \
  --provider xai \
  --output results.json

Each test tracks comprehensive metrics:

• Cost: Total USD spent on API calls
• Tokens: Input/output token counts
• Sub-calls: Number of recursive LLM invocations
• Iterations: Number of code generation cycles
• Reasoning tokens: Extended thinking tokens (for reasoning models)
• Duration: Wall-clock time
• Pass/Fail: Based on accuracy thresholds

Summarization-specific metrics:

• Key point coverage: Percentage of marked key points found in summary
• Compression ratio: Summary length / context length
• Summary quality: Pass requires 70%+ coverage AND ≤0.5 compression ratio

# Sonnet 4.5 orchestrator + Haiku fast sub-tasks (recommended)
--model claude-sonnet-4-5-20250929 --provider anthropic

# Opus 4.6 (most capable)
--model claude-opus-4-6 --provider anthropic

# GPT-5 mini (recommended for testing)
--model gpt-5-mini --provider openai

# GPT-4o
--model gpt-4o --provider openai

# Grok 4 with fast reasoning (recommended)
--model grok-4-1-fast-reasoning --provider xai

# Grok 4 standard
--model grok-4-1 --provider xai

Approximate costs per test (with Grok-4-1-fast-reasoning):

Use --max-cost to set spending limits:

uv run python tests/test_rlm_comprehensive.py --max-cost 1.0  # Stop if any test exceeds $1

# Increase budget or optimize approach
rlm = RecursiveLanguageModel(..., max_cost=10.0)

# Increase iteration limit
result = rlm.run(..., max_iterations=100)

• Check metrics: rlm.print_metrics()
• Enable caching: enable_cache=True
• Use cheaper sub-model: simple_model="claude-haiku-4-5-20251001" or sub_model="gpt-4o-mini"
• Review call history in exported metrics

• Check logs for specific error
• Verify context is valid
• Ensure helpers are used correctly
• Try with enable_security=False for debugging (not recommended for production)

Based on the Recursive Language Model paper concepts:

• What: Recursive sub-calls for long-context processing
• How: REPL-based decomposition with programmatic context access
• Why: Improves accuracy on dense long-context tasks via decomposition
• Patterns: Filtering, chunking, verification, map-reduce

MIT License - See LICENSE file

Contributions welcome! Areas for improvement:

• Async/parallel sub-calls
• Additional model providers (Anthropic, etc.) — Anthropic Claude fully supported
• More advanced helpers
• Visualization of recursion trees
• Performance benchmarks
• Additional examples
• Enhanced test coverage (100k+ token summarization, multi-document fusion)
• Test result visualization and analytics

For issues and feature requests, please open an issue on GitHub.