📖 Read the full blog post: Adaptive Tool Routing: Solving Tool Overload in the MCP Era
Dynamic tool selection for AI agents. ATR filters tools based on user queries before they reach the agent's system prompt, reducing context tokens by ~90% and improving tool selection accuracy.
When agents have access to many tools (50+ MCP tools), two things happen:
- Context explosion - Tool definitions consume 10,000-15,000 tokens before the conversation starts
- Tool selection degradation - Research shows 7-85% accuracy drops with large tool catalogs
ATR intercepts the agent's tool resolution flow and filters tools before they reach the system prompt, using a lightweight LLM (e.g., Claude Haiku, GPT-4o-mini) to select only relevant tools.
Before ATR: 50 tools × 250 tokens = 12,500 tokens
After ATR: 5 tools × 250 tokens = 1,250 tokens
Savings: 90%
User Query ──> ATR Router ──> Lightweight LLM ──> Filtered Tool List ──> Agent
│ │
└── Full tool catalog (50+) └── Only relevant tools (3-5)
- Register your tools with ATR (from any framework - MCP, LangChain, Agno, OpenAI, etc.)
- Before each agent call, pass the user query through ATR's router
- ATR uses a cheap, fast LLM to pick only the relevant tools
- Pass the filtered tools to your agent - smaller context, better accuracy
# Core package (zero dependencies)
pip install adaptive-tools
# With LLM provider
pip install adaptive-tools[openrouter] # Recommended - access to many models
pip install adaptive-tools[openai]
pip install adaptive-tools[anthropic]
# With framework integration
pip install adaptive-tools[langgraph]
pip install adaptive-tools[agno]
pip install adaptive-tools[openai-agents]
pip install adaptive-tools[litellm]
# Everything
pip install adaptive-tools[all]from atr import ToolRouter, ToolSpec
from atr.llm import OpenRouterLLM
# Create router with LLM
router = ToolRouter(llm=OpenRouterLLM())
# Add tools
router.add_tools([
ToolSpec(name="get_stock_price", description="Get current stock price"),
ToolSpec(name="get_company_news", description="Get company news articles"),
ToolSpec(name="get_weather", description="Get weather for a location"),
ToolSpec(name="send_email", description="Send an email"),
ToolSpec(name="create_calendar_event", description="Create a calendar event"),
])
# Route query to filter tools
filtered = router.route("What is AAPL's stock price?")
print(filtered.names) # {'get_stock_price'}from langchain_mcp_adapters.client import MultiServerMCPClient
from atr import ToolRouter
from atr.adapters import LangChainAdapter
from atr.adapters.langchain import filter_tools
from atr.llm import OpenRouterLLM
async with MultiServerMCPClient(server_configs) as client:
all_tools = await client.get_tools()
# Create router from LangChain tools
router = ToolRouter(llm=OpenRouterLLM())
router.add_tools(LangChainAdapter.to_specs(all_tools))
# Route and filter
filtered_specs = router.route("Read the README")
filtered_tools = filter_tools(all_tools, filtered_specs)
# Use filtered tools with your agent
agent = create_react_agent(model, filtered_tools)For LangGraph graphs, use the built-in node creators to add routing as a graph node:
from atr.adapters.langchain import create_async_router_node
# Create a LangGraph-compatible node that routes tools
route_node = create_async_router_node(router, all_tools)
# Add to your graph
graph = StateGraph(AgentState)
graph.add_node("route_tools", route_node)
graph.add_node("agent", agent_node)
graph.add_edge("route_tools", "agent")from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.mcp import MCPTools
from atr import ToolRouter
from atr.adapters import AgnoAdapter
from atr.adapters.agno import filter_tools
from atr.llm import OpenRouterLLM
async with MCPTools(command="npx", args=["-y", "@anthropic/mcp-server-filesystem", "/tmp"]) as mcp:
# Convert toolkit to specs and create router
router = ToolRouter(llm=OpenRouterLLM(), max_tools=5)
router.add_tools(AgnoAdapter.to_specs([mcp]))
# Route and filter
filtered_specs = await router.aroute("List files")
filtered_funcs = filter_tools([mcp], filtered_specs)
# Create agent with filtered tools
agent = Agent(model=OpenAIChat(id="gpt-4o"), tools=filtered_funcs)from atr import ToolRouter
from atr.adapters import OpenAIAdapter
from atr.adapters.openai import filter_tools
from atr.llm import OpenRouterLLM
# Create router from OpenAI function definitions
router = ToolRouter(llm=OpenRouterLLM())
router.add_tools(OpenAIAdapter.to_specs(openai_tools))
# Route and filter
filtered_specs = router.route("What's the weather?")
filtered_tools = filter_tools(openai_tools, filtered_specs)ATR integrates with LiteLLM as a custom hook for automatic tool routing - no manual filtering needed:
import litellm
from atr.adapters.litellm import create_hook
# Create and register the hook
hook = create_hook(
llm_provider="openrouter",
llm_model="anthropic/claude-3-haiku",
max_tools=5,
)
litellm.callbacks = [hook]
# Tools are now automatically filtered before reaching the model
response = await litellm.acompletion(
model="gpt-4",
messages=[{"role": "user", "content": "What's the weather?"}],
tools=all_tools, # ATR filters these automatically
)For LiteLLM proxy, configure in proxy_config.yaml:
litellm_settings:
callbacks:
- atr.adapters.litellm.ATRToolRoutingHook
atr_config:
enabled: true
max_tools: 10
llm_provider: openrouter
llm_model: anthropic/claude-3-haikufrom mcp import ClientSession
from atr import ToolRouter
from atr.adapters import MCPAdapter
from atr.llm import OpenRouterLLM
# Get tools from MCP session
mcp_tools = await session.list_tools()
# Convert and route
router = ToolRouter(llm=OpenRouterLLM())
router.add_tools(MCPAdapter.to_specs(mcp_tools.tools))
filtered_specs = router.route("Read the README")
filtered_tools = MCPAdapter.filter_tools(mcp_tools.tools, filtered_specs)Framework-agnostic tool specification.
ToolSpec(
name="get_weather",
description="Get weather for a location",
parameters={"type": "object", ...}, # JSON Schema (optional)
source="mcp:weather", # Origin identifier (optional)
metadata={}, # Framework-specific data (optional)
)Returned from routing operations. Provides convenient access to filtered tools.
collection = ToolCollection(tools=[...])
collection.names # Set of tool names
collection.filter_by_names(["tool1"]) # Filter by name
collection.to_summaries() # For routing prompts
len(collection) # Number of tools
"tool_name" in collection # Membership check
collection[0] # Index access
collection["tool_name"] # Name-based accessMain router class. Routes queries to select relevant tools.
router = ToolRouter(
llm=OpenRouterLLM(), # Or any RoutingLLM
max_tools=10, # Max tools to return (default: 10)
)
router.add_tools([...])
router.add_tool(single_spec)
router.clear_tools()
filtered = router.route("query") # Sync
filtered = await router.aroute("query") # AsyncAll providers use lazy client initialization and support both sync and async.
from atr.llm import OpenRouterLLM, OpenAILLM, AnthropicLLM
# OpenRouter (recommended - access to many models via single API key)
llm = OpenRouterLLM(model="anthropic/claude-3-haiku") # default model
# OpenAI
llm = OpenAILLM(model="gpt-4o-mini")
# Anthropic
llm = AnthropicLLM(model="claude-3-haiku-20240307")Every adapter follows the same pattern: to_specs() to convert, filter_tools() to filter back.
from atr.adapters import MCPAdapter, LangChainAdapter, AgnoAdapter, OpenAIAdapter, LiteLLMAdapter
# Convert framework tools to ToolSpecs
specs = MCPAdapter.to_specs(mcp_tools)
specs = LangChainAdapter.to_specs(langchain_tools)
specs = AgnoAdapter.to_specs([toolkit_or_function, ...])
specs = OpenAIAdapter.to_specs(openai_tool_defs)
specs = LiteLLMAdapter.to_specs(litellm_tool_defs)
# After routing, filter original tools by the routing result
filtered = MCPAdapter.filter_tools(mcp_tools, filtered_collection)
filtered = LangChainAdapter.filter_tools(langchain_tools, filtered_collection)
filtered = AgnoAdapter.filter_tools([toolkit_or_function, ...], filtered_collection)
filtered = OpenAIAdapter.filter_tools(openai_tool_defs, filtered_collection)
filtered = LiteLLMAdapter.filter_tools(litellm_tool_defs, filtered_collection)Each adapter module also exports a standalone filter_tools() convenience function:
from atr.adapters.langchain import filter_tools
from atr.adapters.agno import filter_tools
from atr.adapters.openai import filter_tools
from atr.adapters.litellm import filter_toolsATR uses a pluggable strategy pattern. The default LLMFilterStrategy uses an LLM, but you can implement your own:
from atr import ToolRouter, ToolCollection, BaseFilterStrategy
class MyCustomStrategy(BaseFilterStrategy):
def filter(self, query: str, tools: ToolCollection) -> ToolCollection:
# Your custom filtering logic (embeddings, keyword matching, etc.)
return tools.filter_by_names(["relevant_tool"])
async def afilter(self, query: str, tools: ToolCollection) -> ToolCollection:
return self.filter(query, tools)
router = ToolRouter(strategy=MyCustomStrategy())Built-in strategies:
LLMFilterStrategy- Uses an LLM to select relevant tools (default whenllmis provided)PassthroughStrategy- Returns all tools unfiltered (default when nollmorstrategyis provided)
- Zero core dependencies - Optional extras only for the frameworks you use
- Fail-open - If routing fails, all original tools are returned
- Protocol-based -
FilterStrategyandRoutingLLMare Protocols, not base classes - duck typing works - Lazy imports - Optional dependencies are loaded only when their adapter is accessed
- Typed - Full type hints with
py.typedmarker, strict mypy config
| Variable | Description |
|---|---|
OPENROUTER_API_KEY |
OpenRouter API key |
OPENAI_API_KEY |
OpenAI API key |
ANTHROPIC_API_KEY |
Anthropic API key |
See the examples/ directory for complete working examples:
basic_usage.py- Core functionality without frameworkslanggraph_example.py- LangGraph with MCP toolsagno_example.py- Agno with MCP and toolkitsopenai_agents_example.py- OpenAI Agents SDKlitellm_example.py- LiteLLM with automatic hook-based routing
git clone https://github.com/yess-ai/atr.git
cd atr
# Install with dev dependencies using uv
uv sync --all-extras
# Run tests
uv run pytest
# Format & lint
uv run ruff format
uv run ruff check --fixMIT License - see LICENSE for details.
Contributions are welcome! Please feel free to submit a Pull Request.