Deploy production-ready AI agents in minutes, not months.
Website โข Documentation โข Examples โข Discord โข Blog
Teleon is a production-grade platform for building, deploying, and scaling AI agents. It's like Vercel for AI Agents โ making deployment as simple as a single command while providing enterprise-grade features out of the box.
# Install
pip install teleon
# Create an agent
from teleon import TeleonClient
client = TeleonClient(api_key="tlk_live_...")
@client.agent(
name="customer-support",
cortex=True,
helix={"min_instances": 1, "max_instances": 100},
sentinel={"content_filtering": True, "pii_detection": True}
)
async def support_agent(customer_message: str, customer_id: str, cortex) -> str:
"""Handle customer support with AI."""
return await process_support_request(customer_message)
# Deploy
$ teleon deploy
# Done! ๐Time to production: 8 minutes
โ
Auto-scaling infrastructure
โ
Persistent, searchable memory
โ
Built-in safety & compliance
โ
Monitoring & tracing
โ
High availability
Building AI agents is easy. Making them production-ready is hard. Most AI prototypes never reach production because of:
- Infrastructure complexity โ Setting up databases, caching, queues, orchestration
- Scalability challenges โ Handling traffic spikes, multi-agent coordination
- Safety & compliance โ Content moderation, PII, GDPR, HIPAA
- Production readiness โ HA, monitoring, memory, cost control
91% of AI prototypes never reach production.
Teleon provides everything you need to go from prototype to production:
| Without Teleon | With Teleon |
|---|---|
| 6โ12 months development | 1 week |
| $200K+ engineering cost | $0 infrastructure engineering |
| Complex setup | One decorator |
| Manual scaling | Automatic |
| Build monitoring | Built-in observability |
| DIY memory systems | Cortex memory included |
| Custom safety tooling | Sentinel guardrails included |
pip install teleonFor specific integrations:
pip install teleon[openai] # OpenAI support
pip install teleon[anthropic] # Anthropic/Claude support
pip install teleon[chromadb] # Vector memory support
pip install teleon[all-storage] # All storage backendsfrom teleon import TeleonClient
client = TeleonClient(api_key="tlk_live_...")
@client.agent(name="hello-world")
async def hello_agent(name: str) -> str:
"""A simple greeting agent."""
return f"Hello, {name}! Welcome to Teleon. ๐"Run it:
# Development mode
teleon dev my_agent.py
# Your agent is now available at http://localhost:8000Test it:
curl -X POST http://localhost:8000/agents/hello-world/execute \
-H "Content-Type: application/json" \
-d '{"name": "Alice"}'teleon init
teleon config set TELEON_API_KEY=<your-api-key>
teleon deploy
# Your agent is live! ๐Cortex gives your agents persistent, searchable memory with 6 simple methods. Enable it with a single parameter.
@client.agent(
name="support",
cortex={
"auto": True,
"scope": ["customer_id"], # Automatic multi-tenant isolation
"auto_context": {
"enabled": True,
"history_limit": 10, # Last N conversations
"relevant_limit": 5, # Top N semantic matches
"max_tokens": 2000
}
}
)
async def support_agent(query: str, customer_id: str, cortex):
# Search for relevant past interactions
results = await cortex.search(
query="previous issues",
filter={"type": "resolution"}
)
# Get recent history
history = await cortex.get(filter={}, limit=10)
# Store new information
await cortex.store(
content=f"Resolved: {query}",
type="resolution"
)
return await generate_response(query, results, history)The 6 Cortex Methods:
| Method | Purpose |
|---|---|
store() |
Save content with any custom fields |
search() |
Semantic search with optional filter |
get() |
Fetch by filter (no semantic search) |
update() |
Update entries matching a filter |
delete() |
Delete entries matching a filter |
count() |
Count entries matching a filter |
Memory Layers for hierarchical knowledge:
@client.agent(
name="assistant",
cortex={
"scope": ["org_id", "user_id"],
"layers": {
"company": {"scope": []}, # Shared across company
"team": {"scope": ["org_id"]}, # Scoped to team
"personal": {"scope": ["user_id"]} # Scoped to user
}
}
)
async def assistant(query: str, org_id: str, user_id: str, cortex):
company_docs = await cortex.company.search(query=query, limit=5)
personal_notes = await cortex.personal.get(filter={}, limit=3)
...Storage Backends:
- Development โ In-memory (default, zero config)
- PostgreSQL + pgvector โ
pip install asyncpg - Redis + RediSearch โ
pip install redis[asyncio]
from teleon.cortex import set_storage_backend, PostgresBackend
backend = PostgresBackend(host="localhost", database="teleon", user="postgres", password="secret")
set_storage_backend(backend)Production-ready runtime that manages agent lifecycle, scaling, health, and LLM costs.
@client.agent(
name="production-agent",
helix={
"min_instances": 2,
"max_instances": 50,
"cpu_limit": 2.0,
"memory_limit_mb": 512,
"health_check_enabled": True,
"health_check_interval": 30,
"startup_timeout": 30,
"shutdown_timeout": 30
}
)
async def production_agent(query: str) -> dict:
return await process(query)Shorthand configuration:
@client.agent(
name="my-agent",
helix={
"min": 2, # min_instances
"max": 10, # max_instances
"cpu": 2.0, # cpu_limit
"memory": 512 # memory_limit_mb
}
)LLM Agent Registration with token budgets and cost control:
from teleon.helix import AgentRuntime, RuntimeConfig, ResourceConfig
runtime = AgentRuntime(RuntimeConfig(environment="production", max_workers=20))
await runtime.register_llm_agent(
agent_id="chat-agent",
agent_callable=chat_handler,
model="gpt-4",
max_tokens=2000,
cost_budget=10.0, # $10/hour budget
resources=ResourceConfig(min_instances=3, max_instances=20)
)Response Caching to cut LLM costs:
from teleon.helix import create_cache, CacheStrategy, CacheEvictionPolicy
cache = create_cache(
max_size=1000,
strategy=CacheStrategy.EXACT,
eviction_policy=CacheEvictionPolicy.LRU,
default_ttl=3600
)
cached = await cache.get(messages, model="gpt-4")
if not cached:
response = await llm.complete(messages, model="gpt-4")
await cache.set(messages=messages, model="gpt-4", response=response, ttl=3600)
stats = cache.get_statistics()
print(f"Hit rate: {stats['hit_rate']*100:.1f}%")
print(f"Tokens saved: {stats['total_tokens_saved']:,}")Health Endpoints for Kubernetes and container orchestration:
from fastapi import FastAPI
from teleon.helix import setup_health_endpoints
app = FastAPI()
health_manager = setup_health_endpoints(app, service_name="my-agent", version="1.0.0")
async def check_database():
await db.ping()
return True, "Database connected"
health_manager.add_check("database", check_database, critical=True)
health_manager.set_ready(True, reason="Initialization complete")| Endpoint | Purpose |
|---|---|
/health |
Overall health (200 healthy / 503 unhealthy) |
/ready |
Readiness probe |
/live |
Liveness probe |
/metrics |
Prometheus-format metrics |
Key Helix features:
- โ Auto-scaling based on CPU, memory, and custom metrics
- โ Health checks and auto-recovery
- โ Load balancing across instances
- โ Zero-downtime deployments
- โ Token tracking and cost budgets per LLM agent
- โ Response caching with LRU/LFU/TTL eviction
- โ Batch processing for high-throughput workloads
- โ Hot reload in development
- โ Production metrics reporter
Real-time guardrails that validate inputs and outputs automatically.
@client.agent(
name="customer-support",
sentinel={
"enabled": True,
"content_filtering": True, # Toxicity, hate speech, profanity
"pii_detection": True, # Emails, phones, SSNs, credit cards, IPs
"compliance": ["gdpr", "hipaa"], # Compliance standards
"moderation_threshold": 0.8, # Sensitivity (0.0โ1.0)
"action_on_violation": "block", # block | flag | redact | escalate
"log_violations": True,
"audit_enabled": True
}
)
async def support_agent(query: str) -> str:
# Sentinel validates input before execution and output after
return await handle_query(query)PII Detection & Redaction:
| Type | Example | Redaction |
|---|---|---|
user@example.com |
[EMAIL_REDACTED] |
|
| Phone (US) | (555) 123-4567 |
[PHONE_REDACTED] |
| Phone (Intl) | +1234567890 |
[PHONE_REDACTED] |
| SSN | 123-45-6789 |
[SSN_REDACTED] |
| Credit Card | 4111-1111-1111-1111 |
[CC_REDACTED] |
| IP Address | 192.168.1.1 |
[IP_REDACTED] |
Compliance Standards:
| Standard | Description |
|---|---|
gdpr |
Data minimization, PII protection, right to erasure |
hipaa |
PHI protection, access control, encryption |
pci_dss |
Credit card data protection |
soc2 |
Security controls, availability, confidentiality |
ccpa |
Consumer privacy rights, data disclosure |
Custom Policies:
from teleon.sentinel.policy_engine import PolicyEngine
policy_engine = PolicyEngine()
policy_engine.add_policy("no_competitors", {
"type": "regex",
"pattern": r"\b(CompetitorA|CompetitorB)\b",
"message": "Competitor mention not allowed",
"severity": "high"
})
policy_engine.add_policy("min_response_length", {
"type": "condition",
"condition": "len(text) < 50",
"message": "Response too short",
"severity": "low"
})Audit Logging:
audit_logger = engine.get_audit_logger()
stats = audit_logger.get_violation_stats(agent_id="my-agent")
# {
# "total_violations": 42,
# "by_type": {"pii_detection": 30, "toxicity": 12},
# "by_action": {"block": 35, "flag": 7}
# }
json_export = audit_logger.export_audit_trail(format="json")
csv_export = audit_logger.export_audit_trail(format="csv")Intelligent routing across multiple LLM providers with caching and fallbacks.
from teleon.llm import LLMGateway
from teleon.llm.providers import OpenAIProvider, AnthropicProvider
gateway = LLMGateway()
gateway.register_provider(OpenAIProvider(api_key="..."))
gateway.register_provider(AnthropicProvider(api_key="..."))
# Simple chat
response = await gateway.chat(message="What is AI?", model="gpt-4")
# Function calling
response = await gateway.chat_with_functions(
message="Get weather in Paris",
functions=[get_weather_function],
model="gpt-4"
)
# Streaming
async for chunk in gateway.stream(message="Write a story", model="gpt-4"):
print(chunk, end="")Supported Providers:
- OpenAI (GPT-4, GPT-3.5)
- Anthropic (Claude)
- Google (Gemini)
- Azure OpenAI
- Groq, Cohere, Together AI
- Local models (Ollama, LM Studio)
116+ production-ready tools across 7 categories.
| Category | Tools | Examples |
|---|---|---|
| Web | 15+ | HTTP requests, web scraping, API calls |
| Data | 20+ | JSON/CSV/XML parsing, data transformation |
| Database | 10+ | SQL queries, MongoDB, Redis operations |
| Files | 15+ | Read/write files, S3, GCS operations |
| Communication | 12+ | Email, Slack, Discord, webhooks |
| Analytics | 8+ | Data analysis, visualization, reporting |
| Utilities | 36+ | Text processing, math, datetime, crypto |
@client.agent(
name="tooled-agent",
tools=["http_request", "web_scraper", "send_email", "sql_query"]
)
async def tooled_agent(task: str) -> dict:
return await execute_with_tools(task)Complete monitoring, tracing, and analytics out of the box.
@client.agent(
name="monitored-agent",
observability={
"tracing": True,
"metrics": True,
"logging": "detailed"
}
)
async def monitored_agent(input: str) -> str:
return await process(input)Production metrics reporting:
from teleon.helix import init_agent_reporter
reporter = await init_agent_reporter(
deployment_id="deploy-123",
api_key="tlk_xxx",
flush_interval=10.0
)
await reporter.report_request(
input_tokens=100,
output_tokens=50,
latency_ms=250.5,
model="gpt-4",
success=True,
cost=0.002
)@client.agent(
name="support-bot",
cortex={
"auto": True,
"scope": ["customer_id"],
"auto_context": {"enabled": True, "history_limit": 20, "relevant_limit": 5}
},
helix={"min_instances": 2, "max_instances": 10},
sentinel={"pii_detection": True, "action_on_violation": "redact"},
tools=["sql_query", "send_email", "slack_message"]
)
async def support_bot(query: str, customer_id: str, cortex) -> dict:
# Auto-context injects relevant history before execution
context = cortex.context.text if cortex.context else ""
similar = await cortex.search(query=query, filter={"type": "resolution"}, limit=3)
response = await llm.chat(
f"Customer query: {query}\nContext: {context}\nSimilar resolutions: {similar}"
)
if "resolved" in response.lower():
await cortex.store(
content=f"Resolved: {query} -> {response}",
type="resolution"
)
return {"status": "resolved", "response": response}@client.agent(
name="healthcare-assistant",
sentinel={
"pii_detection": True,
"compliance": ["hipaa", "gdpr"],
"action_on_violation": "block",
"audit_enabled": True
}
)
async def healthcare_agent(query: str, patient_id: str):
# All PHI is blocked; every violation is logged for HIPAA audit
return await generate_response(query)@client.agent(
name="payments",
sentinel={
"pii_detection": True,
"compliance": ["pci_dss"],
"action_on_violation": "block"
}
)
async def payment_agent(request: dict):
return await process_payment(request)@client.agent(name="data-collector", tools=["web_scraper", "http_request"])
async def data_collector(topic: str) -> dict:
results = []
for source in get_sources(topic):
results.append(await tools.http_request(url=source))
return {"data": results}
@client.agent(name="analyzer", model="gpt-4")
async def analyzer(data: dict) -> dict:
analysis = await llm.chat(f"Analyze this data: {data}", model="gpt-4")
return {"analysis": analysis}
@client.agent(name="research-orchestrator")
async def orchestrator(research_topic: str) -> dict:
data_result = await data_collector(research_topic)
analysis_result = await analyzer(data_result)
return {"topic": research_topic, "analysis": analysis_result["analysis"]}@client.agent(
name="chat",
cortex={"scope": ["session_id"], "auto": False}
)
async def chat_agent(message: str, session_id: str, cortex):
context = await cortex.get(filter={}, limit=10)
response = await llm.chat(message, context)
# Expire session memory after 1 hour
await cortex.store(
content=f"User: {message}\nAssistant: {response}",
type="message",
ttl=3600
)
return responsefrom teleon.scheduler import schedule
from teleon.webhooks import webhook
@client.agent(name="daily-report")
@schedule(cron="0 9 * * *")
async def daily_report() -> dict:
metrics = await gather_daily_metrics()
report = await generate_report(metrics)
await send_email_report(report)
return {"status": "sent"}
@client.agent(name="github-bot")
@webhook(path="/github", methods=["POST"])
async def github_bot(event: dict) -> dict:
if event["type"] == "pull_request":
await review_pr(event["pr_number"])
elif event["type"] == "issue":
await triage_issue(event["issue_number"])
return {"status": "processed"}teleon dev # Hot reload enabled
# Dashboard at http://localhost:8000/dashboardteleon deploy # Fully managed, auto-scaling, 99.99% uptime SLAdocker-compose up -d # Includes Teleon, PostgreSQL, Redis, Prometheus, Grafanateleon deploy --platform kubernetes
helm install teleon ./teleon-chartteleon deploy --platform aws --region us-east-1
teleon deploy --platform azure --region eastus
teleon deploy --platform gcp --region us-central1teleon agents list
teleon agents inspect <agent-id>
teleon agents logs <agent-id> --tail 100
teleon agents exec <agent-name> --input '{"key": "value"}'
teleon agents delete <agent-id>teleon helix status
teleon helix scale <agent-id> --replicas 10
teleon helix health <agent-id>teleon cortex stats <agent-id>
teleon cortex query <agent-id> --query "customer issues"
teleon cortex clear <agent-id> --type episodicteleon sentinel status
teleon sentinel violations <agent-id>
teleon sentinel test <agent-id> --input "test@example.com"
teleon sentinel config <agent-id>
teleon sentinel audit <agent-id> --format jsonteleon dev [file]
teleon test
teleon check
teleon docs generatefrom teleon import TeleonClient
client = TeleonClient(
api_key="tlk_live_...",
environment="production", # "production" | "staging" | "dev"
base_url=None,
verify_key=True
)@client.agent(
name: str, # Agent name (required)
description: str = None,
model: str = "gpt-3.5-turbo",
temperature: float = 0.7,
tools: list[str] = None,
helix: bool | dict = None, # Helix runtime config
cortex: bool | dict = False, # Cortex memory config
sentinel: bool | dict = None, # Sentinel safety config
observability: dict = None,
)# store
entry_id = await cortex.store(content="...", ttl=3600, upsert=True, **custom_fields)
# search (semantic)
results = await cortex.search(query="...", filter={"field": "value"}, limit=10)
# get (filter only, no semantic)
entries = await cortex.get(filter={"field": "value"}, limit=50)
# update
count = await cortex.update(filter={"field": "value"}, content="new", **new_fields)
# delete
deleted = await cortex.delete(filter={"user_id": "alice"})
# count
n = await cortex.count(filter={"type": "query"})
# auto-injected context
cortex.context.entries # List of Entry objects
cortex.context.text # Formatted text for LLM injectionEntry Object:
entry.id # Unique identifier
entry.content # Text content
entry.fields # Dict of all custom fields
entry.created_at # Creation timestamp
entry.updated_at # Last update timestamp
entry.expires_at # Expiration (if TTL set)
entry.score # Relevance score (search only)from teleon.helix import AgentRuntime, RuntimeConfig, ResourceConfig
runtime = AgentRuntime(RuntimeConfig(environment="production", max_workers=20))
# Standard agent
await runtime.register_agent(agent_id, agent_callable, resources=ResourceConfig(...))
# LLM agent with cost tracking
await runtime.register_llm_agent(agent_id, agent_callable, model="gpt-4", cost_budget=10.0)
await runtime.start_agent("my-agent")
await runtime.stop_agent("my-agent", force=False)
await runtime.scale_agent("my-agent", instances=5)
status = await runtime.get_agent_status("my-agent")
# {"status": "running", "instances": 3, "health": "healthy", ...}from teleon.sentinel import SentinelEngine, SentinelConfig, ComplianceStandard, GuardrailAction
config = SentinelConfig(
enabled=True,
content_filtering=True,
pii_detection=True,
compliance=[ComplianceStandard.GDPR, ComplianceStandard.HIPAA],
action_on_violation=GuardrailAction.BLOCK
)
engine = SentinelEngine(config)
result = await engine.validate_input(user_input, agent_name="my-agent")
result = await engine.validate_output(agent_output, agent_name="my-agent")
result.passed # bool
result.action # GuardrailAction
result.violations # List[Dict]
result.redacted_content # Optional[str]from teleon.integrations.azure import AzureOpenAIProvider, AzureStorageClient
from teleon.integrations.aws import BedrockProvider, S3Client, DynamoDBClient
from teleon.integrations.gcp import VertexAIProvider, CloudStorageClientfrom teleon.integrations.slack import SlackClient
from teleon.integrations.discord import DiscordClient
from teleon.integrations.email import EmailClientfrom teleon.integrations.database import PostgresClient, MongoClient
from teleon.integrations.cache import RedisClientfrom teleon.testing import AgentTestCase, MockLLM, MockTool
class TestMyAgent(AgentTestCase):
async def test_agent_response(self):
with MockLLM(response="Mocked response"):
result = await my_agent("test input")
assert result == "Mocked response"
async def test_agent_with_tools(self):
with MockTool("http_request", return_value={"status": 200}):
result = await my_agent("fetch data")
assert "status" in resultLoad testing:
from teleon.testing import load_test
results = await load_test(
agent=my_agent,
input_generator=lambda: {"query": "test"},
concurrent_users=100,
duration_seconds=60
)
print(f"RPS: {results.requests_per_second}")
print(f"P99 latency: {results.p99_latency_ms}ms")
print(f"Error rate: {results.error_rate}%")| Metric | Target | Achieved |
|---|---|---|
| API P99 Latency | < 100ms | 87ms |
| Agent Cold Start | < 5s | 3.2s |
| Agent Warm Start | < 100ms | 45ms |
| Memory Search | < 50ms | 32ms |
| Throughput (API) | 10K req/s | 12.5K req/s |
| Throughput (Agent) | 1K exec/s | 1.3K exec/s |
- Core
@client.agentdecorator - LLM Gateway with multi-provider support
- 116+ built-in tools
- Helix โ Auto-scaling runtime, health checks, token tracking, response caching, batch processing
- Cortex โ Unified memory API (6 methods), scope enforcement, memory layers, auto-context injection, PostgreSQL + Redis backends
- Sentinel โ Content filtering, PII detection, compliance enforcement (GDPR, HIPAA, PCI_DSS, SOC2, CCPA), custom policies, audit logging
- CLI and development server
- Testing framework
- Azure, AWS, GCP integrations
- Enhanced observability dashboards
- Advanced workflow engine
- Custom tool builder UI
- Agent marketplace
- Mobile app for monitoring
- Enhanced security features
- Visual agent builder (no-code)
- Multi-modal agent support
- Edge deployment support
- Advanced cost optimization AI
- Enterprise SSO integration
- Compliance certifications (SOC 2, GDPR)
We welcome contributions! Please see our Contributing Guide for details.
git clone https://github.com/teleonAI/teleon.git
cd teleon
python -m venv venv && source venv/bin/activate
pip install -e ".[dev]"
pytest
black teleon/ && isort teleon/
mypy teleon/Code Standards: Python 3.11+ ยท Black (100 char) ยท isort ยท mypy ยท pytest (80%+ coverage) ยท Conventional commits
Licensed under the Apache License 2.0 โ see LICENSE for details.
Built with: FastAPI ยท Pydantic ยท ChromaDB ยท PostgreSQL ยท Redis ยท OpenTelemetry ยท Docker ยท Kubernetes
- Documentation: docs.teleon.ai
- Discord: Join our community
- Twitter: @Teleon_AI
- Email: founders@teleon.ai
- GitHub Issues: Report bugs
- Discussions: Ask questions
- Getting Started Guide
- Cortex Memory Docs
- Helix Runtime Docs
- Sentinel Safety Docs
- API Reference
- Deployment Guide
- Security Guide