llm-ops

🦍 The API and AI Gateway

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Open Source API and AI Gateway supporting REST, GraphQL, TCP, gRPC and MCP (Model Context Protocol)

Enterprise-grade, commercial-friendly agentic workflow platform for building next-generation SuperAgents.

Julep — durable, composable AI agents. Flows that crash and resume, retry safely, and explain every step.

The platform for LLM evaluations and AI agent testing

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.

The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.

AIConfig is a config-based framework to build generative AI applications.

China's 1st enterprise multi-agent IT ops platform. LLM-powered auto-remediation for Zabbix/Prometheus. Docker deploy.

Open Source Context infrastructure for AI agents. Auto-capture and share your agents' context everywhere.

Declarative governance framework for AI-augmented software development lifecycles

Python SDK for running evaluations on LLM generated responses

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer

OpenTelemetry-native LLM observability CLI. Point any OTEL exporter at it and watch your LLM/agent traces in real time.

High-performance Ollama & LM Studio proxy with per-user fair-share queuing, round-robin scheduling, and a real-time TUI dashboard. Built in Rust.

Static analysis for AI agent configs, tool descriptions, and system prompts — catches vague tool descriptions, missing stop conditions, and schema gaps before they reach runtime. Zero-LLM, deterministic checks, built for CI.

[DEPRECATED] Moved to microsoft/agent-governance-toolkit