TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

Route OpenAI-compatible requests to LLM providers with a fast Rust API gateway

Litellm inspired LLM gateway with cost tracking, guardrails, load balancing and logging.

Fast, streaming indexing, query, and agentic LLM applications in Rust

Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

Simple, Composable, High-Performance, Safe and Web3 Friendly AI Agents and LazAI Gateway for Everyone

Plano is an AI-native proxy and data plane for agentic apps — with built-in orchestration, safety, observability, and smart LLM routing so you stay focused on your agents core logic.

⚙️🦀 Build modular and scalable LLM Applications in Rust

An open source, Native AI Gateway and LLM proxy built in Rust

A production-ready framework for composing AI agents from declarative TOML configuration, with MCP tool integration, RAG pipelines, and an OpenAI-compatible web API.

Code intelligence for humans, machines, and LLMs: receipts, metrics, and insights from your codebase.

A lightweight, secure executor for Claude Agent Skills. Inspired by OpenClaw but strictly designed with zero-trust whitelisting, written in Rust.

Zero-code LLM security & observability proxy. Real-time prompt injection detection, PII scanning, and cost control for OpenAI-compatible APIs. Built in Rust.

A Cookbook of recipes from batuta

A QA playbook system for running through scenario testing

Bionic is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality

Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.

The dashcam and emergency brake for AI agents. A security proxy that physically blocks rogue LLM commands and generates cryptographically proven audit trails for enterprise compliance.

Govern & Secure your AI

High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included