TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.

⚙️🦀 Build modular and scalable LLM Applications in Rust

Delivery infrastructure for agents. Arch is a models-native proxy and data plane for agents that handles plumbing work in AI - like agent routing and orchestration, guardrails, zero-code logs and traces, and unified access to LLMs (OpenAI, Anthropic, Ollama, etc.). Build agents faster and deliver them reliably to prod.

Bionic is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality

AICI: Prompts as (Wasm) Programs

Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.

Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙

A reactive runtime for building durable AI agents

🧠 Motorhead is a memory and information retrieval server for LLMs.

Fast, streaming indexing, query, and agentic LLM applications in Rust

Govern & Secure your AI

High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included

Simple, Composable, High-Performance, Safe and Web3 Friendly AI Agents and LazAI Gateway for Everyone

🧬 The adaptive model routing system for exploration and exploitation.

🧭🧭 An intelligent load balancer with smart scheduling that unifies diverse LLMs.

Robot VLM and VLA (Vision-Language-Action) inference API helping you manage multimodal prompts, RAG, and location metadata

Prompt-centric framework for developing LLM applications in Rust

Burgonet Gateway is an enterprise LLM gateway that provides secure access and compliance controls for AI systems

A Lazy, high throughput and blazing fast structured text generation backend.

[WIP] Sorai is a lightweight, high-performance, and open-source LLM proxy gateway.