A Datacenter Scale Distributed Inference Serving Framework

Cost-efficient and pluggable Infrastructure components for GenAI inference

A local-first routing and coordination engine for long-running agent work.

A workload for deploying LLM inference services on Kubernetes

FlashInfer: Kernel Library for LLM Serving

Enhancements tracking repo for Kubernetes

Rapid and cost-effective operator and best practice for agent sandbox lifecycle management.

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

Production-Grade Container Scheduling and Management

Kubernetes controllers for fast model actuation using vLLM sleep/wake and launcher-based model swapping

LLM inference in C/C++

Delivers efficient, stable, and secure data distribution and acceleration powered by P2P technology, with an optional content‑addressable filesystem that accelerates OCI container launch.

CUDA Library Samples

Nephio is a Kubernetes-based automation platform for deploying and managing highly distributed, interconnected workloads such as 5G Network Functions, and the underlying infrastructure on which tho…

llm-d Router: The intelligent entry point for inference requests

Achieve state of the art inference performance with modern accelerators on Kubernetes

Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.

Variant optimization autoscaler for distributed inference workloads

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

Stateful API logic for agentic applications using vLLM

vLLM 与 SGLang： 大模型高效推理双引擎实战配套代码

A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

Comet: OpenSpec + Superpowers dual-star development workflow

Turn massive model configs into memory-bounded test models that preserve key architecture and inference paths. (把跑不动的超大模型配置，缩成 保留关键架构与推理路径的“可调试测试模型“)

38 editorial diagram types for Claude Code, Codex, and Pi. Self-contained HTML + SVG. No shadows. No Mermaid slop.