agent-sandbox enables easy management of isolated, stateful, singleton workloads, ideal for use cases like AI agent runtimes.

A framework for efficient model inference with omni-modality models

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Perplexity open source garden for inference technology

My learning notes for ML SYS.

Intelligent Router for Mixture-of-Models

💖🧸 Self hosted, you owned Grok Companion, a container of souls of waifu, cyber livings to bring them into our worlds, wishing to achieve Neuro-sama's altitude. Capable of realtime voice chat, Minec…

Supercharge Your LLM with the Fastest KV Cache Layer

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Cost-efficient and pluggable Infrastructure components for GenAI inference

Fast OS-level support for GPU checkpoint and restore

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

Intercept gRPC traffic of containerd with eBPF

This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.

AIOS: AI Agent Operating System

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

A generative world for general-purpose robotics & embodied AI learning.

DRANET is a Kubernetes Network Driver that uses Dynamic Resource Allocation (DRA) to deliver high-performance networking for demanding applications in Kubernetes.

Heterogeneous AI Computing Virtualization Middleware(Project under CNCF)

Visualize your multi-stage Dockerfiles

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

❤️ 充满爱的 AI 结对编程神器 —— 🐌 Guii Devtool，轻松融入现有前端项目，通过自然语言指令即可轻松定制和优化代码。我们不替代创造者或 Hackers，只愿成为他们桌旁的亲密伙伴，✨ 共同创造美好的产品。

groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

Unlock Unlimited Potential! Share Your GPU Power Across Your Local Network!

This is the Rust course used by the Android team at Google. It provides you the material to quickly teach Rust.

LLM inference in C/C++

Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models.