High Performance LLM Inference Operator Library

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization

mKernel: fast multi-node, multi-GPU fused kernels

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

slime is an LLM post-training framework for RL Scaling.

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.

RLinf: Reinforcement Learning Infrastructure for Embodied and Agentic AI

The RL Bridge for LLM-based Agent Applications. Made Simple & Flexible.

A Datacenter Scale Distributed Inference Serving Framework

FlashInfer: Kernel Library for LLM Serving

SGLang is a fast serving framework for large language models and vision language models.

A framework for efficient model inference with omni-modality models

Open Source Continuous Inference Benchmarking Qwen3.5, DeepSeek, GPTOSS - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 vs H100 & soon™ TPUv6e/v7/Trainium2/3

MLX: An array framework for Apple silicon

A high-throughput and memory-efficient inference and serving engine for LLMs

DeepSeek 4 Flash local inference engine for Metal

TokenSpeed is a speed-of-light LLM inference engine.

Lucebox optimization hub: hand-tuned LLM inference, built for specific consumer hardware.

Benchmarking code for running quantized kernels from vLLM and other libraries

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

high-performance linear attention kernel library built on TileLang

Official code base for LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

DFlash: Block Diffusion for Flash Speculative Decoding

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.