SGLang is a high-performance serving framework for large language models and multimodal models.

Official inference framework for 1-bit LLMs

LLM inference in C/C++

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Agentic Kernel Optimization for All — automated GPU kernel optimization for any kernel, any hardware, any language

Better Harness Tools, not merely storing the archive of leaked Claude Code but also make real things done. Now rewriting in Rust.

A high-throughput and memory-efficient inference and serving engine for LLMs

Xilinx QDMA IP Drivers

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

Systematic and comprehensive benchmarks for LLM systems.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Scalable toolkit for efficient model reinforcement

A highly-flexible GPU simulator for AMD GPUs.

eBPF for GPU UVM offloading and scheduling in Linux kernel

NVIDIA Linux open GPU kernel module source

Lightning-Fast RL for LLM Reasoning and Agents. Made Simple & Flexible.

Lab for fudan program analysis 2024 autumn

Extending eBPF Programmability and Observability to GPUs (merged into https://github.com/eunomia-bpf/bpftime)

Curated collection of papers in machine learning systems

Large Language Model (LLM) Systems Paper List

Optimized primitives for collective multi-GPU communication