Stars
Two case studies: megakernels roughly on par with CUDA Graphs + PDL. Code and baselines for further research.
FlashInfer: Kernel Library for LLM Serving
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
llm-d Router: The intelligent entry point for inference requests
Programmable chat templates for LLM training and inference.
Helpful kernel tutorials, examples and SKILLs for tile-based GPU programming
A storage solution for PyTorch tensors with distributed tensor support.
Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.
A high-performance and light-weight router for vLLM large scale deployment
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
https://github.com/gpu-mode/reference-kernels
Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours
Implementing scalable LLMs in pure JAX (no third-party libraries)
A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance…
General plug-and-play inference library for Recursive Language Models (RLMs), supporting various sandboxes.
High-performance distributed data shuffling (all-to-all) library for MoE training and inference
A custom AI chip to be taped out soon!
Perplexity open source garden for inference technology
An experimental implementation of compiler-driven automatic sharding of models across a given device mesh.
A multi-platform desktop application to help you study by automatically creating flashcard decks and tests based on your study materials.