Stars
An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.
FlashInfer: Kernel Library for LLM Serving
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
Tile primitives for speedy kernels
how to optimize some algorithm in cuda.
Mirage Persistent Kernel: Compiling LLMs into a MegaKernel
This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, s…
[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.
Step-by-step optimization of CUDA SGEMM
An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).
Production-ready MoE load balancing via real-time expert replication
B200 Attention from Scratch: understand one of the hardest GPU kernels on the latest hardware
Step by step implementation of a fast softmax kernel in CUDA
A collection of GPU experiments and benchmarks for my personal understanding and research.
Flash Attention in raw Cuda C beating PyTorch