Stars
DeepGEMM: clean and efficient BLAS kernel library on GPU
Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism (NIPS'25)
Modern CUDA Learn Notes with PyTorch for Beginners, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.
[NeurIPS 2025] ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitive
🚀🚀 Efficient implementations of Native Sparse Attention
🚀 Efficient implementations for emerging model architectures
CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge techniques in sparse architecture, speculative sampling and qua…
SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).
🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"
FlashInfer: Kernel Library for LLM Serving
📰 Must-read papers on KV Cache Compression (constantly updating 🤗).
A throughput-oriented high-performance serving framework for LLMs
Recipes to scale inference-time compute of open models
[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
[SIGMOD 2025] PQCache: Product Quantization-based KVCache for Long Context LLM Inference
Free, open source crypto trading bot
A curated list of insanely awesome libraries, packages and resources for Quants (Quantitative Finance)
[ICLR2025 Spotlight] MagicPIG: LSH Sampling for Efficient LLM Generation
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filli…
Tile primitives for speedy kernels
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)
Source code of "FlowWalker: A Memory-efficient and High-performance GPU-based Dynamic Graph Random Walk Framework"