flex-block-attn: an efficient block sparse attention computation library

DeepSeek-V3.2-Exp DSA Warmup Lightning Indexer training operator based on tilelang

Nex Venus Communication Library

Low overhead tracing library and trace visualizer for pipelined CUDA kernels

Perplexity open source garden for inference technology

High-throughput tensor loading for PyTorch

Hand-Rolled GPU communications library

Tutorial Exercises and Code for GPU Communications Tutorial at HOT Interconnects 2025

Contexts Optical Compression

Implement a reasoning LLM in PyTorch from scratch, step by step

A simple DSP crate

A collection of GPU tests and benchmarks for my own research.

Open source version of DOCA GPUNetIO and DOCA Verbs libraries (limited features) to enable GDAKI technology on RDMA (IB and RoCE)

Distributed MoE in a Single Kernel [NeurIPS '25]

DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit

TVM FFI

Checkpoint-engine is a simple middleware to update model weights in LLM inference engines

NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process communication and coordination overheads by allowing programmer…

Flash Attention from Scratch on CUDA Ampere

Ultra and Unified CCL

VeOmni: Scaling any Modality Model Training to any Accelerators with PyTorch native Training Framework