LLM training in simple, raw C/CUDA

Instant neural graphics primitives: lightning fast NeRF and more

A massively parallel, optimal functional runtime in Rust

📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.🎉

DeepEP: an efficient expert-parallel communication library

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Code and data for paper "Deep Painterly Harmonization": https://arxiv.org/abs/1804.03189

Tile primitives for speedy kernels

[MICRO'23, MLSys'22] TorchSparse: Efficient Training and Inference Framework for Sparse Convolution on GPUs.

Source code that accompanies The CUDA Handbook.

A UNIVERSAL MUSIC TRANSLATION NETWORK - a method for translating music across musical instruments and styles.

GPU implementation of a fast generalized ANS (asymmetric numeral system) entropy encoder and decoder, with extensions for lossless compression of numerical and other data types in HPC/ML applications.

FFPA: Extend FlashAttention-2 with Split-D, ~O(1) SRAM complexity for large headdim, 1.8x~3x↑🎉 vs SDPA.

CUDA Matrix Multiplication Optimization

Custom PTX Instruction Benchmark

High-Performance FP32 GEMM on CUDA devices

A curated set of C++ examples for optimization-based elastodynamic contact simulation using CUDA, emphasizing algorithmic convergence, penetration-free, and inversion-free conditions. Designed for …

Aims to implement dual-port and multi-qp solutions in deepEP ibrc transport

Algorithms implemented in CUDA + resources about GPGPU

A GPU-accelerated general-purpose metaheuristic framework for combinatorial optimization

Writing a CUDA software ray tracing renderer with Analysis-Driven Optimization from scratch: a python-importable, distributed parallel renderer.

Real-time GPU accelerated fluid simulator