📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.🎉

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

how to optimize some algorithm in cuda.

This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, s…

Fast CUDA matrix multiplication from scratch

Fastest kernels written from scratch

Algorithms implemented in CUDA + resources about GPGPU

Step by step implementation of a fast softmax kernel in CUDA

TLB Benchmarks