This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, sgemv, sgemm, etc. The performance of these kernels is basically at or near the theoretical limit.

hpc 教程，包含集合通信(mpi、nccl)、cuda 编程、向量化 SIMD、RDMA 通信等

Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial

🍎 One kernel a day keeps high latency away. A hands-on CUDA learning path featuring a rich collection of kernels, from the basics to peak performance, seamlessly integrated as PyTorch C++ extensions.

Parallel Computing starter project to build GPU & CPU kernels in CUDA & C++ and call them from Python without a single line of CMake using PyBind11

CUDA C implementation of Principal Component Analysis (PCA) through Singular Value Decomposition (SVD) using a highly parallelisable version of the Jacobi eigenvalue algorithm.

LOGAN: High-Performance Multi-GPU X-Drop Long-Read Alignment.

WIP · CUDA compatibility for Blaze · https://bitbucket.org/blaze-lib/blaze

🍒 cuRBLAS (Randomized BLAS) is a GPU-accelerated library for accelerating AI and HPC applications.

A GPU benchmark suite for autotuners

A Low-Overhead tool for Floating-Point Exception Detection in NVIDIA GPUs

Record GPU memory accesses of a CUDA program and visualize the access pattern in a browser

This repository contains various CUDA C programs demonstrating parallel computing techniques using NVIDIA's CUDA platform.

Benchmarking CUDA parallelization on an RTX 3060 — from 1 thread to a 2,256× speedup

Documentation for the N8CIR Bede Tier 2 HPC faciltiy

Floating-point program random generator

BLEST: Blazingly Efficient BFS using Tensor Cores

A CUDA concurrency library designed to simplify concurrency programming, offering C++-style wrappers for selected CUDA Runtime APIs

A high-performance computing project that implements the moment representation of the Lattice Boltzmann Method (LBM) on Nvidia GPUs using CUDA.

GPU-parallel (multi)wavelet adaptive finite volume and discontinuous Galerkin shallow water model