My learning notes for ML SYS.

super repo for rocm systems projects

[DEPRECATED] Moved to ROCm/rocm-libraries repo. NOTE: develop branch is maintained as a read-only mirror

Analyze computation-communication overlap in V3/R1.

Modular RDMA Interface

AI Tensor Engine for ROCm

Optimized primitives for collective multi-GPU communication

Public repo for HF blog posts

[ICLR 2026] When it comes to optimizers, it's always better to be safe than sorry

High-Performance C++ Fundamental Library

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

Fast and memory-efficient exact attention

Transformers 库快速入门教程

LLM101n: Let's build a Storyteller

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

LLM training in simple, raw C/CUDA

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

Proxy: Next Generation Polymorphism in C++

Open source code for AlphaFold 2.

Trainable, memory-efficient, and GPU-friendly PyTorch reproduction of AlphaFold 2

The Serenity Operating System 🐞

PyTorch Tutorial for Deep Learning Researchers

Pretrain, finetune and serve LLMs on Intel platforms with Ray

LLM inference in C/C++

A data oriented, simple but powerful DSL language.

Port of OpenAI's Whisper model in C/C++

Tensors and Dynamic neural networks in Python with strong GPU acceleration

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

General technology for enabling AI capabilities w/ LLMs and MLLMs

Hash function quality and speed tests