Transformer related optimization, including BERT, GPT

Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, Qwen3, Llama 4, DeepSeek-R1, Gemma 3, TTS 2x faster with 70% less VRAM.

Fully open reproduction of DeepSeek-R1

CUDA Templates for Linear Algebra Subroutines

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

An innovation library for efficient LLM inference via low-bit quantization and sparsity

A high-throughput and memory-efficient inference and serving engine for LLMs

8-bit CUDA functions for PyTorch

Fast and memory-efficient exact attention

Inference code for LLaMA models

Robust Speech Recognition via Large-Scale Weak Supervision

🏎️ Accelerate training and inference of 🤗 Transformers with easy to use hardware optimization tools

Development repository for the Triton language and compiler

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch

ONNX Runtime: cross-platform, high performance scoring engine for ML models

Open Neural Network Exchange

MatMul Performance Benchmarks for a Single CPU Core comparing both hand engineered and codegen kernels.

Tutorials for creating and using ONNX models

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Samples for Windows ML.