A guidance language for controlling large language models.

Mirage Persistent Kernel: Compiling LLMs into a MegaKernel

A simplified implementation for experimenting with RLVR on GSM8K, This repository provides a starting point for exploring reasoning.

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

[ICML 2024 Spotlight] Differentially Private Synthetic Data via Foundation Model APIs 2: Text

Helpful tools and examples for working with flex-attention

Official repository for ACL 2025 paper "ProcessBench: Identifying Process Errors in Mathematical Reasoning"

A comprehensive Rust translation of the code from Sebastian Raschka's Build an LLM from Scratch book.

Pytorch optimiser for training ANNs with exponentiated gradient desent

Democratizing Reinforcement Learning for LLMs

Prepare for DeekSeek R1 inference: Benchmark CPU, DRAM, SSD, iGPU, GPU, ... with efficient code.

Simple RL training for reasoning

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

An ML Systems Onboarding list

The official repository of Quamba1 [ICLR 2025] & Quamba2 [ICML 2025]

Large Reasoning Models

Demonstrations of Loss of Plasticity and Implementation of Continual Backpropagation

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downs…

Hackable and optimized Transformers building blocks, supporting a composable construction.

Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023

The calflops is designed to calculate FLOPs、MACs and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer(Bert、LlaMA etc Large Language Model)

A bibliography and survey of the papers surrounding o1

Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance…

Never use print for debugging again

📰 Must-read papers on KV Cache Compression (constantly updating 🤗).

FlashInfer: Kernel Library for LLM Serving

Profiling and inspecting memory in pytorch

Efficient Triton Kernels for LLM Training