Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793

Lightning-Fast RL for LLM Reasoning and Agents. Made Simple & Flexible.

Esoteric Language Models

Lightweight coding agent that runs in your terminal

verl: Volcano Engine Reinforcement Learning for LLMs

Muon is Scalable for LLM Training

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

Fully open reproduction of DeepSeek-R1

Muon is an optimizer for hidden layers in neural networks

A bibliography and survey of the papers surrounding o1

NanoGPT (124M) in 2 minutes

SGLang is a high-performance serving framework for large language models and multimodal models.

A family of compressed models obtained via pruning and knowledge distillation

Code release for "Git Re-Basin: Merging Models modulo Permutation Symmetries"

Code release for REPAIR: REnormalizing Permuted Activations for Interpolation Repair

Development repository for the Triton language and compiler

A framework for few-shot evaluation of language models.

The ChatGPT Retrieval Plugin lets you easily find personal or work documents by asking questions in natural language.

Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Toolkit for creating, sharing and using natural language prompts.

Paper List for In-context Learning 🌷

General technology for enabling AI capabilities w/ LLMs and MLLMs

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)