SGLang is a fast serving framework for large language models and vision language models.

Training library for Megatron-based models

A NCCL extension library, designed to efficiently offload GPU memory allocated by the NCCL communication library.

Allow torch tensor memory to be released and resumed later

Utility scripts for PyTorch

Ongoing research training transformer models at scale

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & Mixtral)

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

A high-throughput and memory-efficient inference and serving engine for LLMs

Fast and memory-efficient exact attention

PyTorch bindings for CUTLASS grouped GEMM.

Triton-based implementation of Sparse Mixture of Experts.

This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.