Sentry SDK for Dart and Flutter

SGLang is a fast serving framework for large language models and vision language models.

Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)

Integrate cargo build with flutter plugins and applications.

slime is a LLM post-training framework aiming at scaling RL.

OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)

FlashInfer: Kernel Library for LLM Serving

A framework for few-shot evaluation of language models.

Training library for Megatron-based models

FlashMLA: Efficient Multi-head Latent Attention Kernels

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorR…

A project to improve skills of large language models

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

NVIDIA's launch, startup, and logging scripts used by our MLPerf Training and HPC submissions

Ongoing research training transformer models at scale

DeepEP: an efficient expert-parallel communication library

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory…

Development repository for the Triton language and compiler

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

LongBench v2 and LongBench (ACL 25'&24')

CUDA Templates for Linear Algebra Subroutines