Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Harbor is a framework for running agent evaluations and creating and using RL environments.

A high-throughput and memory-efficient inference and serving engine for LLMs

A high-performance and light-weight router for vLLM large scale deployment

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, 20+ clouds, or on-prem).

SkyRL: A Modular Full-stack RL Library for LLMs

A PyTorch native platform for training generative AI models

Ongoing research training transformer models at scale

veRL: Volcano Engine Reinforcement Learning for LLM

TORCH_LOGS parser for PT2

The 100 line AI agent that solves GitHub issues or helps you in your command line. Radically simple, no huge configs, no giant monorepo—but scores 68% on SWE-bench verified!

Implementation for FP8/INT8 Rollout for RL training without performence drop.

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

A series of math-specific large language models of our Qwen2 series.

Sky-T1: Train your own O1 preview model within $450

Fork of FastChat, an open platform for training, serving, and evaluating large language models. Release repo for Vicuna and FastChat-T5.

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

Train transformer language models with reinforcement learning.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Efficient Triton Kernels for LLM Training

Entropy Based Sampling and Parallel CoT Decoding

Pygloo provides Python bindings for Gloo.