OpenClaw-RL: Train any agent simply by talking

A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.

MOSS-TTSD is a spoken dialogue generation model designed for expressive multi-speaker synthesis. It features long-context modeling, flexible speaker control, and multilingual support, while enabling zero-shot voice cloning from short audio references.

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

MOVA: Towards Scalable and Synchronized Video–Audio Generation

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Open Source Continuous Inference Benchmarking Qwen3.5, DeepSeek, GPTOSS - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 vs H100 & soon™ TPUv6e/v7/Trainium2/3

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

基于SparkTTS、OrpheusTTS等模型，提供高质量中文语音合成与声音克隆服务。

gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR、TTS、文生图、图片编辑和文生视频的开源框架。

sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems

Efficient LLM inference on Slurm clusters.

A PyTorch native library for training speculative decoding models

Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.

A tool for benchmarking LLMs on Modal

SGLang model provider for Strands Agents for on-policy agentic RL training.

Automated system for LLM evaluation via agents.

Boosting GPU utilization for LLM serving via dynamic spatial-temporal prefill & decode orchestration

DeepSeek-V3, R1 671B on 8xH100 Throughput Benchmarks