A high-throughput and memory-efficient inference and serving engine for LLMs

Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

FlashMLA: Efficient Multi-head Latent Attention Kernels

CUDA Templates and Python DSLs for High-Performance Linear Algebra

A Datacenter Scale Distributed Inference Serving Framework

EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL

A fast multimodal LLM for real-time voice

A framework for efficient model inference with omni-modality models

Entropy Based Sampling and Parallel CoT Decoding

how to optimize some algorithm in cuda.

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

CUDA/Metal accelerated language model inference

Discrete Diffusion Forcing (D2F): dLLMs Can Do Faster-Than-AR Inference

documentation for content creation

Manages vllm-nccl dependency

Cross-platform transformer training