verl: Volcano Engine Reinforcement Learning for LLMs

VeOmni: Scaling any Modality Model Training to any Accelerators with PyTorch native Training Framework

Async pipelined version of Verl

Enjoy the magic of Diffusion models!

[WIP] The all in one inference optimization solution for ComfyUI, universal, flexible, and fast.

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters

A comprehensive toolkit for reliably locking, packing and deploying environments for ComfyUI workflows.

HunyuanVideo: A Systematic Framework For Large Video Generation Model Training

[CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

The best OSS video generation models

faster parallel inference of mochi video generation model

(WIP) Parallel inference for black-forest-labs' FLUX model.

[WIP] Context parallel attention that works with torch.compile

images for xdit

Quantized Attention that achieves speedups of 2.1x and 2.7x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

text and image to video generation: CogVideoX (2024) and CogVideo (ICLR 2023)

Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

Optimizing AlphaFold Training and Inference on GPU Clusters

📰 Must-read papers and blogs on Speculative Decoding ⚡️