DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Making large AI models cheaper, faster and more accessible
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
A GPipe implementation in PyTorch
Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.
飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。
InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
A curated list of awesome projects and papers for distributed training or inference
Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
Serving Inside Pytorch
Decentralized LLMs fine-tuning and inference with offloading
Split FLUX.2 and LTX 2.3 across two GPUs (LAN or same-machine) — NVENC compresses activations live on the wire. Icarus (ComfyUI node) + Daedalus (back-half server).
Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
An Efficient Pipelined Data Parallel Approach for Training Large Model
Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.
An Efficient and Versatile Inference Engine for Distributed LLM Serving
[ICPP'25] TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
FTPipe and related pipeline model parallelism research.
An implementation of an autoregressive language model using an improved Transformer and DeepSpeed pipeline parallelism.
To associate your repository with the pipeline-parallelism topic, visit your repo's landing page and select "manage topics."