deepspeedPyTorch library for distributed training of very large models with ZeRO43.3k starsApache-2.0last commit Oct 2026
87
100
81
45
85 out of 100score
3
verlReinforcement learning training library for LLM post-training with PPO and GRPO24k starsApache-2.0last commit Oct 2026
74
100
81
85
85 out of 100score
#4
peftPython library for LoRA and other parameter-efficient fine-tuning of Transformers models
Live demo ↗
(opens in a new tab)21.8k starsApache-2.0last commit Oct 2026
71
100
90
70
85 out of 100score
#5
trlPost-training library with SFT, DPO, GRPO and KTO trainers19.6k starsApache-2.0last commit Oct 2026
66
100
88
70
83 out of 100score
#6
megatron-lmDistributed training library for large transformer models on NVIDIA GPUs18.3k starscustom licenselast commit Oct 2026
snorkelPython library for programmatic labeling of machine learning training data6k starsApache-2.0last commit Apr 2026
10
43
0
0
16 out of 100score
#39
torchtunePyTorch library of LLM fine-tuning recipes driven by YAML configs5.8k starsBSD-3-Clauselast commit Apr 2026
2
46
0
0
15 out of 100score
Momentum, Verified build, Docs and Privacy are not measured yet; their weight goes to the signals shown. A dash means the signal is not scored for that kind of project. Hover a number for its rating in words.
Unsloth ships as a native desktop app, a web UI (Unsloth Studio) and a Python library (Unsloth Core). It runs GGUF, MLX, diffusion, audio and embedding models, serves them over an OpenAI-compatible API, and fine-tunes them with LoRA, QLoRA, full fine-tuning, GRPO, DPO and FP8. The README claims 2x faster training with 70% less VRAM.
Strengths
Covers both running and fine-tuning models, with export to GGUF, NVFP4 and FP8
Supports NVIDIA, AMD, Intel GPUs, CPU and Vulkan on Windows, Linux, WSL and macOS
Free Colab and Kaggle notebooks for many models, including GRPO and TTS
`unsloth start` connects Claude Code, Codex and other agents to local models
Weaknesses
Speed and VRAM figures (2x, 70% less) are the project's own claims
Server-side tools are on by default; exposing it needs a password or --disable-tools
Three install paths (Desktop, Studio, Core) with many environment flags add complexity
RAM and VRAM minimums are not stated in the README
PyTorch library for distributed training of very large models with ZeRO
43.3k starsApache-2.0last commit Oct 2026
DeepSpeed is a PyTorch library that makes large-model training fit and scale across GPUs using ZeRO, ZeRO-Infinity offload, 3D parallelism, Ulysses sequence parallelism and DeepSpeed-MoE. It is configured through a JSON file and integrates with Hugging Face Transformers, Accelerate, Lightning, MosaicML Composer, Determined and MMEngine. C++/CUDA ops are built just-in-time with ninja by default.
Strengths
Used to train models such as BLOOM 176B and Megatron-Turing NLG 530B
Integrates with Hugging Face Transformers and Accelerate, Lightning, Composer and Determined
Supports NVIDIA, AMD, Intel Gaudi and XPU, Huawei Ascend NPU and CPU accelerators
Installs with pip install deepspeed; ds_report checks which ops are compatible
Weaknesses
PyTorch must be installed first, and a CUDA or ROCm compiler is needed for JIT ops
Windows lacks async IO and GDS support
Tested mainly on NVIDIA Pascal to Hopper and AMD MI100/MI200; other hardware is less validated
Training-focused; tuning requires understanding the JSON config and ZeRO stages
GPU optional
Docker
Needs PyTorch, CUDA or ROCm compiler (nvcc/hipcc), ninja
Reinforcement learning training library for LLM post-training with PPO and GRPO
24k starsApache-2.0last commit Oct 2026
verl is the open-source implementation of the HybridFlow paper from ByteDance Seed. It runs RL and supervised fine-tuning for LLMs using FSDP, FSDP2 or Megatron-LM for training and vLLM, SGLang or HF Transformers for rollout generation. It covers PPO, GRPO, RLOO, ReMax and others, with multi-turn tool calling, VLM support and scaling to 671B models.
Strengths
Pluggable backends: FSDP/FSDP2/Megatron-LM for training, vLLM/SGLang/HF for rollout
Many RL algorithms built in: PPO, GRPO, ReMax, RLOO, REINFORCE++
Scales to 671B models and hundreds of GPUs via expert parallelism
Supports NVIDIA, AMD and Ascend hardware; LoRA RL to save memory
Weaknesses
Needs multi-GPU clusters in practice; minimum hardware not stated in README
Many algorithm recipes live in a separate verl-recipe repository and submodule
Version-sensitive: README warns against vLLM 0.7.x and requires >=0.8.2
Several features remain under verl/experimental until merged into the main library
PEFT wraps a pretrained model with adapter methods such as LoRA, soft prompts and IA3 so only a small set of extra parameters is trained. Adapters are saved as small checkpoints (19MB versus 11GB for T0_3B in the README's example). It integrates with Transformers, Diffusers, Accelerate and TRL, and can be combined with quantization such as QLoRA.
Strengths
Adapter checkpoints are MBs instead of GBs; 19MB vs 11GB for T0_3B
LoRA trained a 12B model (mt0-xxl) on one 80GB GPU where full finetuning ran out of memory
Integrates with Transformers, Diffusers, Accelerate and TRL; works with quantized base models
Install with pip; get_peft_model wraps an existing model in a few lines
Weaknesses
Library only: no training UI, scheduler or serving; training loop is your own or Trainer
Transformers integration lacks some PEFT features, such as merging adapters into the base model
Models without official support need manual config for target modules
Memory figures in the README assume an A100 80GB with over 64GB CPU RAM
Post-training library with SFT, DPO, GRPO and KTO trainers
19.6k starsApache-2.0last commit Oct 2026
TRL provides trainer classes (SFTTrainer, GRPOTrainer, DPOTrainer, KTOTrainer, RewardTrainer) that wrap the Hugging Face Transformers trainer for post-training language models. It scales through Accelerate with DDP, DeepSpeed ZeRO and FSDP, supports LoRA/QLoRA via PEFT, and ships a `trl` CLI for running SFT, DPO and KTO without code.
Strengths
Covers SFT, DPO, GRPO, KTO and reward modeling with one trainer API
Scales from one GPU to multi-node via Accelerate, DDP, DeepSpeed and FSDP
Integrates PEFT (LoRA/QLoRA) and Unsloth for training on modest hardware
CLI commands (trl sft, dpo, kto) run fine-tuning without writing code
Weaknesses
Code-first Python library with no web UI
Features under trl.experimental may change or be removed without notice
Built around the Hugging Face Transformers ecosystem
Distributed training library for large transformer models on NVIDIA GPUs
18.3k starscustom licenselast commit Oct 2026
Megatron Core is a Python library of transformer building blocks with tensor, pipeline, data, expert and context parallelism, plus FP16, BF16, FP8 and FP4 mixed precision. The repo also ships Megatron-LM, a reference set of pre-configured training scripts built on it. The README reports training models from 2B to 462B parameters, reaching up to 47% MFU on H100 clusters.
Strengths
Supports TP, PP, DP, EP and CP parallelism for multi-node training
Mixed precision down to FP8 and FP4
Benchmarked at 462B parameters on 6,144 H100 GPUs, up to 47% MFU
Includes MoE, hybrid Mamba, post-training and RL code
Weaknesses
Aimed at multi-GPU NVIDIA clusters; not practical for single-machine hobby fine-tuning
Source builds can run out of memory; MAX_JOBS must be limited
Python 3.10 support dropped; downstream projects need Python 3.12 or later
Some features, such as DeepSeek-V4, are only on the experimental dev branch
GPU required
Docker
Needs PyTorch, NVIDIA GPUs, NGC container (optional)
Axolotl runs LLM post-training from a single YAML config covering dataset preprocessing, training, evaluation, quantization and inference. It supports full fine-tuning, LoRA/QLoRA, QAT, DPO/KTO/ORPO, GRPO and reward modelling, with FSDP2, DeepSpeed, and Torchrun or Ray multi-node. Models come from the Hugging Face Hub, including multimodal and audio ones.
Strengths
One YAML config reused across preprocessing, training, evaluation, quantization and inference
Broad methods: full FT, LoRA/QLoRA, QAT, DPO/KTO/ORPO, GRPO, reward models
Multi-GPU and multi-node via FSDP2, DeepSpeed, Torchrun and Ray, plus context and expert parallelism
Fast model support updates; GGUF export via `axolotl export` for llama.cpp and Ollama
Weaknesses
Requires NVIDIA (Ampere or newer for bf16) or AMD GPU; no CPU path stated
Needs Python 3.12+ and PyTorch 2.13+, a narrow environment
Configuration surface is large; many options and fast-moving features
No web UI; command-line and YAML only
GPU required
Compose
Needs PyTorch, Hugging Face Hub, DeepSpeed (optional), uv
Fine-tuning, RL, quantization and serving toolkit for 600+ text and 400+ multimodal LLMs
15.9k starsApache-2.0last commit Oct 2026
ms-swift is a Python framework from the ModelScope community for training, evaluating, quantizing and serving LLMs and multimodal models through the `swift` CLI, a Gradio Web-UI or Python. It covers LoRA/QLoRA and full-parameter SFT, DPO/KTO and GRPO-family reinforcement learning, with Megatron parallelism (TP/PP/CP/EP) for large and MoE models. Inference and deployment run on Transformers, vLLM, SGLang or LMDeploy, and evaluation uses EvalScope.
Strengths
Covers SFT, DPO/KTO/RM, GRPO variants, embedding and reranker training in one CLI
Megatron TP/PP/CP/EP support for full-parameter and LoRA training of MoE models
Runs on NVIDIA CUDA, AMD ROCm, Ascend NPU, Apple MPS and CPU
Quick start LoRA-tunes Qwen3-4B in about 13GB on a single 3090
Weaknesses
Large dependency matrix (torch, transformers, peft, trl, deepspeed, vllm) with version pins
No Docker image or compose file in the README
Downloads models and datasets from ModelScope by default; Hugging Face needs --use_hf true
4.x/3.x split; the older 3.x line requires checking out a release branch
LlamaFactory fine-tunes language and multimodal models through YAML-configured CLI runs or the LLaMA Board Gradio UI. It covers pre-training, SFT, reward modeling, PPO, DPO, KTO and ORPO, with full-tuning, freeze-tuning, LoRA and 2 to 8-bit QLoRA. Trained models can be served through an OpenAI-style API using vLLM or SGLang workers.
CLI for fine-tuning and post-training LLMs from a single YAML config
9.6k starsApache-2.0last commit Oct 2026
Soup is a Python CLI that runs LoRA/QLoRA fine-tuning and preference or RL post-training (SFT, DPO, GRPO, PPO, KTO, ORPO and others) from one soup.yaml. It also covers batch inference, an OpenAI-compatible server, merge and GGUF export, and a local web dashboard. An opt-in beta layer-streaming mode trains an 8B NF4 model in about 3.3 GB peak VRAM.
Strengths
Many tasks in one config: SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO, vision, audio
Layer streaming trains Llama-3.1-8B NF4 in 3.32 GB peak on a 4 GB laptop GPU
Same tool merges adapters, exports GGUF, serves an OpenAI-compatible API, has a web UI
Light install without PyTorch for data tools; 100+ ready-made model recipes
Weaknesses
Python 3.10-3.12 only; 3.13+ is unsupported
Layer streaming is opt-in and still beta; a correctness repair changed its benchmark numbers
Recent releases are breaking: unused settings and unknown config keys now fail at load
AWQ and GPTQ export are deprecated and slated for removal in v0.77
Removes refusal behavior from transformer language models via automated abliteration
36.3k starsAGPL-3.0last commit Oct 2026
Heretic applies directional ablation (abliteration) to the attention out-projection and MLP down-projection matrices of a transformer model. An Optuna TPE optimizer searches the ablation parameters, co-minimizing refusals and KL divergence from the original model. After a run it can save the model, upload it to Hugging Face, chat with it, or run benchmarks.
Strengths
One command, no configuration needed; Optuna TPE tunes ablation parameters automatically
README reports 3/100 refusals at 0.16 KL on gemma-3-12b-it, versus 0.45 and 1.04 for manual abliterations
Supports most dense models, several MoE architectures, and some multimodal and hybrid models
Reinforcement learning framework for training agents through a model-request proxy
18.8k starsMITlast commit Oct 2026
Agent Lightning is a roughly 3,500-line Python framework that trains agents with RL while they run in their real harnesses. A gateway proxies the agent's model requests and records them as training data, a rollout controller runs agents locally or as Kubernetes Jobs, and a trainer built on verl and vLLM updates the policy. Examples include a Qwen3.5-9B coding agent, which the README reports going from 41.8% to 56.4% on SWE-bench Verified.
Strengths
Agents run unchanged; the gateway captures model calls as training data
Runs rollouts as Kubernetes Jobs without an external sandbox service
Full coding-agent pipeline released: data cleaning, reward-hacking prevention, training scripts
Examples cover Calc-X, GSM8K, ScienceWorld, Search-R1, LLM-in-Sandbox and coding agents
Weaknesses
Requires a verl and vLLM GPU stack; install shown for CUDA 13.0 only
No Docker or compose setup shipped in the repo
v1.0 is a full refactor; older releases live on a separate v0.x branch
Hardware and RAM requirements are not stated in the README
GPU required
Needs verl, vLLM, uv, Kubernetes (optional, for rollouts)
FinGPT is a collection of LoRA adapters, instruction datasets and training notebooks for financial NLP tasks such as sentiment analysis, relation extraction, headline classification, NER and stock-movement forecasting. Adapters are published on Hugging Face for Llama2, Falcon, BLOOM, MPT, ChatGLM2 and Qwen bases. The README reports a Llama2-13B sentiment adapter trained on one RTX 3090 in about 17 hours.
Strengths
Published LoRA adapters and datasets on Hugging Face for several base models
Notebooks fine-tune on a single RTX 3090 in 8-bit or int4 QLoRA
Benchmark table reports weighted F1 on FPB, FiQA-SA, TFNS and NWGI with cost per run
Forecaster has a hosted Hugging Face Space demo, so no install is needed to try it
Weaknesses
Forecaster is trained on US DOW 30 data only; other markets need your own data and fine-tuning
Most releases date from 2023 and use older bases such as Llama2 and ChatGLM2
Research and notebook code, not a packaged app or service; no Docker setup
Cost comparisons with BloombergGPT rely on the project's own GPU-hour assumptions
GPU optional
Needs Hugging Face Transformers, PEFT, OpenAI API (optional cloud provider)
CLI and Python toolkit for fine-tuning, evaluating and deploying foundation models
9.4k starsApache-2.0last commit Oct 2026
Oumi covers the model lifecycle from one YAML-driven CLI: data synthesis with LLM judges, training (SFT, LoRA, QLoRA, DPO, GRPO), evaluation, and inference through vLLM, SGLang or commercial APIs. Recent versions add tool-calling SFT, executable tool environments and GRPO via verl. Jobs can run locally or be launched on AWS, Azure, GCP, Lambda, Modal or Slurm. The README labels it beta.
Strengths
Ready-made YAML recipes for many model families (Qwen, Gemma, Llama, gpt-oss, Phi)
Supports SFT, LoRA, QLoRA, DPO, KTO and GRPO, with FSDP, DeepSpeed and DDP
`oumi launch` runs jobs on AWS, Azure, GCP, Lambda and Slurm
Includes eval, LLM-as-judge, data synthesis and an `oumi-mcp` MCP server
Weaknesses
README says beta; some advanced features may change
Broad scope means many dependencies (Transformers, TRL, vLLM, verl) to keep aligned
GPU extras needed for most real training; hardware requirements not stated
Dedicated endpoint deploy targets specific providers (Fireworks, Parasail)
Ray and vLLM based RLHF training framework for LLMs and VLMs
10.1k starsApache-2.0last commit Sep 2026
OpenRLHF trains language and vision-language models with reinforcement learning from human feedback. It uses Ray to schedule actor, reward, reference and critic models across GPUs, vLLM for sample generation, and DeepSpeed ZeRO-3 for training. It implements PPO, REINFORCE++, GRPO and RLOO, plus SFT, DPO and reward model training, with single-turn or multi-turn agent rollouts.
Strengths
Several RL algorithms (PPO, REINFORCE++, GRPO, RLOO) selectable via one estimator flag
MiniMind is a from-scratch implementation of a small Qwen3-style decoder-only LLM (64M dense, 198M-A64M MoE) with each training stage written in native PyTorch: pretrain, SFT, LoRA, DPO, PPO/GRPO/CISPO, agentic RL and distillation. It ships datasets, a 6,400-token tokenizer, an OpenAI-compatible server and a Streamlit chat UI. The README states one SFT epoch takes about 2 hours on a single 3090.
Strengths
Core algorithms (LoRA, DPO, PPO/GRPO) written in plain PyTorch, not wrapped from trl or peft
Open datasets for each stage, with mini variants (1.2GB pretrain, 1.6GB SFT) for quick runs
Weights convert to transformers, llama.cpp, vllm and ollama; ollama and vllm usage is documented
Checkpoint resume, DDP and DeepSpeed multi-GPU, wandb or swanlab logging
Weaknesses
Models are tiny (64M), so output quality is far below mainstream LLMs
README is primarily in Chinese; the English version is a separate file
Training data is largely synthetic and distilled, with mixed source licenses
Native MoE training is about 50% slower than dense, per the README
GPU optional
Needs PyTorch, transformers, streamlit (optional WebUI), swanlab or wandb (optional)
Models: minimind-3 (64M), minimind-3-moe (198M-A64M), minimind2 series, Transformers format, llama.cpp
Fine-tunes a Qwen VL model on your Telegram chats to build a chat avatar
18.3k starsAGPL-3.0last commit Oct 2026
WeClone exports Telegram chat history, strips personal data with Microsoft Presidio and a blocklist, then LoRA fine-tunes Qwen2.5-VL-7B-Instruct through LLaMA Factory. The tuned model is served as an OpenAI-style API (README example port 8005) and can be connected through AstrBot or LangBot to Telegram, Discord or Slack. Personal WeChat deployment is listed, based on openclaw-weixin.
Strengths
Covers the full pipeline: export, PII filtering, LoRA training, API serving, bot deployment
Presidio redaction plus a manual blocked_words list; encrypted storage default in new templates
Handles image data; other models usable through LLaMA Factory
QLoRA 4-bit on a 7B model needs about 6GB VRAM per the README table
Weaknesses
Only Telegram works as a data source; WhatsApp, Discord and Slack imports are marked planned
README says 7B results are average; 14B or larger works better
Fine-tuned model has no tool calling; AstrBot tools must be turned off
Requires CUDA 12.6+; Windows not rigorously tested (WSL suggested)
GPU required
Needs CUDA 12.6+, LLaMA Factory, Microsoft Presidio, uv, AstrBot or LangBot (for chat bots)
YAML-configured training and fine-tuning for LLMs, multimodal and tabular models
11.8k starsApache-2.0last commit Sep 2026
Ludwig trains, fine-tunes and serves models from a YAML config and a CLI, with no training loop code. It covers LLM fine-tuning (SFT, DPO, KTO, ORPO, GRPO, LoRA and other PEFT adapters, 4-bit QLoRA, torchao QAT), plus multimodal, tabular, timeseries and image models. Runs scale from local to multi-GPU and Ray clusters by changing the backend setting.
Strengths
Whole experiment defined in one YAML file; ludwig train, predict and serve cover the workflow
RL post-training framework connecting Megatron training with SGLang rollout
8.7k starsApache-2.0last commit Oct 2026
slime is a reinforcement-learning post-training framework for LLMs. It runs Megatron for training and SGLang for rollout generation, coordinated by Ray and a shared Data Buffer. Custom generate functions, reward functions, verifiers and environments plug in through documented interfaces. The README says it is the RL framework behind the GLM-4.5 through GLM-5.3 releases.
Strengths
Used for RL training of GLM-4.5 through GLM-5.3, per the README
Megatron args pass through directly; SGLang args use the --sglang- prefix
Custom generate and reward hooks cover multi-turn, tool-use and agentic rollouts
Triton kernels that cut memory and raise throughput for LLM training
6.7k starsBSD-2-Clauselast commit Oct 2026
Liger Kernel is a PyTorch library of Triton kernels for RMSNorm, RoPE, SwiGLU, CrossEntropy and FusedLinearCrossEntropy, plus chunked post-training losses such as DPO, ORPO, SimPO, KTO and JSD. A one-line patch swaps them into Hugging Face models. The README reports about 20% higher multi-GPU throughput and 60% lower memory on LLaMA 3-8B, with exact (non-approximated) computation.
Strengths
One-line patching of Hugging Face models; also usable as standalone modules
Works with FSDP, DeepSpeed and DDP, and integrates with Axolotl, LLaMA-Factory and TRL
Exact computation, checked by unit and convergence tests against runs without Liger
Chunked post-training losses (DPO, ORPO, SimPO, KTO) claim up to 80% memory savings
Weaknesses
Benchmarks come from one setup: LLaMA 3-8B on 8 A100s with FSDP1
Model patching APIs need transformers >= 5.0.0, and per-model support varies
cuTile and CuTe DSL backends are opt-in and CUDA-only, with newer-GPU targets (Hopper, Blackwell)
Training-side only; it does not serve or run inference
Asynchronous reinforcement learning system for training reasoning and agentic LLMs
5.9k starsApache-2.0last commit Oct 2026
AReaL trains LLMs with reinforcement learning using a fully asynchronous rollout and training loop. It ships GRPO, PPO, DAPO, GSPO, DPO, SFT and related algorithms, with SGLang or vLLM for inference and Megatron or PyTorch FSDP for training. Agentic RL works with black-box agents by pointing them at an AReaL endpoint through base_url.
Strengths
Asynchronous RL; setting max_head_offpolicyness=0 gives the synchronous variant
Many algorithms with example configs: GRPO, GSPO, PPO, DAPO, RLOO, M2PO, DPO, SFT
Agentic RL for existing agents by swapping base_url; examples for SWE, search and tau2
Backends: SGLang or vLLM inference; Megatron, FSDP or Archon training; Ray and local schedulers
Weaknesses
Needs CUDA GPUs; install requires a matching flash-attn wheel
Model support varies by training backend; other Hugging Face LLMs only on FSDP
2.0 microservice refactor and v1/v2 options add configuration complexity
Ascend NPU support lives on a separate branch
GPU required
Docker
Needs SGLang or vLLM, Megatron or PyTorch FSDP, flash-attn, Ray (optional)
Colossal-AI is a PyTorch-based library for distributed training and inference of large models. It provides data, pipeline, 1D/2D/2.5D/3D tensor and sequence parallelism, ZeRO, auto-parallelism and heterogeneous memory management. The repo also hosts applications such as Colossal-LLaMA-2, ColossalChat (RLHF) and Stable Diffusion acceleration examples.
Strengths
Covers data, pipeline, tensor (1D to 3D), sequence parallelism and ZeRO in one library
Includes worked examples for LLaMA, GPT-2/3, BERT, ViT, MoE and Stable Diffusion
Publishes H200 and B200 benchmarks for 7B and 70B Llama-like training with parallel configs
Train game agents with reinforcement learning inside Unity environments
19.8k starscustom licenselast commit Oct 2026
Unity SDK plus a PyTorch-based Python trainer that turns Unity scenes into reinforcement learning environments. It ships PPO, SAC, MA-POCA and self-play, plus BC and GAIL imitation learning, and exposes environments through a Python API, Gym and PettingZoo wrappers. Trained policies run in-game through Unity's Inference Engine.
Strengths
Ships PPO, SAC, MA-POCA, self-play, BC and GAIL trainers out of the box
Gym and PettingZoo wrappers plus a low-level Python API for custom training code
17+ example Unity environments and curriculum learning and environment randomization support
Trained models run natively in Unity across platforms via the Inference Engine
Weaknesses
Requires the Unity editor and a Unity project; not usable standalone
Last tagged release is from 2025; develop branch is described as unstable
Web docs are deprecated; docs live in the Unity package documentation
License shown as NOASSERTION by detection, while the README badge says Apache-2.0
PyTorch-native training stack for pretraining, fine-tuning and RL of large models
5.8k starsBSD-3-Clauselast commit Oct 2026
torchtitan is a PyTorch codebase for training large generative models, covering pretraining, SFT/LoRA and RL (TitanRL). It composes FSDP2, tensor, pipeline, context and expert parallelism through DeviceMesh and DTensor, and runs are defined as Python recipe functions. It supports Llama 3, Qwen3, DeepSeek V3, gpt-oss, Kimi and FLUX, with MXFP8 and NVFP4 low-precision training.
Strengths
Stacks FSDP2, TP, PP, CP and EP into N-D layouts without rewriting the model
Config is a Python recipe function, so runs are readable and copyable
TitanRL shares model definitions and kernels with vLLM, with a bitwise-parity mode
Reads and writes Hugging Face safetensors; async distributed checkpointing
Weaknesses
Needs a recent PyTorch nightly; stable releases pin nightly torch and torchao
Verified recipes exist for only some models; others have CI debug configs only
Low-precision and RL features need extra nightly installs (torchao, vLLM, Monarch)
Aimed at multi-GPU clusters; README shows only a debug model on one GPU
Minimal PyTorch codebase to train and chat with a small GPT-2-class LLM
58.9k starsMITlast commit Jul 2026
nanochat covers tokenizer training, pretraining, SFT, RL, evaluation and a CLI chat interface in one small Python codebase. A single `--depth` setting derives width, heads, learning rate and training horizon, so one knob produces a compute-optimal model series. The reference speedrun trains a GPT-2-grade model in about 1.65 hours on an 8xH100 node per the leaderboard.
Strengths
Covers tokenizer, pretraining, SFT, RL, eval and inference in one repo
Single `--depth` dial sets all other hyperparameters automatically
Public time-to-GPT-2 leaderboard with commits and CORE scores
Same scripts run on one GPU via gradient accumulation, or on CPU/MPS
Weaknesses
Reference speedrun needs an 8xH100 node, roughly $48 of rented compute
Under 80GB VRAM requires manually lowering `--device-batch-size`
CPU/MPS path only trains a tiny model with weak results
Chat is CLI-only in the README; resulting model is small and hallucinates
Merge pre-trained language models in weight space from YAML configs
7.4k starsLGPL-3.0last commit Oct 2026
mergekit combines pre-trained language models by operating on their weights, driven by a YAML config and the `mergekit-yaml` command. Tensors load lazily, so merges run on CPU or with as little as 8 GB of VRAM. It supports many methods (linear, SLERP, TIES, DARE, DELLA, SCE, Model Stock), plus layer-wise frankenmerges, MoE merging and LoRA extraction.
Strengths
Lazy tensor loading: merges run on CPU or with as little as 8 GB VRAM
Many methods: linear, SLERP, TIES, DARE, DELLA, SCE, Model Stock and others
Also covers MoE merging, LoRA extraction, evolutionary merging and tokenizer transplantation
Declarative YAML configs with per-layer and per-tensor parameter gradients
Weaknesses
Command-line only; no bundled UI
No Docker image or compose file provided
Install is from a git clone with pip install -e; no PyPI install documented
Models with different vocabularies need manual tokenizer configuration
GPU optional
Needs PyTorch, Hugging Face Hub (optional, for upload)
Models: Llama, Mistral, GPT-NeoX, StableLM, raw PyTorch models
PyTorch LLM implementations with recipes to pretrain, finetune, evaluate and serve
13.7k starsApache-2.0last commit Sep 2026
LitGPT reimplements 20+ open LLM families (Llama, Gemma, Phi, Qwen, Mistral and others) from scratch in PyTorch, without abstraction layers. A single CLI covers download, chat, finetune (LoRA, QLoRA, adapter), pretrain, continued pretraining, evaluate and serve, plus a Python `LLM` API. Training supports FSDP, flash attention and reduced precision (fp4/8/16/32) across 1 to 1000+ GPUs or TPUs.
Strengths
Single CLI covers download, finetune, pretrain, evaluate, chat and serve
Readable single-file model implementations, easy to debug and modify
Supports LoRA, QLoRA, adapter finetuning and full pretraining from scratch
Apache-2.0 license with YAML training recipes
Weaknesses
Built-in `litgpt serve` exposes a simple /predict endpoint, not documented as OpenAI-compatible
Only the listed model families are supported; new architectures need code added
No Docker or compose files in the repo
README promotes Lightning Cloud for GPUs; hardware requirements per model not stated
Self-configuring U-Net framework for biomedical image segmentation
9k starsApache-2.0last commit Oct 2026
nnU-Net analyzes a training dataset, builds a fingerprint, and configures 2D and 3D U-Net variants with matching preprocessing, training, model selection and inference. It targets supervised semantic segmentation, mainly biomedical images, and doubles as a baseline for segmentation research. Installed with pip install nnunetv2 on top of PyTorch.
Strengths
Derives preprocessing and network configuration from a dataset fingerprint, with little manual tuning
Covers the full workflow: preprocessing, training, model selection and inference
Supports 2D and 3D data, arbitrary channel definitions and multiple image formats
Documented residual encoder presets and finetuning from nnssl checkpoints
Weaknesses
Supervised semantic segmentation only; not a general vision or LLM toolkit
Requires setting up nnUNet_raw, nnUNet_preprocessed and nnUNet_results paths
Last tagged release is 2024-04-21, though commits continue
README does not state RAM or VRAM requirements
Needs PyTorch
Models: U-Net (2D, 3D fullres, cascade), Residual encoder U-Net presets
Training scripts for medical LLMs: pretraining, SFT, RLHF, DPO, GRPO
5.9k starsApache-2.0last commit Sep 2026
MedicalGPT is a set of Python scripts that train domain LLMs through continued pretraining, supervised fine-tuning, reward modeling with PPO/RLOO, DPO, ORPO, GRPO and on-policy distillation. It supports Qwen, Llama-3, Mixtral and other model families, with LoRA/QLoRA and DeepSpeed ZeRO configs. Gradio, FastAPI and OpenAI-style demo servers are included for inference.
Strengths
Covers PT, SFT, RM, PPO, DPO, ORPO, GRPO and OPD in one repo with shell launchers
Ships DeepSpeed ZeRO-1/2/3 configs and a VRAM table (QLoRA 4-bit 7B about 6GB)
Supports tool-call fine-tuning via SFT and DPO using ShareGPT-format jsonl
Colab notebooks run full PT+SFT+DPO or PPO pipelines in roughly 15-20 minutes
Weaknesses
Script collection, not a packaged library or UI; you edit shell scripts and arguments
Stages are chained by hand, merging LoRA adapters between each one
README is mostly Chinese; the English version is a separate file
No Docker or compose files; setup is pip install -r requirements.txt
GPU required
Needs PyTorch, Hugging Face Transformers, PEFT, DeepSpeed
Turns documents into LLM fine-tuning and evaluation datasets
15k starscustom licenselast commit May 2026
Easy Dataset parses PDF, Markdown, DOCX, TXT and EPUB files, splits them into chunks, and uses an LLM to generate questions, answers and chain-of-thought for single-turn, multi-turn or image QA datasets. It also builds evaluation sets with a judge model and a double-blind arena, and exports to Alpaca or ShareGPT, LLaMA Factory configs, or Hugging Face. It runs as a web app on port 1717 or as a desktop client.
Strengths
Covers raw documents to Alpaca/ShareGPT export in JSON or JSONL
Generates evaluation sets with judge-model scoring and a human blind test arena
Works with any OpenAI-format API, plus Ollama for local models
Generates LLaMA Factory configs and uploads to Hugging Face Hub
RL training framework for language agents, on any harness or sandbox
5.9k starsApache-2.0last commit Oct 2026
rLLM trains language agents with reinforcement learning (GRPO, REINFORCE, RLOO, SFT, on-policy distillation). A model gateway captures token IDs and logprobs from your existing agent code, so the same agent runs for eval and training. Backends are verl, tinker and fireworks, chosen with one flag, and 60+ benchmarks run through `rllm eval`.
Strengths
Same agent code for eval and training; gateway captures token IDs and logprobs
Switches between verl, tinker and fireworks backends with one flag
60+ benchmarks, including SWE-bench and Terminal-Bench 2.0, via `rllm eval`
Wraps LangGraph, OpenAI Agents SDK or plain OpenAI client code with @rllm.rollout
Weaknesses
Default install targets the tinker backend; multi-GPU needs the verl extra
README installs from a git URL, not from PyPI
Hardware requirements for training are not stated
Training recipes and results live mostly in external blog posts
GPU optional
Docker
Needs Python >= 3.11, verl with vLLM or SGLang (multi-GPU), Tinker API, Fireworks, Docker, Daytona or Modal (sandboxes)
Toolbox for full, LoRA and LISA finetuning of large language models
8.5k starsApache-2.0last commit Aug 2026
LMFlow is a Python toolbox that finetunes decoder language models with full-parameter training, LoRA/QLoRA or LISA, driven by shell scripts and Accelerate. It adds conversation templates, optional DeepSpeed, FlashAttention-2, vLLM or SGLang inference, DPO via TRL, and a Gradio chatbot demo. The README lists estimated memory per model size, for example 16GB for 7B with LoRA.
Strengths
Covers full, LoRA, QLoRA and LISA finetuning from ready-made shell scripts
README gives memory estimates per method, from 0.5B to 70B models
Optional extras for DeepSpeed, FlashAttention, vLLM, SGLang and TRL-based DPO
Preset conversation templates such as llama3, llama2, phi3 and chatml
Weaknesses
Tested only on Ubuntu 20.04; macOS and Windows may hit unexpected errors
vLLM and SGLang need incompatible CUDA/PyTorch versions, so separate envs are required
LISA script is only tested on a single GPU
Last release is 2025-07-11; no Docker image or compose file
GPU required
Needs Python 3.9+, conda, mpi4py, PyTorch, WandB (optional, on by default)
Models: Hugging Face decoder models, LLaMA, Llama-2, Llama-3, Qwen
Agent loop that edits a small GPT training script and keeps changes that lower val_bpb
98.5k starsno licenselast commit Mar 2026
A coding agent (Claude, Codex or similar) is pointed at a single-GPU nanochat-derived training setup and edits train.py in a loop. Each experiment trains for a fixed 5 minutes and is kept or discarded based on validation bits per byte (val_bpb). The human edits program.md, the agent instructions, rather than the Python code.
Strengths
Fixed 5-minute budget makes runs comparable: about 12 experiments per hour, around 100 overnight
Small codebase: agent edits only train.py, so diffs stay reviewable
val_bpb is vocab-size independent, so architecture changes compare fairly
Works with any coding agent that can read program.md
Weaknesses
Requires a single NVIDIA GPU (tested on H100); no CPU, MPS or AMD support in main repo
Results are not comparable across different hardware because of the fixed time budget
Default program.md is a bare baseline for one agent; multi-agent setups are left to the user
README says to disable agent permissions, which is a safety risk without extra sandboxing
GPU required
Needs uv, PyTorch, Python 3.10+, a coding agent (Claude, Codex or similar)
Models: GPT (nanochat-style, trained from scratch), Muon + AdamW optimizer
EleutherAI's Megatron/DeepSpeed library for pretraining large language models
7.5k starsApache-2.0last commit Oct 2026
GPT-NeoX is a training library for autoregressive language models with billions of parameters, built on NVIDIA Megatron and DeepSpeed with ZeRO and 3D parallelism. It launches through Slurm, MPI, pdsh or the IBM Job Step Manager and runs on NVIDIA and AMD GPUs. The README says it targets from-scratch large-scale training and recommends Hugging Face transformers for generic inference.
Strengths
Used by 40+ organizations to train 50+ models, including Pythia and StableLM
ZeRO and 3D parallelism, flash attention, Transformer Engine, rotary and ALiBi embeddings
Preset configs for Pythia, PaLM, Falcon and LLaMA 1 and 2
Exports to Hugging Face; logs to WandB, Comet or TensorBoard
Weaknesses
Not meant for small-scale training or generic inference, per the README
Depends on DeeperSpeed, a DeepSpeed fork; environment isolation strongly advised
Tested on Python 3.8-3.10 and PyTorch 1.8-2.0; other versions not guaranteed
Last tagged release is from March 2023
GPU required
Docker + Compose
Needs PyTorch, DeeperSpeed, Slurm, MPI or pdsh for multi-node, Flash Attention (optional), Transformer Engine (optional), WandB, Comet or TensorBoard (optional)
Models: GPT-NeoX, Pythia, LLaMA 1 and 2, Falcon, PaLM
LLM training, fine-tuning, compression and inference toolkit on PaddlePaddle
13k starsApache-2.0last commit May 2026
PaddleNLP is a Python toolkit built on PaddlePaddle for pretraining, fine-tuning, quantizing and serving LLMs. It supports 4D parallel training, Unified Checkpoint for elastic resume, FlashMask-based fine-tuning, and inference with FP8, INT8 and 4-bit quantization. Hardware targets include NVIDIA GPU, Kunlun XPU, Ascend NPU, Enflame GCU and Hygon DCU.
Strengths
Runs on NVIDIA GPU plus Kunlun XPU, Ascend NPU, Enflame GCU and Hygon DCU
Trainer configures data, sharded, tensor and pipeline parallelism through settings
Training and evaluation code for speculative-decoding draft models
7.2k starsMITlast commit Jul 2026
DeepSpec is a Python codebase for training and evaluating draft models used in speculative decoding. It covers data preparation (prompt download, answer regeneration with the target model, target cache), a training script that spawns one worker per GPU, and an eval script measuring acceptance on benchmarks such as gsm8k, math500 and humaneval. It implements DSpark, DFlash and Eagle3 and links released checkpoints on Hugging Face for Qwen3 and Gemma 4 targets.
Strengths
Implements three draft algorithms (DSpark, DFlash, Eagle3) in one pipeline with per-model configs
Released checkpoints on Hugging Face for Qwen3 4B/8B/14B and Gemma 4 12B
Eval covers nine benchmarks, including gsm8k, humaneval, mt-bench and arena-hard-v2
MIT license
Weaknesses
Target cache for the default Qwen3-4B setup needs roughly 38 TB of storage
Default configs assume a single node with 8 GPUs
Data preparation needs a separately served inference engine for the target model
No release tags; only four target models are covered by released checkpoints
GPU required
Needs Inference engine to serve the target model (data prep), Hugging Face models and datasets
The t5 library is the code behind the T5 paper. It defines text-to-text Tasks and Mixtures on tf.data, evaluates them with built-in metrics, and trains or fine-tunes released checkpoints through Mesh TensorFlow (TPU or GPU) or an experimental Hugging Face PyTorch shim. The README says it is no longer actively developed and recommends T5X (JAX/Flax) for new work.
Strengths
Task and Mixture classes bundle data, preprocessors, tokenizer and metrics for multi-task training
Operative configs for the paper's experiments are published in gs://t5-data/experiments
Fine-tunes released T5 checkpoints from the command line via t5_mesh_transformer and gin configs
Exports a SavedModel for serving; Colab tutorial runs on a free TPU
Weaknesses
TensorFlow/MeshTF version is no longer actively developed; README points to T5X
Hugging Face PyTorch shim is experimental, single-GPU, and subject to change
Workflow assumes GCP TPUs and GCS buckets; GPU use is a secondary path
Building C4 needs about 7 TB download and 335 CPU-days of preprocessing
GPU optional
Needs TensorFlow, Mesh TensorFlow, TensorFlow Datasets, SentencePiece, gin-config, Google Cloud TPU and GCS (for TPU use)
Python library for programmatic labeling of machine learning training data
6k starsApache-2.0last commit Apr 2026
Snorkel is a Python library for building and managing training data programmatically, using weak supervision instead of hand labeling. It started at Stanford in 2015 and installs with pip or conda. The team now focuses on the commercial Snorkel Flow platform, and the last release was 2024-02-27.
Strengths
Installs with pip or conda; requires Python 3.11 or later
Apache-2.0 licensed, with docs and a separate tutorials repository
Backed by over sixty peer-reviewed publications on weak supervision
Tutorials cover several tasks, domains and labeling techniques
Weaknesses
Team focus has moved to proprietary Snorkel Flow, not this library
Last release was 2024-02-27
README does not mention LLM or foundation model workflows
Limited Windows testing; Docker or WSL recommended
PyTorch library of LLM fine-tuning recipes driven by YAML configs
5.8k starsBSD-3-Clauselast commit Apr 2026
torchtune provides PyTorch-native recipes for SFT (full, LoRA/QLoRA), knowledge distillation, DPO, PPO, GRPO and quantization-aware training, run through a `tune` CLI and YAML configs. It includes model implementations for Llama, Gemma, Mistral, Phi and Qwen families. The README states development wound down in 2025 and the library is no longer actively maintained.
Strengths
Covers SFT, KD, DPO, PPO, GRPO and QAT recipes; multi-GPU and multi-node support varies
README gives measured memory figures, e.g. Llama 3.1 8B QLoRA at 7.4 GiB peak on one 4090
YAML configs with CLI overrides, plus `tune cp` to copy and edit configs locally
Runs on NVIDIA, Intel XPU, AMD ROCm, Apple MPS and Ascend NPU devices
Weaknesses
No longer actively maintained; development wound down in 2025
Last release was 2025-04-07; newest models listed are Qwen3 and Llama 4
Tested only against latest stable PyTorch (2.6.0 in README) and nightlies
Many recipe gaps, e.g. no full-weight KD and no LoRA for PPO or GRPO
GPU required
Needs PyTorch, torchvision, torchao, Hugging Face Hub