Reinforcement learning, inference and multimodal LLMs.
Publications all papers →
-
Chenduo Hao, Chuanbao Gao, Pinjun Zeng, Jingze Zhu, Chonghan Liu, Zidong Liu, Xu Yang
-
Ziyi Wang, Li Li, Aolin Zhou, Yankun Shen, Chonghan Liu, Shuxia Lin, Xu Yang
-
Rethinking LLM Ensembling from the Perspective of Mixture Models spotlight ICML 2026
Jiale Fu, Yuchu Jiang, Chonghan Liu, Joey Tianyi Zhou, Xu Yang
-
Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models poster ACL Main 2026
Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu, Che Liu
-
d²Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching poster ICLR 2026
Yuchu Jiang, Yue Cai, Xiangzhong Luo, Jiale Fu, Jiarui Wang, Chonghan Liu, Xu Yang
-
Jiaming Fan, Cao Daming, Xiangzhong Luo, Jiale Fu, Chonghan Liu, Xu Yang
-
Yuliang Liu, Junjie Lu, Zhaoling Chen, Chaofeng Qu, Jason Klein Liu, Chonghan Liu, Zefan Cai, Yunhui Xia, Li Zhao, Jiang Bian, Chuheng Zhang, Wei Shen, Zhouhan Lin
-
Hao-Run Cai, Si-Yang Liu, Zi-Jian Cheng, Kun-Yang Yu, Jin-Hao Sheng, Guo Yu, Chonghan Liu, Zhi Zhou, Jun-Peng Jiang, Lan-Zhe Guo, Han-Jia Ye
-
Ming Ma, Yi Zhu, Yiran Zhong, Feida Zhu, Chonghan Liu, Pengkun Jiao, Qichao Wang, Yanhao Jia, Tianming Yang, Steven Hoi
-
Bin Chen, Xinye Liao, Yiming Liu, Xin Liao, Chonghan Liu
-
Chonghan Liu, Yimin Du, Qi An, Xin He, Cunqi Zhai, Fei Tan, Weijia Lin, Xiaochun Gong, Yongchao Deng, Shousheng Jia, et al.
-
Modality Gap-Driven Subspace Alignment Training Paradigm for Multimodal Large Language Models arXiv 2026
Xiaomin Yu, Yi Xin, Yuhui Zhang, Wenjie Zhang, Chonghan Liu, Hanzhen Zhao, Chen Liu, Xiaoxing Hu, Ziyue Qiao, Hao Tang, Xiaobin Hu, Chengwei Qin, Hui Xiong, Yu Qiao, Shuicheng Yan
-
Zhiming Ma, Shiyu Gan, Junhao Zhao, Xianming Li, Qingyun Pan, Peidong Wang, Mingjun Pan, Yuhao Mo, Jiajie Cheng, Chengxin Chen, Zhonglun Cao, Chonghan Liu, Shi Cheng
-
Chonghan Liu, Haoran Wang, Frank Henry, Peng Miao, Yifan Zhang, Yue Zhao, Peiran Wu
Open Source all PRs →
-
NVIDIA-NeMo/Automodel PyTorch-native LLM/VLM training framework
- Speculative decoding (EAGLE / DFlash / DSpark / Domino / JetSpec / MSD): grew this from a single Llama EAGLE-1 recipe into Automodel's full draft-model training stack: P-EAGLE parallel drafting, DFlash, DeepSeek V4 DSpark, Domino online training, and JetSpec causal parallel drafting, plus a vLLM/SGLang serving bridge and an end-to-end tutorial. Extended it to Kimi K3 EAGLE-3, DFlash, and DSpark drafts, Gemma4 targets, and vision-language models through a multimodal speculative decoding (MSD) core and ViSpec VLM draft training. Most recently added EAGLE-4 two-stage draft training and export, a remote target backend so DSpark trains against a frozen target running on other GPUs or another node, and a fix for Gemma 4's released drafter diverging under joint fine-tuning.
- Scaling & evaluation: added sequence packing across EAGLE-1/2, DFlash, DSpark, Domino, and JetSpec, later DeepSeek V4 and V4.1 DSpark; Ulysses context parallelism for the EAGLE-3 draft; FP8 draft training and LoRA adaptation; distributed offline feature precomputation; on-policy target-logit regeneration; and simulated and real acceptance-length evaluation with multi-dataset benchmark sweeps. A shared draft-recipe base class that folds the checkpoint, schedule, and logging lifecycle of the four draft recipes into one is under review.
- Model & MoE support: added training support for DeepSeek V4 Flash (plus Multi-Token Prediction) and DeepSeek V4.1 Flash (its DSpark draft now trains on the released chat template), Hy3-preview, and Hy-MT2-30B-A3B; brought Kimi Linear to context parallelism with packed sequences and tensor and sequence parallelism; plus Rollout Routing Replay (R3) for MoE RL training.
- Distributed & VLM training: TP+PP and Context Parallelism for Gemma4 VLM, TransformerEngine attention injection into HF models, and fixes for NaN loss, activation checkpointing, multi-image token expansion, and mRoPE under packing; added THD packed-sequence support for Qwen3-VL-MoE and base-model mRoPE resolution for packed VLM training; model-owned context parallelism for Gemma4 Unified is under review.
- Checkpointing & training infra: deferred distributed async checkpoint consolidation, DCP CPU offload, and a checkpoint staging wait option; per-parameter-group learning rates, parallel pre-tokenization before packing, and a token-budget dynamic batch sampler (under review); a SwanLab mirror of the W&B run for regions where W&B is unreachable.
- Diffusion models: HunyuanImage-3.0 text-to-image fine-tuning, an 80B-total / 13B-active MoE trained with FSDP2 and expert parallelism, as LoRA or full fine-tuning, with preprocessing for the release's own VAE and prompt format.
- Knowledge distillation: VLM KD recipe with chunked loss and teacher-model offload.
- Agent / tool-calling SFT: multi-turn dataset adapter, turn-aware history truncation, reasoning-content masking, tool-call accuracy evaluator, SynthTraces agent SFT, prompt masking through mask_generation_prompt, chat_template_kwargs for ChatDataset and DSpark (Qwen3's enable_thinking, DeepSeek's thinking_mode, set from the config), and an SFT tutorial.
-
NVIDIA-NeMo/Megatron-Bridge bridge between HF checkpoints and Megatron-Core training
- Model bridges: added a MiniMax M3 language-model bridge and recipes, and fixed router expert_bias mapping across the DeepSeek family and the wrong GELU variant in the Gemma-1 bridge.
- Training & data correctness: fixed a training_log argument-misbinding bug, stale non-persistent checkpoints surviving most_recent_k=0, an Energon test split silently aliasing validation, a MegatronMIMO padding mask keyed on token value instead of position, and a missing pipeline-parallelism guard in QwenVLInferenceWrapper.
- Speculative decoding: brought DSpark draft training to Megatron-Core with a design doc and the draft heads and loss, then excluded dropped anchor blocks from the DSpark loss and metrics.
-
NVIDIA-NeMo/Gym environments for evaluating and improving models and agents
- Added an OpenAI-compatible /v1/embeddings endpoint to the model server, declared as a non-generating route, so memory- and retrieval-based agents can send embedding calls through Gym instead of a provider directly.
-
NVIDIA-NeMo/RL scalable toolkit for model reinforcement learning
- Opened fixes for the VAPO positive-example NLL term, which normalized by the microbatch instead of the global token count and never fired because rewards were not in the GRPO train batch, plus total_reward being corrupted across dynamic-sampling rounds.
-
NVIDIA-NeMo/labs-molt agentic-first, HuggingFace-native RL framework for research
-
verl-project/verl RLHF training framework
-
verl-project/verl-SpeCo speculative-decoding drafter-training overlay for verl
- Drafter backends: ported Automodel's draft-model training into the verl RL stack as pluggable backends: EAGLE-1 / EAGLE-2, Domino, P-EAGLE parallel drafting, and DFlash2, with DFlash2 co-training against SGLang rollout and standalone offline draft training.
- Fixes: drafter gradients scaled down by world size, P-EAGLE publishing the wrong head and embedding, and Domino curriculum restarts on resume.
- Hardening: made P-EAGLE train under FSDP2, added an opt-in bubble-time profiler for online draft training, and an audit batch fixing d2t defaults, reduced draft vocabularies without a mapping, t2d construction, packed document-length validation, SGLang hidden-state requests for every drafter, and unsupported vLLM architecture aliases for DFlash2.
-
vllm-project/speculators speculative-decoding draft training for vLLM
- Fixed an eager-path draft-token mask that inverted additive mask semantics and added a warning when a cached token-frequency distribution is silently reused; opened fixes for full-precision optimizer state in checkpoints, resumed LR schedulers seeded by the global step, the conditional-accuracy denominator, and CLI convert choices.
-
sgl-project/sglang-omni serving runtime for interactive, multimodal agent systems
- Native MLX C++ speech-to-text runtimes for Voxt: Whisper large-v3-turbo, Whisper large-v3 fp16, Whisper small fp16, and Cohere Transcribe 03-2026 with a Silero VAD v6 port for long-audio segmentation, each with a supervised server, parity CLI, and frozen-corpus checks.
-
hiyouga/LlamaFactory unified efficient fine-tuning framework for 100+ LLMs & VLMs
- Multimodal (VLM) training, as collaborator: fixing VLM training hangs under ZeRO-3/FSDP via dummy-image injection, lazy-loading multimodal inputs to cut preprocessing disk usage, Qwen2-VL mRoPE fixes, ViT gradient checkpointing, and VLM utility fixes; that work later moved to EasyR1.
- Data & examples: second ShareGPT conversation format support, a single-GPU full-parameter batch prediction example, and a TRL PPOv2 implementation reference.
-
chatchat-space/Langchain-Chatchat RAG & agent app framework over local LLMs
- Built the pluggable model-provider callback/validator architecture and its core provider/model manager, including Xinference integration.