Official Repo of "Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving"

[ICML2026] Official Pytorch Implement for "Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models"

Official Implementation of "GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling"

[ICML 2025] LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning

Pretraining and inference code for a large-scale depth-recurrent language model

Official PyTorch implementation of CD-MOE

TODO has been accepted by ICLR 2025.

Official Pytorch Implementation of "Outlier-weighed Layerwise Sampling for LLM Fine-tuning" by Pengxiang Li, Lu Yin, Xiaowei Gao, Shiwei Liu

[ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, Souvik Kundu, Zhangyang Wang

Dynamic Sparsity Is Channel-Level Sparsity Learner [Neurips 2023]

Official Pytorch Implementation of "Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity"

[CVPR23] "Understanding and Improving Visual Prompting: A Label-Mapping Perspective" by Aochuan Chen, Yuguang Yao, Pin-Yu Chen, Yihua Zhang, and Sijia Liu

Boosting Driving Scene Understanding with Advanced Vision-Language Models

[NeurIPS 2020] "The Lottery Ticket Hypothesis for Pre-trained BERT Networks", Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, Michael Carbin

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Generate a simple shape dataset with different colors, shapes, thicknesses, and heights.

This is the official code for UGTs.

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

Git Re-Basin: Merging Models modulo Permutation Symmetries in PyTorch

Official Pytorch Implementation of “Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training”

Official Pytorch Implementation of “Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost”

Implementation of Dynamic ReLU on Pytorch