Parallel-trained transformers with recurrent inference through shifted memory passes.

Open implementation of Attention Residuals (Kimi Team, arXiv:2603.15031)

Official Implementation for Pre-print "T2MLR: Transformer with Temporal Middle-Layer Recurrence"

Experiments on the impact of depth in transformers and SSMs.

Official repository for Amortized Factor Inference Networks (AFINs).

Curated collection of research on the limitations of next-token prediction and methods that go beyond it.

Optimization benchmark for diffusion model training on dynamical systems

Fast Polar Decomposition for Muon

Code for the analyses in "The Supervision Horizon: An Exploration of the Mechanics of On-Policy Self-Distillation"

norm balancing optimizers

Code for "What really matters in matrix-whitening optimizers?"

Local coordinate frames

Code for the paper "Function-Space Learning Rates"

This notebook provides clean and minimal PyTorch implementations of classic pairwise preference models, including Bradley-Tery, Davidson, and Thurstone-Mosteller.

[ICLR 2025] MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts

AI-powered SDK for Software Development

A framework for steering MoE models by detecting and controlling behavior-linked experts.

REAP: Router-weighted Expert Activation Pruning for SMoE compression

(ICLR'26 + Netflix) Rank-GRPO: Training LLM-based Conversational Recommender Systems with Reinforcement Learning

Qwen3 MoE Router Statistics | 큐웬3 MoE

[ICLR'26] A scalable, effective, and general way to augment LLMs with billion-scale knowledge graphs using very little GPU memory cost.

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

An extension of the nanoGPT repository for training small MOE models.