Starred repositories
A PyTorch library for implementing flow matching algorithms, featuring continuous and discrete flow matching implementations. It includes practical examples for both text and image modalities.
gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI
A JAX-based Differentiable Density Functional Theory Framework for Materials
verl: Volcano Engine Reinforcement Learning for LLMs
BigOBench assesses the capacity of Large Language Models (LLMs) to comprehend time-space computational complexity of input or generated code.
Everything about the SmolLM and SmolVLM family of models
LeanUniverse: A Library for Consistent and Scalable Lean4 Dataset Management
Stackfish is an open-source LLM-powered pipeline designed to automatically solve competitive programming problems.
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。
StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024
What would you do with 1000 H100s...
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
how to optimize some algorithm in cuda.
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
PyTorch extensions for high performance and large scale training.
Ongoing research training transformer models at scale
Type annotations and runtime checking for shape and dtype of JAX/NumPy/PyTorch/etc. arrays. https://docs.kidger.site/jaxtyping/
JAX (Flax) implementation of algorithms for Deep Reinforcement Learning with continuous action spaces.