Minimal reproduction of DeepSeek R1-Zero

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

Memory Mosaics are networks of associative memories working in concert to achieve a prediction task.

Vector (and Scalar) Quantization, in Pytorch

Schedule-Free Optimization in PyTorch

Reaching LLaMA2 Performance with 0.1M Dollars

The interactive graphing library for Python ✨

AlphaFold Meets Flow Matching for Generating Protein Ensembles

Modeling, training, eval, and inference code for OLMo

Locating and editing factual associations in GPT (NeurIPS 2022)

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

A library for mechanistic interpretability of GPT-style language models

Code for paper "Multiple Physics Pretraining for Physical Surrogate Models

A CPU+GPU Profiling library that provides access to timeline traces and hardware performance counters.

Fast Laplacian estimation

A modular embodied agent architecture and platform for building embodied agents

Hardware accelerated, batchable and differentiable optimizers in JAX.

VISSL is FAIR's library of extensible, modular and scalable components for SOTA Self-Supervised Learning with images.

JAX (Flax) implementation of algorithms for Deep Reinforcement Learning with continuous action spaces.

Library for Model Based RL

Code for the paper "On Energy-Based Models with Overparametrized Shallow Neural Networks"

Research language for array processing in the Haskell/ML family

Code for the paper: "Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss" (Chizat and Bach)