Ongoing research training transformer language models at scale, including: BERT & GPT-2

A crude RLHF layer on top of nanoGPT with Gumbel-Softmax trick

Code for CIKM2020 "S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization"

Verifiers for LLM Reinforcement Learning

Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

MATE: the Multi-Agent Tracking Environment.

A collection of MuJoCo based environments.