Open-source release accompanying Gao et al. 2025

Small language model built in Minecraft.

a small library for combinatorial iters of dicts, useful for config/hyperparameter sweep management

Evaluation of measurement tampering detection techniques on the datasets from Benchmarks for Detecting Measurement Tampering

A debugging and profiling tool that can trace and visualize python code execution

Keeping language models honest by directly eliciting knowledge encoded in their activations.

A simple and efficient tool to parallelize Pandas operations on all available CPUs

A dataset of alignment research and code to reproduce it

Starlark Language

Dataset of Canada goose images with annotations of bounding boxes with object classes, suitable for testing object detection algorithms.

Massively-Parallel Natural Extension of Reference Frame

Simple, elegant, Pythonic functional programming.

Model parallel transformers in JAX and Haiku

PyTorch package for the discrete VAE used for DALL·E.

A Python library for integrating model-based and judgmental forecasting

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

Open-AI's DALL-E for large scale training in mesh-tensorflow.

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

URL downloader supporting checkpointing and continuous checksumming.

Using queues, tqdm-multiprocess supports multiple worker processes, each with multiple tqdm progress bars, displaying them cleanly through the main process. It offers similar functionality for pyth…

A framework for few-shot evaluation of language models.

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

Download, parse, and filter data from Court Listener, part of the FreeLaw projects. Data-ready for The-Pile.

Universal markup converter

Babysit your preemptible TPUs

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.