Scaling Laws for Linear Complexity Language Models

A 1D analogue of the MNIST dataset for measuring spatial biases and answering Science of Deep Learning questions.

A toolkit for scaling law research ⚖

Functional local implementations of main model parallelism approaches

A prize for finding tasks that cause large language models to show inverse scaling

Pen and paper exercises in machine learning

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

Code for "The Intrinsic Dimension of Images and Its Impact on Learning" - ICLR 2021 Spotlight https://openreview.net/forum?id=XJk19XzGq2J

PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)

PyTorch - FID calculation with proper image resizing and quantization steps [CVPR 2022]

Compute FID scores with PyTorch.

Release for Improved Denoising Diffusion Probabilistic Models

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

Learned Hyperparameter Optimizers

Denoising Diffusion Probabilistic Models

code for Scaling Laws for Language Transfer Learning

Repository for reproducing `Model-Based Robust Deep Learning`

PyTorch Implementation of OpenAI's Image GPT

An implementation of the BADGE batch active learning algorithm.

PyTorch extensions for high performance and large scale training.

Code from the article: "The Role of Disentanglement in Generalisation" (ICLR, 2021).

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Scaling scaling laws with board games.

Use python's argparse module in shell scripts

Repository for theory and methods for Out-of-Distribution (OoD) generalization