This website serves as a living companion to the tutorial manuscript and to the tutorial presentation at ICML 2025. It dreams of being a one-stop shop for learning all things about Associative Memory. It’s definitely not there yet.

The website is a (growing) collection of notebook demos on Associative Memory. Each notebook is primarily a blog post on this site, but it is also fully runnable on colab and as a raw .ipynb file using the uv environment setup below.

• Dense binary storage, also distributed as colab notebook and raw .ipynb.
• Energy Transformer, also distributed as colab notebook and raw .ipynb.
• Diffusion as Memory, also distributed as colab notebook and raw .ipynb.
• Distributed Associative Memory, also distributed as colab notebook and raw .ipynb.

See the overview in tutorials for a bit more detail.

To add new examples, edit the source tutorial notebooks (as either .ipynb or plain text .qmd files) saved in nbs/tutorial/.

pip install amtutorial

We aim for simplicity and clarity in the notebooks. Thus, we migrate some helper functions (particularly around loading and processing data) to a pypi package called amtutorial to avoid cluttering the notebooks. An added benefit of this is that all dependencies needed to run these notebooks can be installed using pip install amtutorial.

pip install amtutorial

## Install torch to run the `diffusion as memory` notebook. CPU or CUDA versions work
# pip install torch --index-url https://download.pytorch.org/whl/cpu

## OPTIONAL: For rendering videos in notebooks, use ffmpeg. Can use conda to install as
#conda install conda-forge::ffmpeg conda-forge::openh264 

Then open up the .ipynb notebooks in tutorial_ipynbs/ in your favorite notebook editor, using the same env where you installed amtutorial.

Pre-requisites

• Install uv using curl -LsSf https://astral.sh/uv/install.sh | sh
• Install quarto
• We use conda (or better yet, mamba) for managing the ffmpeg dependency, which only matters if ffmpeg is not already installed on your system.

Setting up the environment

From the root of the repo:

uv sync
source .venv/bin/activate

# Expose venv to ipython
uv run ipython kernel install --user --env VIRTUAL_ENV $(pwd)/.venv --name=amtutorial 

## Install torch to run the `diffusion as memory` notebook. CPU or CUDA versions work
# pip install torch --index-url https://download.pytorch.org/whl/cpu

# OPTIONAL: For rendering videos in notebooks
conda install conda-forge::ffmpeg conda-forge::openh264 

Development pipelines

View a local version of the website with:

uv run nbdev_preview

Pushes to main deploy the website. The site will be live after a few minutes on github.

git checkout main

# Update the website. Takes a moment even with cached training runs
make deploy && git add . && git commit -m "Update site" && git push

Make a minor-patch update to the pypi package (preferably, only if amtutorials/src was updated):

make pypi && uv run nbdev_pypi

Useful scripts (for reference only)

uv run nbdev_preview                         # Preview website locally
bash scripts/prep_website_deploy.sh          # Sync dependencies, export qmd notebooks to ipynb for colab, and build website
bash scripts/export_qmd_as_ipynb.sh          # Export qmd notebooks to ipynb for colab
uv run python scripts/sync_dependencies.py   # Sync nbdev and pyproject.toml dependencies
uv run python scripts/prep_pypi.py           # Bump patch version and sync dependencies
uv run nbdev_pypi                            # Push to pypi