Out of Distribution . Notes from the tail
August 2026
Aug 13

Doom, compiled into a transformer

No training anywhere: a compiler sets the weights of a stock Phi-3-architecture checkpoint so autoregressive generation executes Doom's classic renderer.

Aug 09

A calculator, compiled into a transformer

Setting a transformer's weights by hand -- no training -- to build four exact calculators and compare how they use layers, parameters, and generated tokens.

July 2026
Jul 23

Introducing torchwright

A compiler that turns computation graphs into the weights of a stock transformer -- no training anywhere. Where it came from and how it works.

August 2018
Aug 02

What does a "difficult" MNIST digit look like?

The MNIST digits a simple convnet gets wrong, and what they suggest about how the state of the art compares to human performance.