Writing an LLM inference loop
A ground-up look at tokenization, logits, temperature, softmax, and next-token sampling.
I’m a Production Engineer at Meta in the Bay Area, and I write to make complex systems easier to understand.
Latest writing
All postsInference: Prefill and Decode
Why LLM inference separates prompt processing from token-by-token generation.
Learnings from Mankind’s Search for Meaning
Notes on purpose, hope, and what remains when circumstances strip almost everything else away.
File Descriptors: Part 2
The process, open-file, and inode tables that sit behind a Linux file descriptor.
File Descriptors: Part 1
An introduction to file descriptors and the interface they provide between a process and the kernel.