Pinned Loading

1. lm-sampling-strategies lm-sampling-strategies Public

   Empirical comparison of four batch sampling strategies — random, shuffle, circular, and circular+shuffle — and their effect on convergence in a character-level transformer language model.

   Python

2. lm-research-pilot lm-research-pilot Public

   A CPU-friendly experimental pipeline for analyzing how linguistic structure forms during early transformer training.

   Python

3. pytorch-rope-embeddings pytorch-rope-embeddings Public

   A lightweight, pure PyTorch implementation of Rotary Positional Embedding (RoPE) and its variants (e.g., Llama 4's iRoPE scaling), designed for easy integration into custom transformer attention la…

   Python

4. llama4-moe-viz llama4-moe-viz Public

   An interactive, step-by-step 3D visualisation of the Mixture-of-Experts forward pass used in Llama 4.

   JavaScript

5. TInyTransformer TInyTransformer Public

   Transformer Implementations from scratch

   Jupyter Notebook

6. lasso lasso Public

   Studing the Llama 4 code

   Python 1