Official repo for Sometimes I am a Tree

Official code repository for the paper: Learning Syntax Without Planting Trees: Understanding When and Why Transformers Generalize Hierarchically

Code for "Structure Here, Bias There: Hierarchical Generalization by Jointly Learning Syntactic Transformations" (SCiL 2021)

Fast and memory-efficient exact attention

Algorithms for byte-level language modelling

A platform for community discussion. Free, open, simple.

Training code for Baby-Llama, our submission to the strict-small track of the BabyLM challenge.

Universal Dependencies for Ancash Quechua

a simple repository with all the boilerplate for making and training an autoregressive transformer decoder model. exposes all the code necessary to allow easily modifying the architecture and compa…

nanoGPT-like codebase for LLM training

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.

Code and data associated with the AmbiEnt dataset in "We're Afraid Language Models Aren't Modeling Ambiguity" (Liu et al., 2023)

Nix Packages collection & NixOS

Source files for the SBL Greek New Testament.

arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

Bibtex Normalizer - Normalizing BibTeX entries to a common format

Unsupervised text tokenizer for Neural Network-based text generation.

Neural Machine Translation in PyTorch

Memory-efficient optimum einsum using opt_einsum planning and PyTorch kernels.

An icon font for academics

Turn pixel art pngs into svgs with optimizing options, retaining the original colors and the alpha channel.

Code for "Does syntax need to grow on trees? Sources of inductive bias in sequence to sequence networks"

🦀 Small exercises to get you used to reading and writing Rust code!

Neural Networks and the Chomsky Hierarchy

A PyTorch framework for creating, running, and reproducing experiments on seq2seq models.