Preconditioned Optimizers for MoE Training at scale, with out-of-the-box support for MuP and FSDP support for Muon, built on top of Megatron-LM and TransformerEngine.

An open attempt at reproducing a simplified version of Google's Genie World Model API on a single H100.

Nyxt - the hacker's browser.

OpenTTD is an open source simulation game based upon Transport Tycoon Deluxe

Go HTTP client with browser-identical TLS/HTTP2 fingerprinting. Bypass bot detection by perfectly mimicking Chrome, Firefox, and Safari at the cryptographic level (JA3/JA4, Akamai fingerprint, head…

Lightly-reviewed collection of community environments

beautiful text plots for terminals.

llm social deduction

Auto-distill your LLMs with synthetic data

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

A list of microgrant programs for your good ideas

Benchmarking tool for assessing LLM models' performance across different hardwares

An multi-column SQL inspired vector DB with implicit embeddings

learning rust from "the book"

SPAR Project repository: follow-up work to "Simple Synthetic Data Reduces Sycophancy in Large Language models" by Wei et al., 2023: https://arxiv.org/abs/2308.03958

Easily generate synthetic data for classification tasks using LLMs

snap number of files in half

irresponsible innovation. Try now at https://chat.dev/

notebooks for workshop on "understanding neural networks: from basics to reverse engineering them" at codeday lucknow

Identifying Circuit behind Pronoun Prediction in GPT-2 Small

mechanistic interpretability notebooks