Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs

Official Implementation of ACL 2021 paper “GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning”.

The official evaluation suite and dynamic data release for MixEval.

RewardBench: the first evaluation tool for reward models.

What would you do with 1000 H100s...

Puzzles for learning Triton

18.06 course at MIT

Robust recipes to align language models with human and AI preferences

Scalable training for dense retrieval models.

Python pdb for multiple processes

Machine Learning Engineering Open Book

Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama mode…

Train transformer language models with reinforcement learning.

The simplest, fastest repository for training/finetuning medium-sized GPTs.

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Expanding natural instructions

Ongoing research training transformer models at scale

Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.

Reference implementation for DPO (Direct Preference Optimization)

Supercharge Your LLM Application Evaluations 🚀

Inference code for Llama models

[NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333

Oh my tmux! My self-contained, pretty & versatile tmux configuration made with 💛🩷💙🖤❤️🤍

This repository is to prepare for Machine Learning interviews.

Instruct-tune LLaMA on consumer hardware

An interactive exploration of Transformer programming.