SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs

LLM KV cache compression made easy

[COLM 2025] Official PyTorch implementation of "Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models"

A series of technical report on Slow Thinking with LLM

Code repo for the paper "LLM-QAT Data-Free Quantization Aware Training for Large Language Models"

[ICML 2024] BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

Code repo for the paper "SpinQuant LLM quantization with learned rotations"

The official code of LM-Debugger, an interactive tool for inspection and intervention in transformer-based language models.

A library for finding knowledge neurons in pretrained transformer models.

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Code and documentation to train Stanford's Alpaca models, and generate the data.

Tensors, for human consumption

A framework for few-shot evaluation of language models.

[ICLR 2023] Code for our paper "Selective Annotation Makes Language Models Better Few-Shot Learners"

Ream: A paper manager

PyTorch implementation of MAE https//arxiv.org/abs/2111.06377

Code for paper "UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning", ACL 2022

Performance Prediction for NLP Tasks

Code for TACL 2020 paper "An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models"

Implementation of paper "Towards a Unified View of Parameter-Efficient Transfer Learning" (ICLR 2022)

Bilinear attention networks for visual question answering

Code for the ICML 2021 (long talk) paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"

Code for the paper "VisualBERT: A Simple and Performant Baseline for Vision and Language"

Beyond Accuracy: Behavioral Testing of NLP models with CheckList

Elegant PyTorch implementation of paper Model-Agnostic Meta-Learning (MAML)

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.