Stars
Code for the preprint "Cache Me If You Can: How Many KVs Do You Need for Effective Long-Context LMs?"
Easily fine-tune, evaluate and deploy Gemma 4, Qwen3.5, Qwen3.6, gpt-oss, DeepSeek-R1, or any open source LLM / VLM!
[NeurIPS 2023] MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333
Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"
[NeurIPS 2024 Spotlight] Code and data for the paper "Finding Transformer Circuits with Edge Pruning".
[NeurIPS 2024] SimPO: Simple Preference Optimization with a Reference-Free Reward
[ACL 2024] Long-Context Language Modeling with Parallel Encodings
[ICML 2024] Selecting High-Quality Data for Training Language Models
https://acl2023-retrieval-lm.github.io/
Official implementation of Privacy Implications of Retrieval-Based Language Models (EMNLP 2023). https://arxiv.org/abs/2305.14888
[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
[ICLR 2024] Evaluating Large Language Models at Evaluating Instruction Following
[EMNLP 2023] C-STS: Conditional Semantic Textual Similarity
[EMNLP 2023] Adapting Language Models to Compress Long Contexts
[EMNLP 2023] MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
[EMNLP 2023] Enabling Large Language Models to Generate Text with Citations. Paper: https://arxiv.org/abs/2305.14627
[NeurIPS 2023] Learning Transformer Programs
Findings of ACL'2023: Optimizing Test-Time Query Representations for Dense Retrieval
EMNLP 2022: "MABEL: Attenuating Gender Bias using Textual Entailment Data" https://arxiv.org/abs/2210.14975
EMNLP 2022: Finding Dataset Shortcuts with Grammar Induction https://arxiv.org/abs/2210.11560
A Kernel-Based View of Language Model Fine-Tuning https://arxiv.org/abs/2210.05643
[EMNLP 2022] Training Language Models with Memory Augmentation https://arxiv.org/abs/2205.12674
Official repo for the paper: Recovering Private Text in Federated Learning of Language Models (in NeurIPS 2022)
EMNLP 2022: Generating Natural Language Proofs with Verifier-Guided Search https://arxiv.org/abs/2205.12443
NAACL 2022: Can Rationalization Improve Robustness? https://arxiv.org/abs/2204.11790
Princeton NLP's pre-training library based on fairseq with DeepSpeed kernel integration 🚃
[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
[ACL 2022] Ditch the Gold Standard: Re-evaluating Conversational Question Answering
[NAACL 2021] Factual Probing Is [MASK]: Learning vs. Learning to Recall https://arxiv.org/abs/2104.05240