Stars
[ICML 2025] |TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥
LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation
Multilingual Corpus of Web Fiction
Efficient LLM Inference over Long Sequences
LOFT: A 1 Million+ Token Long-Context Benchmark
Awesome List of Attention Modules and Plug&Play Modules in Computer Vision
[ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
Unofficial PyTorch/🤗Transformers(Gemma/Llama3) implementation of Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Acceptance rates for the major AI conferences
Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.
Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
Extend existing LLMs way beyond the original training length with constant memory usage, without retraining
Ring attention implementation with flash attention
Foundational Models for State-of-the-Art Speech and Text Translation
Code for the paper "Evaluating Large Language Models Trained on Code"
From Chain-of-Thought prompting to OpenAI o1 and DeepSeek-R1 🍓
Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models
Self-Supervised Speech Pre-training and Representation Learning Toolkit
This is the official implementation of "Progressive-Hint Prompting Improves Reasoning in Large Language Models"
Reproducible code for paper "qEUBO A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization" from AISTATS 2023