[ICML 2025] XAttention: Block Sparse Attention with Antidiagonal Scoring

[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filli…

Nano vLLM

A sparse attention kernel supporting mix sparse patterns

FlashInfer: Kernel Library for LLM Serving

Fast and memory-efficient exact attention

NEO is a LLM inference engine built to save the GPU memory crisis by CPU offloading

SGLang is a fast serving framework for large language models and vision language models.

A high-throughput and memory-efficient inference and serving engine for LLMs

LLM inference in C/C++

a repo to understand llama.cpp

Low-bit LLM inference on CPU with lookup table

Doing simple retrieval from LLM models at various context lengths to measure accuracy

⏰ Collaboratively track deadlines of conferences recommended by CCF (Website, Python Cli, Wechat Applet) / If you find it useful, please star this project, thanks~

[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

Vim-fork focused on extensibility and usability

A Neovim plugin for managing line bookmarks with Telescope integration and SQLite storage. Mark, organize, and quickly navigate between important locations in your codebase.

Lightweight coding agent that runs in your terminal

Tensor library for machine learning

This repository is a read-only mirror of https://gitlab.arm.com/kleidi/kleidiai

SIMD Library for Evaluating Elementary Functions, vectorized libm and DFT

A modern, C++-native, test framework for unit-tests, TDD and BDD - using C++14, C++17 and later (C++11 support is in v2.x branch, and C++03 on the Catch1.x branch)