Implementation of Deterministic Parameter-Free Projection (DPFP) from the paper "Linear Transformers Are Secretly Fast Weight Memory Systems"

Patch-Based Stochastic Attention (efficient attention mecanism)

Basic Flash attention Implmentation

Code for the paper "Cottention: Linear Transformers With Cosine Attention"

easy naive flash attention without optimization base on origin paper

🤖FFPA: Extend FlashAttention-2 with Split-D, ~O(1) SRAM complexity for large headdim, 1.8x~3x↑🎉 vs SDPA EA.

A simple implementation of PagedAttention purely written in CUDA and C++.

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

FlashInfer: Kernel Library for LLM Serving