FlashAttention for sliding window attention in Triton (fwd + bwd pass)
-
Updated
Jun 25, 2025 - Python
#
flashattention
Here are 8 public repositories matching this topic...
This repository contains multiple implementations of Flash Attention optimized with Triton kernels, showcasing progressive performance improvements through hardware-aware optimizations. The implementations range from basic block-wise processing to advanced techniques like FP8 quantization and prefetching
-
Updated
Sep 17, 2025 - Python
Cross-platform FlashAttention-2 Triton implementation for Turing+ with custom configuration mode
machine-learning deep-learning gpu optimization pytorch triton attention hopper implementation attention-mechanism turing ampere implementation-from-scratch blackwell transfromers large-language-models llm flash-attention flash-attention-2 flashattention
-
Updated
Dec 16, 2025 - Python
HRM-sMoE LLM training toolkit.
training windows ai ubuntu toolkit evaluation inference moe toolkits hrm mixture-of-experts aios ai-os deepspeed llm context-length sparse-mixture-of-experts flashattention hierarchical-reasoning-model smoe
-
Updated
Dec 18, 2025 - Python
An minimal CUDA implementation of FlashAttention v1 and v2
-
Updated
Jun 5, 2025 - Python
A high-performance kernel implementation of multi-head attention using Triton. Focused on minimizing memory overhead and maximizing throughput for large-scale transformer layers. Includes clean-tensor layouts, head-grouping optimisations, and ready-to-benchmark code you can plug into custom models.
transformers parallelism triton memory-efficiency gpu-optimization multi-head-attention kernel-programming flashattention
-
Updated
Aug 12, 2025 - Python
FlashAttention2 Analysis in Triton
-
Updated
Oct 24, 2025 - Python
Pytorch implementation of the paper FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
machine-learning deep-learning gpu optimization cuda pytorch transformer attention large-language-model flash-attention flashattention
-
Updated
Dec 15, 2025 - Python
Improve this page
Add a description, image, and links to the flashattention topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the flashattention topic, visit your repo's landing page and select "manage topics."