flash-attention

Here are 5 public repositories matching this topic...

llama.cpp-gfx906

kernel rocm amd-gpu llama-cpp flash-attention gfx906 mi50 vega20

Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.

gpu cuda inference nvidia mha mla multi-head-attention gqa mqa llm large-language-model flash-attention cuda-core decoding-attention flashinfer flashmla

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

gpu cuda inference nvidia cutlass mha multi-head-attention llm tensor-core large-language-model flash-attention flash-attention-2

Vulkan & GLSL implementation of FlashAttention-2

Poplar implementation of FlashAttention for IPU

deep-learning transformers pytorch ipu graphcore poplar flash-attention flash-attention-2

Add a description, image, and links to the flash-attention topic page so that developers can more easily learn about it.

To associate your repository with the flash-attention topic, visit your repo's landing page and select "manage topics."