CUDA kernels for linear attention variants, written in CuTe DSL and CUTLASS C++.

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

MSLK (Meta Superintelligence Labs Kernels) is a collection of PyTorch GPU operator libraries that are designed and optimized for GenAI training and inference, such as FP8 row-wise quantization and …

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.

Hackable and optimized Transformers building blocks, supporting a composable construction.

Triton-based Symmetric Memory operators and examples

A high-throughput and memory-efficient inference and serving engine for LLMs

CUDA Templates and Python DSLs for High-Performance Linear Algebra

Fast and memory-efficient exact attention

[ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Pytorch domain library for recommendation systems

Clustered Compositional Embeddings