TVM FFI

FlashInfer: Kernel Library for LLM Serving

A high-throughput and memory-efficient inference and serving engine for LLMs

Development repository for the Triton language and compiler

Open deep learning compiler stack for cpu, gpu and specialized accelerators

An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).

common in-memory tensor structure

Hidet: A compilation-based DNN inference framework

Large Language Model Text Generation Inference

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

A list of awesome compiler projects and papers for tensor computation and deep learning.

Visualizer for neural network, deep learning, and machine learning models

Open standard for machine learning interoperability

Samples for CUDA Developers which demonstrates features in CUDA Toolkit