FlashInfer: Kernel Library for LLM Serving

Open deep learning compiler stack for cpu, gpu and specialized accelerators

Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

TVM FFI

Repository which handles configuration of TVM CI infrastructure.

Terraform module for scalable GitHub action runners on AWS

Our first fully AI generated deep learning system

Building the Virtuous Cycle for AI-driven LLM Systems

A Datacenter Scale Distributed Inference Serving Framework

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

SGLang is a fast serving framework for large language models and vision language models.

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

CUDA Templates for Linear Algebra Subroutines

A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer

Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

A generative world for general-purpose robotics & embodied AI learning.

Development repository for the Triton language and compiler

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

Empowering everyone to build reliable and efficient software.

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

A high-throughput and memory-efficient inference and serving engine for LLMs

LLM training in simple, raw C/CUDA

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Bringing large-language models and chat to web browsers. Everything runs inside the browser with no server support.

Backward compatible ML compute opset inspired by HLO/MHLO

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

Temp repo for prototyping relax(relay next), the effort will be upstreamed. We use the wiki pages on this repo to host design docs.