SGLang is a fast serving framework for large language models and vision language models.

Kata Containers is an open source project and community working to build a standard implementation of lightweight Virtual Machines (VMs) that feel and perform like containers, but provide the workl…

Lightweight coding agent that runs in your terminal

Allow torch tensor memory to be released and resumed later

NVIDIA Inference Xfer Library (NIXL)

DeepEP: an efficient expert-parallel communication library

Infiniband Verbs Performance Tests

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

FlashInfer: Kernel Library for LLM Serving

A high-throughput and memory-efficient inference and serving engine for LLMs

Development repository for the Triton language and compiler

Ring attention implementation with flash attention

Easy SSH servers in Golang

Unsupervised text tokenizer for Neural Network-based text generation.