modern chat interface with thread feature.

A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

A lightweight inference engine supporting speculative speculative decoding (SSD).

Supercharge Your LLM with the Fastest KV Cache Layer

The best ChatGPT that $100 can buy.

Algorithms for latent compaction

Development repository for the Triton language and compiler

SGLang is a high-performance serving framework for large language models and multimodal models.

An educational resource to help anyone learn deep reinforcement learning.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

AMD ROCm™ Software - GitHub Home

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality

An annotated implementation of the Transformer paper.