DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Fast, Flexible and Portable Structured Generation

Materials for learning SGLang

SGLang is a fast serving framework for large language models and vision language models.

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

Gorilla: An API store for LLMs

ChatGPT web interface using the OpenAI API

A high-throughput and memory-efficient inference and serving engine for LLMs

FlashInfer: Kernel Library for LLM Serving

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

A repo to upload ACM 2022 Architecture notes.