SGLang is yet another fast serving framework for large language models and vision language models.

Tile primitives for speedy kernels

🚀 Efficient implementations of state-of-the-art linear attention models in Torch and Triton

verl: Volcano Engine Reinforcement Learning for LLMs

Fast and memory-efficient exact attention

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

A Datacenter Scale Distributed Inference Serving Framework

Applied AI experiments and examples for PyTorch

Development repository for the Triton language and compiler

FlashInfer: Kernel Library for LLM Serving

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)

A high-throughput and memory-efficient inference and serving engine for LLMs

Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

PaddleSlim is an open-source library for deep model compression and architecture search.

Multi-platform high performance deep learning inference engine (『飞桨』多平台高性能深度学习预测引擎）

A treasure chest for image classification powered by PaddlePaddle

End-to-end image segmentation kit based on PaddlePaddle.

Documentations for PaddlePaddle

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

a Compiler Infrastructure for Neural Networks

OCR toolkit based on PaddlePaddle （基于飞桨的OCR工具库，包含总模型仅8.6M的超轻量级中文OCR，同时支持多种文本检测、文本识别的训练算法、服务部署和端侧部署）

Open deep learning compiler stack for cpu, gpu and specialized accelerators