A heap memory profiler for Linux

基于《cuda编程-基础与实践》（樊哲勇 著）的cuda学习之路。

Fast and customizable text tokenization library with BPE and SentencePiece support

A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.5, DeepSeek-R1, GLM-5, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Llava, Phi4, ...)…

📚 从零开始构建大模型

《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程

Nano vLLM

UniFace: A Unified Face Analysis Library in Python built on ONNX Runtime

MobileGaze: Real-Time Gaze Estimation models using ResNet 18/34/50, MobileNet v2 and MobileOne s0-s4 | In PyTorch >> ONNX Runtime Inference

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

这是一个基于C++实现的从零开始的大模型推理框架

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

A tool to modify ONNX models in a visualization fashion, based on Netron and Flask.

🤖FFPA: Extend FlashAttention-2 with Split-D, ~O(1) SRAM complexity for large headdim, 1.8x~3x↑🎉 vs SDPA EA.

Fast C++ logging library.

cudnn_frontend provides a c++ wrapper for the cudnn backend API and samples on how to use it

Implement custom operators in PyTorch with cuda/c++

Fast, Flexible and Portable Structured Generation

Community maintained hardware plugin for vLLM on Ascend

fast log and exp functions for AVX2/AVX-512

校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。

The Hugging Face course on Transformers

🤗 Optimum ONNX: Export your model to ONNX and run inference with ONNX Runtime

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

《动手学大模型Dive into LLMs》系列编程实践教程

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Abseil Common Libraries (C++)

Lightweight C++ command line option parser

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling