Stars
Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
State-of-the-art LLM compression, built for production inference with vLLM
Modern C++ Programming Course (C++03/11/14/17/20/23/26)
2025年 最新收录整理 500+ 个高质量的 C/C++ 项目，包括但不限于核心开发、基础工具、系统与并发、系统编程、图形处理、网络通信、数据处理、应用框架、开源工具、嵌入式开发等多个领域。适合学习、参考和实战。
🔥 LeetCode for PyTorch — practice implementing softmax, attention, GPT-2 and more from scratch with instant auto-grading. Jupyter-based, self-hosted or try online.
use vLLM with nvidia/LocateAnything-3B (develop and test and 2080ti s)
Port of Nvidia LocateAnything-3B on ggml
NVIDIA Linux open GPU with P2P support
NVIDIA Linux open GPU with P2P support
A tutorial on modern GPU programming for machine learning systems
2023年最新整理 c++后端开发，1000篇优秀博文，含内存，网络，架构设计，高性能，数据结构，基础组件，中间件，分布式相关
Machine Learning Engineering Open Book
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
53AI Hub is an open-source AI portal and knowledge base for managing enterprise knowledge, AI agents, prompts, and AI tools, seamlessly integrating with Coze, Dify, FastGPT, RAGFlow. 一个AI知识库与Agent门户
MiniCPM5: SOTA on-device LLMs, small yet powerful.
[ICLR 2026] ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
DFlash: Block Diffusion for Flash Speculative Decoding
nano-vllm是开源的一个gpu推理项目，基于开源版本弄的一个ascend npu版本推理小demo，旨在帮助初学者了解推理的整体流程，区别于vllm，nano-vllm体量更小，麻雀虽小五脏俱全，更有助于初学者学习。
🧠 Train a 64M-parameter LLM from scratch in just 2h!
🚀 Efficient implementations for emerging model architectures
Flash Attention 2 implementation for Turing GPUs
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthr…
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.