An Open Source Machine Learning Framework for Everyone

LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Port of OpenAI's Whisper model in C/C++

C++那些事

A library for efficient similarity search and clustering of dense vectors.

Productive, portable, and performant GPU programming in Python.

ncnn is a high-performance neural network inference framework optimized for the mobile platform

An MCP-based chatbot | 一个基于MCP的聊天机器人

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba. Full multimodal LLM Android App:[MNN-LLM-Android](./apps/Android/MnnLlmChat/READ…

Tensor library for machine learning

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

PaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge (飞桨高性能深度学习端侧推理引擎）

a language for fast, portable data-parallel computation

Transformer related optimization, including BERT, GPT

The public CGAL repository, see the README below

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its…

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Fast inference engine for Transformer models

fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

百度NLP：分词，词性标注，命名实体识别，词重要性

Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.

C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)

A Toolkit for Industrial Topic Modeling

ThunderSVM: A Fast SVM Library on GPUs and CPUs

llm deploy project based mnn. This project has merged into MNN.

Nearest Neighbor Search with Neighborhood Graph and Tree for High-dimensional Data