An Open Source Machine Learning Framework for Everyone

LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation

Carbon Language's main repository: documents, design, implementation, and related tools. (NOTE: Carbon Language is experimental; see README)

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

Tensor library for machine learning

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

CUDA Templates and Python DSLs for High-Performance Linear Algebra

High-speed Large Language Model Serving for Local Deployment

Samples for CUDA Developers which demonstrates features in CUDA Toolkit

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

High performance server-side application framework

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

a language for fast, portable data-parallel computation

Transformer related optimization, including BERT, GPT

Production-grade client-side tracing, profiling, and analysis for complex software systems.

fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

Simplify your onnx model

A composable and fully extensible C++ execution engine library for data management systems.

A Very Low-Bitrate Codec for Speech Compression

LightSeq: A High Performance Library for Sequence Processing and Generation

A family of header-only, very fast and memory-friendly hashmap and btree containers.

C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)

An efficient video loader for deep learning with smart shuffling that's super easy to digest

Automatically Discovering Fast Parallelization Strategies for Distributed Deep Neural Network Training

Havenask is a large-scale distributed information search system widely used within Alibaba Group

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

llm deploy project based mnn. This project has merged into MNN.

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/