LLM inference in C/C++

Port of OpenAI's Whisper model in C/C++

A library for efficient similarity search and clustering of dense vectors.

DuckDB is an analytical in-process SQL database management system

Windows Subsystem for Linux

Android real-time display control software

An MCP-based chatbot | 一个基于MCP的聊天机器人

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba. Full multimodal LLM Android App:[MNN-LLM-Android](./apps/Android/MnnLlmChat/READ…

FlashMLA: Efficient Multi-head Latent Attention Kernels

WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime for cloud native, edge, and decentralized applications. It powers serverless apps, embedded functions, microservices,…

Conversion between Traditional and Simplified Chinese

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

Sampling CPU and HEAP profiler for Java featuring AsyncGetCallTrace + perf_events

High-speed Large Language Model Serving for Local Deployment

lightweight, standalone C++ inference engine for Google's Gemma models.

Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,...) inference in pure C/C++

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

A lightweight library for portable low-level GPU computation using WebGPU.

Fast Open-Source Search & Clustering engine × for Vectors & Arbitrary Objects × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C#, GoLang, and Wolfram 🔍

C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)

全球最小的桌面级双轮腿机器人！

WinDirStat is a disk usage statistics viewer and cleanup tool for Microsoft Windows

The AI-Native Search Database. Unifies vector, text, structured and semi-structured data in a single engine, enabling hybrid search and in-database AI workflows.

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

llama.cpp fork with additional SOTA quants and improved performance

A highly optimized LLM inference acceleration engine for Llama and its variants.

Low-bit LLM inference on CPU/NPU with lookup table

Suno AI's Bark model in C/C++ for fast text-to-speech generation