LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Protocol Buffers - Google's data interchange format

🔍 A Hex Editor for Reverse Engineers, Programmers and people who value their retinas when working at 3 AM.

JSON for Modern C++

Port of OpenAI's Whisper model in C/C++

GoogleTest - Google Testing and Mocking Framework

Telegram Desktop messaging app

Fast C++ logging library.

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

An MCP-based chatbot | 一个基于MCP的聊天机器人

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

mold: A Modern Linker 🦠

FoundationDB - the open source, distributed, transactional key-value store

Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk

Tensor library for machine learning

World's largest Contributor driven code dataset | Used in Quark Search Engine, @OpenGenus IQ, OpenGenus Visual Project

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

A Multithreaded Fork of Redis

Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Andr…

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

The POCO C++ Libraries are powerful cross-platform C++ libraries for building network- and internet-based applications that run on desktop, server, mobile, IoT, and embedded systems.

High-speed Large Language Model Serving for Local Deployment

lightweight, standalone C++ inference engine for Google's Gemma models.

Transformer related optimization, including BERT, GPT

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

Fast inference engine for Transformer models

Kernels & AI inference engine for mobile devices.

A machine learning compiler for GPUs, CPUs, and ML accelerators

Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU