BLOCKSET: Efficient out of core tree ensemble inference

TinyChatEngine: On-Device LLM Inference Library

[ICLR 2019] ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Simplified AI runtime integration for mobile app development

A curated collection of LLM-powered Flutter apps built using RAG, AI Agents, Multi-Agent Systems, MCP, and Voice Agents.

Cactus Flutter plugin: Run AI locally in your Flutter apps

High-performance On-Device MoA (Mixture of Agents) Engine in C++. Optimized for CPU inference with RadixCache & PagedAttention. (Tiny-MoA Native)

Free AI-powered malware scanner for Linux. Detects evasive threats without signatures. Scans offline, open-source CLI with optional cloud intelligence.

On-device AI SDK powering ToolNeuron — LLM chat & tool calling (llama.cpp), Stable Diffusion image generation (QNN/MNN), image processing (upscale, segment, inpaint, depth, style), and TTS. Native C++ + Kotlin JNI. Fork or clone to use in your own app.

Run powerful AI models entirely on your Android device. 100% offline, private, no API keys. Built with Kotlin, Jetpack Compose, Material 3, and llama.cpp. Download GGUF models from Hugging Face. On-device LLM inference for Android.

Talk to your Mac, query your docs, no cloud required. On-device voice AI + RAG

onenm_local_llm is a Flutter plugin that simplifies on-device language model inference on Android using llama.cpp. It removes the complexity of setting up native runtimes, model loading, and inference pipelines, so developers can integrate local AI into their apps through a simple API.

Low-latency edge speech framework for streaming ASR, fluency event detection, and assistive phrase completion

[MobiSys 2026] On-device parallel ultra-low-bit (ternary) LLM inference with LUT-based mpGeMM kernel.

Declarative way to run AI models in React Native on device, powered by ExecuTorch.

On-device Neural Engine

Production ready toolkit to run AI locally