LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features

Clojure native interop through JavaCPP

A fast Clojure Tensor & Deep Learning library

Fast Clojure Machine Learning Model Integration

Clojure LLM

Official inference framework for 1-bit LLMs

Language model tokenization at GB/s

Measure object memory consumption in Clojure

Jlama is a modern LLM inference engine for Java

An uber-fast parallelized Java classpath scanner and module scanner.

FlashSampling: Fast and Memory-Efficient Exact Sampling (https://huggingface.co/papers/2603.15854)

On-device AI across mobile, embedded and edge for PyTorch

Py4J enables Python programs to dynamically access arbitrary Java objects

Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Andr…

Matplot++: A C++ Graphics Library for Data Visualization 📊🗾

⚡ Datoviz: high-performance GPU rendering for scientific data visualization

BLAS-like Extensions for Neanderthal, Fast Clojure Matrix Library

lightweight, standalone C++ inference engine for Google's Gemma models.

APComp

Helpful kernel tutorials, examples and SKILLs for tile-based GPU programming

Jack Audio Connection Kit - Clojure API

Up to 100x faster strings for C, C++, CUDA, Python, Rust, Swift, JS, & Go, leveraging NEON, AVX2, AVX-512, SVE, GPGPU, & SWAR to accelerate search, hashing, sorting, edit distances, sketches, and m…

onnxruntime-extensions: A specialized pre- and post- processing library for ONNX Runtime

Generative AI extensions for onnxruntime

Awesome LLM Books: Curated list of books on Large Language Models

Unsupervised text tokenizer for Neural Network-based text generation.

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production