MLX

Hundreds of models & providers. One command to find what runs on your hardware.

Run any agent on any model, models stored as OCI images

PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.

Open source, terminal-first AI coding agent with fully transparent multi-model routing. Local (Ollama, LM Studio, MLX, llama.cpp) or cloud, your keys, one TUI. No black box.

A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.

Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more

The free, open-source meeting-notes app for your Mac — offline transcription and AI notes, free in the cloud or 100% on-device.

Run and serve local models headlessly

Pure rust ML compiler with IR and support for multiple backends

ML model ports (ASR · TTS · LLM · VLM · vision · audio codecs) on the rlx multi-backend compiler + runtime — CPU/Metal/MLX/CUDA/wgpu

A rust mlx server, ui, and model router for fast, dependency free inference on apple architecture

One Mac process. Many models. Real speed. Multi-model LLM serving with prefix reuse, MTP acceleration, and OpenAI APIs — built for Apple Silicon, measured against mlx-lm and llama.cpp.

A CLI in Rust to generate synthetic data for MLX friendly training

Set one RAM ceiling. Run bigger sparse language and vision models on Apple Silicon — experts stream from SSD when they don't fit, without hidden quantization. https://aosama.github.io/astronomical/ Join Discord Server https://discord.gg/dc4E6r4WD

Which local LLMs can your computer run? Hardware-aware CLI, TUI and GUI: yes/slow/no verdicts and tok/s from your GPU VRAM and RAM, then verified installs via Ollama, llama.cpp, MLX and LM Studio. Native Rust.

The local inference server for coding agents. Pure Rust, one binary, Apple Silicon + NVIDIA. Anthropic + OpenAI APIs; the KV cache survives across turns, so turn 20 starts as fast as turn 2. OPD training on the same runtime.

One serving daemon for local LLM inference — across runtimes and machines.