Run any agent on any model.

Claude Code, Codex, OpenCode and eight others, pointed at a model running on your own machine, or at any hosted provider, in one command.

llmman launch claude --model qwen3.8

Downloads a llama.cpp build for your GPU, starts a server, loads the model, and opens Claude Code against it.

Install GitHub

Terminal recording: llmman launch claude --model qwen3.8 starts a local server and opens Claude Code answering from the local model

Install

Linux, macOS

curl -fsSL https://llmmanorg.github.io/install.sh | sh

Homebrew

brew install llmmanorg/tap/llmman

Windows

irm https://llmmanorg.github.io/install.ps1 | iex

What else it does

Local or hosted, same command

--provider sends the same agent to OpenRouter, Groq, Anthropic or any other provider listed on models.dev. Your key travels per request and is never written to the agent’s config.

llmman launch codex --provider openrouter --model qwen/qwen3-coder

One endpoint your existing tools already speak

llmman serve answers Ollama, OpenAI and Anthropic routes on one port. Anything built for one of those works unchanged. GGUF runs on upstream llama.cpp, safetensors on vllm or mlx-lm.

OLLAMA_HOST=127.0.0.1:17434 ollama run unsloth/Qwen3.5-0.8B-GGUF

No curated library, no account

Models are plain OCI images. Pull from Hugging Face or from any container registry, or copy one straight into your own registry without it touching your disk. Sign it with cosign; refuse unsigned pulls if you want to.

llmman transfer hf.co/unsloth/Qwen3.5-0.8B-GGUF docker.io/you/qwen3.5:0.8b

Several machines, one endpoint

Name your other machines and a request to any of them runs wherever the model is already loaded, or wherever there is the most room for it. No leader, nothing shared.

llmman config set aggregation.peers asahi,spark

From the blog

Oct 3, 2026 Find and pull models from the web UI
The web UI has a Models page now: popular models from Docker Hub and Hugging Face, a search of both, and a card per model with every tag or quantization you can pull, its size, and whether it fits your machine.
Sep 30, 2026 llmman now speaks System One
POST /v1/systemone turns a state and a few typed questions into a probability per answer, read from a local model's own token probabilities on ggml. Nothing is generated, so there is no text to parse.
Sep 29, 2026 llmman now runs Qwen-Image 2.1
llmman run qwen-image-2.1 turns a prompt into a 1024×1024 picture, straight from the model's Diffusers safetensors, on ggml. Done in collaboration with Unsloth.
Sep 27, 2026 GPU acceleration for Adreno on Windows on ARM
llama.cpp's only prebuilt GPU binary for Windows on ARM is an OpenCL one, not Vulkan. llmman now probes for it, so machines having it stop silently falling back to CPU.
Sep 15, 2026 llmman now speaks Gemini
Antigravity's CLI talks to models over the Gemini API. llmman now translates that protocol, so AGY can run against a local model or a hosted one.

All posts →