Local AI Coding Agent for AMD Strix Halo (Ryzen AI Max+ 395)

Qwen3.8 125B on the Radeon 8060S iGPU, four small models on the NPU, cloud optional — a self-hosted pi coding-agent harness built on llama.cpp.

A coding-agent harness for the AMD Strix Halo (Ryzen AI Max+ 395 / gfx1151): the pi coding agent driving Qwen3.8 Flash-Next (125B) on the iGPU, with the NPU running the small side jobs (prompt-injection screening, fast routing decisions, embedding dedup) and a llama.cpp sidecar writing commits and compaction summaries. One-command setup, crash recovery that survives a killed server, and the 262k ctx window the engine is capable of.

It runs with no API key and no outbound calls by default, and switching to a cloud model mid session is a supported shape rather than a fallback: the NPU tools and the whole sidecar suite stay on-device either way, so screening, search and compaction never follow your code off the box — the hybrid, measured.

The model was never the hard part. This repo is everything around the model: the launch scripts, the extension suite, the allocator pre-flight, the speed tooling, and the measured numbers behind every claim. Every benchmark here is logged — see BENCHMARKS.md and the local vs cloud head-to-head, seven runs of one real debugging task rated against verified ground truth.

Measured on an ASUS ROG Flow Z13 (Ryzen AI Max+ 395, Radeon 8060S, 128 GB) at 70 W sustained.

• One-command setup — ./setup.sh is idempotent: extensions, providers, sidecar download, keep-alive units, halogen launch, memory check. It touches nothing you configured yourself and probes every piece instead of trusting it — the NPU guard answers a real /v1/moderations probe, the vision tower shows enabled: true on /health, and the smoke test checks its own file artifact rather than the model's report.
• Crash recovery — progress-tracker checkpoints task and partial state; relaunch and say "continue". Verified across six scenarios, including kill -9 at response 1, server death mid-task, and 15-deep compaction chains.
• NPU side jobs — the npu-retrieval extension ships triage (a constrained routing decision with per-option probabilities in ~120 ms) and dedup_scan (near-duplicate files by embedding similarity); auto-guard screens every prompt on the guard model as a fail-open tripwire (42 % recall / 0 % false positives on the 30-prompt canary — not a security boundary). See The NPU.
• Semantic code search — the pinned semble MCP server (search + find_related), wired into mcp.json by setup. A/B on 18 ground-truth queries against the NPU RAG pipeline it replaced: 17/18 vs 13/18 top-3 hits, 2-3x faster queries, ~40x faster indexing. It needs no NPU and indexes any local path or remote git URL, on first query.
• Sidecar suite off the main context — ling-tiny-compaction (non-blocking, thinking off, 10/10 rule retention on the canary), ling-tiny-commit, ling-tiny-branch-summary, ling-tiny-repomap. Diffs and transcripts never touch halogen's prompt cache, so a side call can never evict your session prefix.
• Speed tooling — scripts/halogen-top.sh (live prefill/decode dashboard) and scripts/session-audit.sh --perf (per-session throughput from the server logs).
• bench/ and config/ — harness-ab.sh (baseline vs kill+resume A/B, one command), report-accuracy.py (grades every file:line claim in a report against the tree), the guard-canary.py / compaction-retention.py canaries, check-sidecar.sh + check-fixes.sh (offline regression checks), plus models.json.example and the compaction snippet. Two numbers there must be understood, not copied: reserveTokens is per-model, and on halogen maxTokens must stay <= 8192 or the server 400s past ~32.7k ctx.
• Also shipped — harness-tune (/tune live knobs), turn-timer, subagent/ (worker/scout/reviewer/planner, session-model inheritance), /gauntlet and /speedtest prompt templates, and opt-in voice (./setup.sh --voice: privateer-speak on a local Lemonade server — whisper STT 0.43 s, kokoro TTS 0.3 s, both on CPU). Measured no-ops were deleted; receipts in extensions/README.md.

One box, four model servers (five with voice), three pieces of silicon:

• halogen (:8731, iGPU) — Qwen3.8 Flash-Next 125B from a local checkpoint: the agent's main model, 262,144 ctx x 4 slots, prompt cache on.
• NPU sidecars (:8731, NPU) — four 0.6-0.8B models on the same server: guard, decider, embedder (and one idle reranker). Detailed in The NPU.
• ling3.0-tiny (:8090, iGPU, llama.cpp) — the sidecar for commit messages, compaction summaries and repo maps.
• semble (child process, CPU) — semantic code search over uvx; no model server.
• Lemonade (:13305, CPU, opt-in --voice) — whisper STT and kokoro TTS.

Flow: every prompt is screened by the NPU guard before the main model sees it; the main model generates on the iGPU with its prompt cache warm; side jobs that need a judgement in milliseconds (is this injection, which of these three is it, which files are near-duplicates) go to the NPU instead of the 125B model; sidecars read diffs and transcripts directly so they never evict the cached prefix; and search runs on CPU against an index built at first query. Halogen is the only engine this harness provisions — what breaks on another engine is measured, too.

Local vs cloud: TTFT and decode by tier	Wall-clock per run, seven runs
halogen 0.16.2 to 0.17.1: NPU latency 4000ms to 130ms, decode 44 to 64 tok/s	Every number in this README logged in docs/BENCHMARKS.md

# one-time install: extensions, providers, sidecar, pi defaults
./setup.sh

# launch halogen from your local checkpoint (zero downloads)
./setup.sh --halogen auto

# status check any time, ends with a live model ping
./setup.sh --doctor

--halogen auto finds *.hgn in ., /models, ~/models, ~/Downloads, or take a path explicitly; restart later with ./start-halogen.sh (written on first launch); --no-halogen installs pi without container management (laptop pi against your own providers, or against a remote Strix Halo appliance via baseUrl + PI_NPU_BASE). Other flags: --voice, --halogen-upgrade [tag], --uninstall. Prerequisites: docker (setup will not sudo-install it), pi (setup installs it via npm if missing), uvx from uv for semantic search, and an IOMMU-enabled gfx1151 box for the NPU driver. The launch is pinned to local files (no HALOGEN_DOWNLOAD), so the engine makes no outbound connections.

Plain pi also works after setup: startup defaults point at halogen, and existing overrides are never touched.

From huggingface.co/peonist-ai/halogen-qwen3.8-flash-next, plus four separate NPU repos:

One directory (e.g. ~/models/halogen-qwen3.8-flash-next/), NPU models in a lowercase npu/ subdirectory; the launcher mounts it read-only at /models inside the container, so the engine reads its NPU models from /models/npu/.

pip install -U "huggingface_hub[cli]"   # provides the hf command
hf download peonist-ai/halogen-qwen3.8-flash-next \
  qwen38-flash-next-v2.hgn qwen38-flash-next-ngram.hgn qwen38-flash-next-vision.hgn tokenizer/ \
  --local-dir ~/models/halogen-qwen3.8-flash-next
for n in decider-0.8b qwen3-embedding-0.6b qwen3guard-gen-0.6b qwen3-reranker-0.6b; do
  hf download peonist-ai/halogen-npu-$n --local-dir ~/models/halogen-qwen3.8-flash-next/npu/$n
done

Skip the rest (w4b.hgn, ht43.hgn, MTP.hgn, w4b.overlay*.hgn, qwen3.5-2b) — the checkpoint carries its own draft head. Upstream also ships an optional smaller/faster ht43.hgn checkpoint; the default stays v2 — pass its path explicitly to use it. First cold boot pins the 47.7 GiB lookup table from disk, so a few minutes of startup is normal.

# daily driver: heals any dead server, then opens the pi TUI
./start.sh ~/my-project

In the session: read/edit/bash as usual, mcp__semble__search for "where is X implemented", commit for a sidecar-written message, triage when a decision is a one-word call, and /tune to change compaction headroom, output budget, temperature or ubatch mid-session. Codebase search needs no setup — semble indexes on first query.

The default launch is halogen's native max (262,144 ctx x 4 slots) behind an allocator pre-flight gate; if order-9 pages are low it recovers or fails in seconds with the exact fix command. Context knobs, the vm-compact.timer and the small-context opt-out are in Operations.

The NPU is not where the big model runs. It runs four small models whose whole job is to answer, in 0.1-1 s, questions the 125B model would take 5-90 s on — and without touching the main model's prompt cache. halogen exposes them on the same :8731 server.

Everything on it is optional and every extension degrades quietly: with no halogen server, auto-guard fails open, triage and dedup_scan return an error string, and the agent keeps working — search, commit messages and compaction never needed the NPU.

# which NPU models the server has configured vs actually loaded (they load on first use)
curl -s localhost:8731/health | python3 -c "import json,sys;print(json.load(sys.stdin)['npu'])"

# trim the loaded set — the reranker is dead weight unless you re-add reranking yourself
HALOGEN_NPU_MODELS=decider-0.8b,qwen3-embedding-0.6b,qwen3guard-gen-0.6b ./start-halogen.sh

pi dies, you kill it, the server dies mid-task — doesn't matter. Relaunch ./start.sh in the same directory and say "continue": progress-tracker wrote a checkpoint (task + partial state) and the fresh session picks it up. Verified across six scenarios, including kill -9 at response 1, server death mid-task, and 15-deep compaction chains (reliable to ~8-10 compactions; deeper chains deserve a re-orientation pass).

Known limits, measured and documented: one pi task per repo (concurrent sessions race on the checkpoint — set PI_PROGRESS_FILE per task for parallel work), and a stale checkpoint from a previous task is data, not instructions: rm PROGRESS.md when switching tasks.

The image tag is pinned in server/start-halogen.sh (HALOGEN_IMAGE_TAG, default the version this harness was last validated on). One command does it safely:

./setup.sh --halogen-upgrade          # default: latest
./setup.sh --halogen-upgrade 0.18.0   # or a specific tag

It pulls the image, refreshes the NPU model files (new images can need new NPU device files — 0.18 added the embedding u10.elf), restarts, then runs the acceptance battery: health + version match, prompt cache on, NPU probes < 1 s, guard smoke, MTP decode > 30 t/s, and a pi -p smoke. Only then does the pin move; any failed check restores the previous NPU files and restarts on the previous pin automatically. Read the changelog first — tool-calling, schema, cache and log-routing behavior have all changed between releases.

./setup.sh --uninstall

Removes everything the installer provisioned — extensions, templates, skills, theme, pi defaults, the two providers (only where they still match what setup set) and the systemd units — after backing all of it up to ~/pi-harness-uninstall-backup-*.tar.gz. Deliberately left alone: your own model entries, mcp.json, the sidecar GGUF (the reclaim command is printed), the halogen container and your checkpoint files.

• BENCHMARKS.md — measured numbers: decode/prefill, NPU latencies, canary results, and every claim in this README with its logged run.
• Local vs cloud model comparison — seven runs, one real debugging task, answers rated against verified ground truth.
• Engines — the two stacks, what breaks on a non-halogen engine, sidecar isolation, and running a cloud model with the local NPU tools.
• llama.cpp path — the full-control engine: engine build, GGUFs and drafters, GPU-memory boot args, pi wiring.
• Operations — context size and fragmentation, watching prefill/decode, troubleshooting table.
• extensions/README.md — per-extension notes and the measured value table.
• CHANGELOG.md — release history, including what was removed after testing.

• Qwen3.8-27B on Strix Halo at 31 t/s decode, the full stack guide (r/LocalLLaMA)
• Running a local coding agent on Strix Halo with pi + llama.cpp (r/StrixHalo)
• Qwen3.8-Flash-Next on Strix Halo: 40 t/s sustained decode (r/StrixHalo)
• Neon Ladder: a playtest-graded benchmark for local LLM stacks (r/LocalLLaMA)

Companion repo: neon-ladder — the playtest-graded benchmark this stack was validated on. Benchmark protocol questions live there; harness questions live here.

MIT