Local AI, measured at home

Hi, I'm Younes.

I run open models and an AI agent on a Mac Studio at home, and write down what actually happens: how fast they are, how much memory they take, and what breaks. Every number comes with its raw data.

unified memory
96 GB
local model speed
119 tok/s
agent jobs a day
6
containers
40

See the benchmarksHow my agent works

A slice of my workflow
Me, on TelegramA message or a voice note, in the right topic.
Hermes AgentPlans the task, calls the tools, asks before risky steps.
Mac StudioQwen3.8 Flash Next on sushi, local.
OpenRouterBigger models, when a task needs one.
The resultA reply, a deploy, a post.
The whole workflow

AI pulse

All items

What's moving in AI, picked and summed up by Hermes, my agent, as it reads. Updated through the day.

  1. Mistral ships Vibe Desktop for macOS and Windows

    Mistral agent now runs local and cloud coding sessions side by side in one desktop app, using Git worktrees to isolate parallel tasks in the same repo, with IDE hooks for VS Code, JetBrains and Zed via ACP.

    agentsdocs.mistral.ai
  2. Microsoft releases MAI Code 2 and Code 2 Flash coding models

    Microsoft in-house coding models land in GitHub Copilot with a 1M-token context and promo prices down to $0.05/$0.25 per million tokens for Flash, undercutting the hosted models Copilot has been routing to.

    modelmicrosoft.ai
  3. Google ships Gemini 4 Neon at $3/$15 per million tokens

    The fourth Gemini 4 model in a month sits between Carbon and Hydrogen: 1M-token context, $3/$15 per million tokens and #3 on the Artificial Analysis Intelligence Index, giving high-frequency agent work a cheaper frontier option.

    newsblog.google
  4. llama.cpp b12503 reworks the OpenVINO backend for NPU and GPU

    Expanded model support plus reworked sliding-window cache handling and graph reuse cut mid-sequence recompiles (~9s for Gemma-4 at 1K context), a real win for Intel NPU and GPU inference.

    toolgithub.com
  5. Apple releases FastFrames open video models at interactive latency

    1.3B and 14B autoregressive two-step video diffusion models distilled from Wan2.1-14B generate 480p video at interactive latency, with all training checkpoints under Apple's AMRL license.

    modelhuggingface.co

Start here

The first question with any local model: will it fit?

So I built a calculator. Pick a model from Hugging Face, a quantization and a context length, and it tells you how much memory you need and which Mac can run it.

Custom architecture
Quantization
KV cache type
7.0GB of memory, about

  • Weights
  • KV cache
  • Buffers

    Estimate. Parameter counts and KV cache shapes come from each model's files on Hugging Face (read 2026-11-09). Weights use the average bits per weight of each GGUF type; buffers are a flat 1 GB. macOS lets the GPU use about three quarters of memory by default.

    Other things I built to answer my own questions

    I also help teams run this on their own hardware: picking a model, running it as a private service, putting agents to work. If that's you, here's how I work.

    Photos via Wikimedia Commons:Yasu (CC BY-SA 3.0), Paul Hudson (CC BY 2.0), AzureSaturn (CC0).