Bench logs and commands for local LLM, image, and video models on RTX 3060 12GB.
Reference only: hardware, flags, KV, prefill (pp), generation (tg), VRAM. Not a guide or blog.
X: @ItsmeAjayKV
| GPU | RTX 3060 12GB (~11887 MiB) |
| CPU | Ryzen 9 9900X · usually -t 12 |
| RAM | 64 GB DDR5 @ 6000 MT/s |
| OS | Ubuntu 24.04 |
Hybrid MoE uses system RAM for experts. Always log -ctk / -ctv.
Details: HARDWARE.md · techniques/kv-cache.md
| RESULTS.md | All snapshot tables (pp + tg + MTP + image + video) |
| models/ | Per-model commands and full benches |
| image/ | Bonsai Image 4B |
| video/ | MiniMax H3 (video + audio) |
| runtimes/ | llama.cpp / ik / tq3 / turboquant / benchy |
| techniques/ | KV, MoE offload, MTP, power limit |
| recipes/ | Short copy-paste commands |
| recipes/threejs-game-qwen-mtp.md | Three.js game dev: Qwen MTP Q6 server + ik_llama |
| data/underclock/ | GPU CSVs / plots |
HTML cards (GitHub Pages). Listing: ajay9o9.github.io/runs-on-a-3060.
| Report | |
|---|---|
| Ornith 1.5 35B-A3B | RAM floors + t/s · Q6_K / Q4_K_M / AD mixed |
| Model | Kind | Page |
|---|---|---|
| Qwen3.6 35B-A3B | MoE LLM | models/qwen3.6-35b-a3b.md |
| Qwen3.6 27B MTP | dense LLM + MTP | models/qwen3.6-27b.md |
| Gemma 4 12B | dense LLM (± QAT / MTP) | models/gemma4-12b.md |
| Gemma 4 26B-A4B | MoE LLM | models/gemma4-26b-a4b.md |
| Diffusion Gemma 26B-A4B | text diffusion LLM | models/diffusion-gemma.md |
| Ternary Bonsai 27B | ternary dense LLM | models/bonsai-ternary-27b.md |
| Laguna S-2.1 | large MoE (118B-A8B) | models/laguna-s-2.1.md |
| Ling-3.0-flash | MoE LLM (AtomicChat Q4_K_M GGUF) | models/ling-3.0-flash.md |
| Ornith 1.5 35B-A3B | MoE LLM (Q6_K / Q4_K_M / AD mixed) | models/ornith-1.5-35b.md |
| Bonsai Image 4B | image gen | image/bonsai-image-4b.md |
| MiniMax H3 | video + audio gen | video/minimax-h3.md |
| pp | tg | Link | |
|---|---|---|---|
| Qwen 35B TQ3 (4k) | ~610 | ~54 | RESULTS |
| Qwen 35B TQ3 (128k) | ~515 | ~59 | same |
| Qwen 35B Q6 hybrid (ik) | ~805 | ~47 | same |
| Qwen 35B MTP n-max 2–3 vs off | — | ~33 vs ~21 | RESULTS |
| Qwen 35B MTP Q6 ik vs llama prefill | ~821 vs ~473 | ~46 | three.js use case |
| Gemma 4 12B Q5 | ~1100 | ~32 | RESULTS |
| Gemma 4 12B QAT @ 32k + MTP | ~1011 | ~40.5 | RESULTS |
| Gemma 4 26B-A4B (ncmoe 20) | ~690 | ~39 | RESULTS |
| Laguna S-2.1 IQ3_S hybrid (f16 ~32k) | ~202 | ~23 | RESULTS · models |
| Laguna S-2.1 Q4_K_M (q4_0, ncmoe 46, ~164k) | ~54 | ~12 | same |
| Ling-3.0-flash Q4_K_M hybrid (ncmoe 42) | ~87–135* | ~16–21 | RESULTS · models |
| Ornith 1.5 IQ4_XS-IQ3_S 16 GB (ncmoe 22, d112k) | — | ~30.5 | RESULTS · models |
| Ornith 1.5 Q4_K_M 24 GB (ncmoe 24, d112k) | — | ~30.6 | same |
| Ornith 1.5 Q6_K 32 GB (ncmoe 28, d112k) | — | ~27.6 | same |
| Ternary Bonsai 27B | ~440 @24k | ~23–25 | RESULTS |
| Bonsai Image 4B | — | ~10 s / ~6.6 GB | RESULTS |
| MiniMax H3 480p/10 s + audio | — | ~6:10–6:47 | RESULTS · video |
*Ling first pp4096 cold (~30 t/s); steadier band is pp8k–32k.
Full rows, KV, offload, and depths → RESULTS.md.
| Repo | |
|---|---|
| llama.cpp | ggml-org/llama.cpp |
| ik_llama.cpp | ikawrakow/ik_llama.cpp |
| TQ3 | turbo-tan/llama.cpp-tq3 |
| TurboQuant (KV) | TheTom/llama-cpp-turboquant |
| TurboQuant (AtomicChat Ling) | AtomicBot-ai/atomic-llama-cpp-turboquant |
| Diffusion Gemma | llama.cpp branch nvidia-diffusion-gemma (lnigam/llama.cpp) |
| llama-benchy | eugr/llama-benchy |
| Bonsai Image | PrismML-Eng/Bonsai-image-demo |
Side-by-side Qwen forks: runtimes/comparison.md
runtime + commit
-ngl / -ncmoe (or exps=CPU) / -t
-ctk / -ctv
pp t/s + prompt size
tg t/s
peak VRAM (+ host RAM if hybrid)
MTP: n-max / draft model if used
export MODEL_DIR=/path/to/ggufs
./build/bin/llama-bench ... -ctk <type> -ctv <type> ...| Name | Kind |
|---|---|
| Ternary Bonsai 27B | text LLM |
| Bonsai Image 4B | image gen |
| Diffusion Gemma | text diffusion LLM |
| Laguna S-2.1 | large MoE LLM |
| Ling-3.0-flash | MoE LLM |
| Ornith 1.5 35B-A3B | MoE LLM |
| MiniMax H3 | video + audio gen |