14 hours of measurements on a single consumer card. Every answer was computed by a reference implementation before the model was asked. Every wrong turn is kept in section 13 rather than quietly edited out.
If Qwen3.8-27B crashes on every image on your Ampere card:
export GGML_CUDA_CUBLAS_COMPUTE_TYPE=fp32Undocumented. Only fp32 works (fp16 does not). Zero measured performance
cost. Upstream: ggml-org/llama.cpp#24999.
A benchmark of Qwen3.8-27B (hybrid SSM + attention, 27B params) running
4-bit quantised on a single RTX 3090 (24 GB) via llama.cpp. It measures only
things with a verifiable answer — no taste-based scoring.
Full report: benchmark.html (self-contained, 64 KB)
| Vision crash | Fixed by one env var, zero measured cost |
| Usable context | 131,072 tokens with vision, verified by filling it |
| Decode, sustained | 63.7 tok/s at 380 W, 79 °C |
| MTP draft length | n=2 wins on 3090 (not n=3 — that's an RTX 6000 result) |
| Quantisation | AtomicChat AD-Q4_K_M is 765 MiB smaller than Unsloth UD-Q4_K_XL, worth ~28,000 tokens of context. Same quality (perplexity gap 13-17× smaller than error bars). |
| Reasoning for code | Off for medium tasks (same correctness, 5-85× faster). On for the capability edge (c4, c5). |
| Temperature | A speed knob, not a quality knob. 0.6-0.7 gives ~10% more throughput than the recommended 1.0. |
benchmark.html # Full self-contained report (open in any browser)
suite/ # The reusable benchmark suite
README.md # What it measures, how to run it, 8 methodological pitfalls
run-suite.sh # Entry point: ./run-suite.sh model.gguf [mmproj.gguf]
probes/ # Individual probes (profile, vram, mtp, ppl, ladder, effort, tools, ctxfill, power, preserve)
results/ # Raw results from the Qwen3.8-27B run
pruebas/ # Raw measurement data
razonamiento/ # Reasoning probes, telemetry, A/B logs
code_tasks/ # c1-c5 specs, reference implementations, hidden test suites
serve-qwen38.sh # The exact launch command used for all measurements
perfilar-modelo.py # GPU telemetry sampler (nvidia-smi, 2s interval, 0.0% CPU overhead)
barrido-potencia.sh # Power-limit sweep script
The exact launch command (see serve-qwen38.sh):
export GGML_CUDA_CUBLAS_COMPUTE_TYPE=fp32
llama serve -m Qwen3.8-27B-AD-Q4_K_M.gguf \
--mmproj mmproj-F16.gguf \
-ngl 999 -fa on --jinja -np 1 \
-c 131072 -ub 512 -b 4096 \
--cache-type-k q4_0 --cache-type-v q4_0 \
--spec-type draft-mtp --spec-draft-n-max 2 \
--reasoning off \
--temp 0.7 --top-p 0.95 --top-k 20 --min-p 0.0 \
--host 0.0.0.0 --port 8080The suite is designed to be run against any other GGUF and to be broken. A benchmark nobody can falsify is worth nothing.
| Component | Spec |
|---|---|
| GPU | NVIDIA RTX 3090, 24 GB (GA102, sm_86) |
| CPU | AMD Ryzen 9 5950X, 16C/32T |
| RAM | 31 GiB |
| OS | Debian 13 (trixie), KDE/Wayland |
| Driver | 610.57.04, CUDA UMD 13.3 |
| llama.cpp | b10217-ddd4ec142 |
| Publisher | File | Size |
|---|---|---|
| Unsloth | UD-Q4_K_XL | 17,093 MiB |
| AtomicChat | AD-Q4_K_M | 16,328 MiB |
Both with mmproj-F16.gguf (927 MB).
MIT — run it against your own model, break it, submit a PR.