fp8

Docker compose configs for serving Qwen3.8-27B on a DGX Spark (GB10): vLLM+MTP current, SGLang+DSPARK + FP8 rollback stacks

Field-tested guide: multi-GPU vLLM tensor-parallel (TP=2/TP=4) on Intel Arc Pro B70 (Battlemage BMG-G31, Xe2) on Linux. Driver setup (xe force_probe=e223), bare-metal vLLM + oneAPI 2025.3, the compute-runtime multi-root USM + triton-xpu init_devices fixes, FP8/int4-AutoRound quant, root-cause error reports. AI-agent readable (AGENTS.md).

Build, run, and setup scripts for the complete TensorRT-LLM pipeline on RTX A6000 Ada (SM89). Reproducible path from HuggingFace checkpoint to deployable .engine file, with FP16 baseline and FP8 quantization. Companion material to the 4-part blog series on ai-box.eu — in preparation for the NVIDIA TensorRT Edge-LLM ecosystem.

Minimal AITER LDS and ROCr idle-CPU fixes for Radeon AI PRO R9700 on vLLM 0.29.0 / ROCm 7.2.3

TP=4 serving recipe for GLM-5.3-Flash (320B FP8) on 4x DGX Spark GB10: env-driven SGLang launch, RoCE tuning, benchmark, runbook

Standalone eugr/spark-vllm-docker recipes for Qwen3.8-27B NVFP4 and official FP8 on one DGX Spark

vLLM deployment and tuning notes for Qwen3.8-27B-Uncensored-FP8 on 2x RTX 5090, with FP8 KV cache, prefix caching, TP=2, 131K context, and benchmark findings.

vLLM serving stack for Gemma 4 31B on RTX PRO 6000 Blackwell, with FP8 KV cache, MTP speculative decoding, and an async FastAPI logging proxy in front.

Pinned eugr recipes and native vLLM/B12X runtime for DeepSeek-V4-Flash-Vision-Exp on 2x DGX Spark with TP2, FP8 KV, 327K context, vision, and DSpark K6.