DFlash: Block Diffusion for Flash Speculative Decoding

Paper | Blog | Models

DFlash is a lightweight block diffusion model designed for speculative decoding. It enables efficient and high-quality parallel drafting.

Feel free to open a GitHub issue to request support for additional models. We will also open-source the training recipe soon, so you can train your own DFlash draft model to accelerate any LLM.

Use a separate virtual environment for each to avoid conflict.

vLLM: DFlash support requires the nightly build:

uv pip install -e ".[vllm]"
uv pip install -U vllm --torch-backend=auto --extra-index-url https://wheels.vllm.ai/nightly

Note: I've been testing primarily with the vLLM backend on a single A100 80GB. SGLang setup worked fine too but requires the SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 env var or you'll hit context length errors on longer prompts.

vllm serve Qwen/Qwen3.5-27B \
  --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.5-27B-DFlash", "num_speculative_tokens": 15}' \
  --gpu-memory-utilization 0.9 \
  --max-model-len 8192

Personal note: I've found num_speculative_tokens between 10 and 15 to be the sweet spot on my setup (2x A100 40GB). Going above 15 didn't improve throughput meaningfully and sometimes hurt acceptance rate.

SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 python -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-27B \
  --speculative-algorithm dflash \
  --speculative-draft-model-path z-lab/Qwen3.5-27B-DFlash \
  --speculative-num-steps 5 \
  --speculative-eagle-topk 4 \
  --speculative-num-draft-tokens 15

If you find DFlash useful, please cite:

@article{dflash2025,
  title={DFlash: Block Diffusion for Flash Speculative Decoding},
  year={2025}
}