NVIDIA DFlash Speculative Decoding Boosts Blackwell Inference Up to 15x

NVIDIA published results for DFlash, an open source block-diffusion drafter for speculative decoding, showing up to 15x higher throughput for gpt-oss-120b on Blackwell Ultra at the same interactivity level compared with autoregressive decoding. The company also reports that DFlash nearly doubles interactivity over EAGLE-3 speculative decoding for Llama 3.1 8B at matched concurrency.

How DFlash changes the decode path

Speculative decoding traditionally uses a lightweight autoregressive model to draft future tokens, which the target model verifies in parallel. DFlash replaces that drafter with a block-diffusion model that predicts a block of masked future tokens in a single forward pass. This turns sequential drafting into block-parallel GPU work, while the target model still verifies the proposed tokens and preserves its output distribution.

NVIDIA attributes the gains to three techniques: block-diffusion drafting, target hidden-state conditioning, and KV injection. The drafter uses context features from the target model and injects them into its key-value projections across layers, which helps maintain high acceptance rates.

On an eight-GPU NVIDIA DGX B300 system running TensorRT-LLM with the SPEED-Bench coding dataset, DFlash delivered more than 15x throughput over autoregressive decoding in the 500 to 600 tokens per second per user interactivity range, and 1.5x higher than EAGLE-3. At batch size 1, DFlash more than doubled interactivity on Blackwell. The company notes that in the decode-constrained region, inference is often limited by memory movement and sequential token generation rather than raw compute, and DFlash shifts part of that work into parallel block drafting and verification.

NVIDIA also published speedup tables for gpt-oss-120b and Llama 3.1 8B Instruct across Speed-Bench datasets at matched user concurrency. For gpt-oss-120b, DFlash averaged 2.3x interactivity speedup versus 1.7x for EAGLE-3. For Llama 3.1 8B, DFlash averaged 2.8x versus 2.2x for EAGLE-3.

Framework integration and checkpoints

Researchers at UC San Diego released the DFlash paper in February 2026. Since then, the team has published 20 DFlash model checkpoints on Hugging Face with recipes for NVIDIA Blackwell and Hopper GPUs, covering Qwen, Kimi K2.6, Llama, Gemma, and gpt-oss families.

On vLLM, developers can swap an EAGLE-3 checkpoint for a DFlash checkpoint with no code changes outside the config, using the open source Speculators library. On Gemma 4 31B running on a single Blackwell Ultra GPU, this path delivered up to 5.8x higher throughput over autoregressive decoding at the same concurrency, with speedups ranging from 3.0x on MT-Bench to 5.8x on Math500.

On SGLang, migrating from EAGLE to DFlash requires updating the speculative decoding algorithm and providing the matching draft checkpoint. On Qwen3 8B running on a single NVIDIA B200 GPU, this path delivered up to 5.1x throughput over autoregressive decoding on Math500 and 4.2x on HumanEval.

NVIDIA states that DFlash is supported in SGLang, vLLM, and TensorRT-LLM on NVIDIA GPUs, allowing teams to adopt it through existing serving stacks without application refactoring. The company positions DFlash as a practical optimization for agentic workloads that need strict per-user token latency while scaling concurrency.