vLLM v0.29.0 Makes Model Runner V2 Default, Adds New Models and Spec Decode Metrics

vLLM v0.29.0, published on 2026-09-09, makes Model Runner V2 (MRV2) the default execution path for all models, completing a rollout that began with pooling models. The release includes 594 commits from 277 contributors, with 91 new contributors. MRV2 now includes CUDA graph memory profiling for KV cache auto-sizing, batch-sharded sampling that reduces per-step logits memory by a factor of the tensor parallel size, prompt embeds, hidden state extraction for speculative decoding, padded full CUDA graph dispatch for uniform decode under speculation, and DP-sync skipping before EAGLE or MTP draft prefill. MRV1 remains in use for a few ROCm models and for features MRV2 does not yet support, such as sequence parallelism, dual-batch overlap, elastic expert parallelism, custom logits processors, and certain speculative decoding methods. vLLM will fall back to MRV1 if any of these features are configured. The project targets MRV1 removal in v0.32 and will not accept new MRV1-specific improvements.

New models and performance work

New model support includes Hy4-preview, Tencent's 770B parameter MoE with 49B active parameters using Gated DeepSeek Sparse Attention and native MTP; Qwen3.8-Flash-Next with BF16, FP8, and NVFP4 precision plus MTP; GraniteSWA and GraniteMoeSWA; NemotronH_Omni_Reasoning_V3 with MTP; and Kimi K3 NVFP4 checkpoints. Several models moved to the Transformers modeling backend: FlexOlmo, Olmo3, and Hunyuan V1/VL. Ten deprecated model architectures were removed.

Performance work targets Kimi K3 and DeepSeek V4. For Kimi K3, fused MXFP4 top-k finalization in the latent tail gives about 5% end-to-end latency improvement, Mamba metadata preparation now runs in a single Triton launch with a 6.6 to 7.6x kernel speedup, and a tuned Hopper low-latency GEMM now also dispatches on SM100 and is used for the eh_proj projection with a 12.9 to 25.2% kernel speedup. For DeepSeek V4, shared experts are fused into MegaMoE, adaptive top-k width re-landed, a native SwiGLU clamp kernel for Humming MoE added, and an opt-in FlashInfer expert parallel backend is available.

Speculative decoding and RL workflows

Speculative decoding gains per-request acceptance statistics in OpenAI API responses via the per-request-spec-decode-metrics option. Adaptive verification now extends to logprobs. SM100 sparse MLA support was added for GLM-5.2 and DeepSeek V4 on SM90. Qwen3-Omni DSpark drafts, PLaMo3 EAGLE-3 and DFlash, and DFlash2 loading from the speculators format are new.

Reinforcement learning weight sync gets a sharded_rdt P2P backend where each worker pulls only its tensor parallel and expert parallel slice over NIXL or Ray Direct Transport. Rank-local IPC weight updates, sparse checkpoint-coordinate updates through native weight loaders, and routed expert loading for gpt-oss are included.

New defaults and breaking changes

FlashInfer all-reduce is now enabled by default for TP CUDA groups, with opt-out via the VLLM_ALLREDUCE_USE_FLASHINFER environment variable set to 0. Prefix-cache NONE_HASH is deterministic by default, so distributed KV cache users no longer need to pin PYTHONHASHSEED. New admission-control flags max-num-queued-reqs and max-num-queued-tokens limit queue growth.

Breaking changes include the removal of the PyAV video decoder backend, deprecation of the python -m vllm.entrypoints.openai.api_server entry point in favor of vllm serve, and removal of the VLLM_TEST_FORCE_FP8_MARLIN and VLLM_ROCM_USE_AITER_FP4_ASM_GEMM environment variables. Mamba prefix caching now uses internal prefill checkpoints for a 9% to 25% time-to-first-token improvement, and the prefix_cache_retention_interval is a CLI argument defaulting to 0, with dense retention automatically restored for hybrid models using EAGLE or MTP.

Release artifacts include Python wheels for CUDA 13.0, CUDA 12.9, ROCm, XPU, CPU, and macOS, plus Docker images for CUDA 13.0, CUDA 12.9, Ubuntu 24.04 variants, ROCm, CPU, and XPU.