vLLM v0.31.0 adds fast restart, DeepSeek-V4.1-Flash optimizations, and security gates

vLLM v0.31.0, published October 5, 2026, includes 717 commits from 307 contributors, with 96 new contributors. The release focuses on DeepSeek-V4.1-Flash performance on SM100 hardware, a fast restart mechanism, Model Runner V2 expansion, and several breaking changes around security and quantization.

DeepSeek-V4.1-Flash and large-scale serving

FlashMLA mega attention with the NVFP4 compressed KV cache is now the default on SM100. DeepGEMM sparse MQA logits for the indexer, Mega-Gate fusing the gate GEMM with expert selection, and decoder boundaries that fuse the TP all-reduce with mHC input preparation and the MoE finalize are included. A fused small-batch WO-A with inverse RoPE and MXFP8 quant targets SM100 and SM103, and the MXFP8 wo_b GEMM is fused with the sequence-parallel reduce-scatter. Engram wkv is sharded across TP ranks, and Engram host tables are shared across co-located DP replicas by default. Encoder CUDA graphs for the vision tower and SWA bounded replay that keeps sliding-window KV out of prefix caching round out the model-specific work.

For large-scale serving, MoonEP provides a balanced EP all2all backend via the all2all-backend moonep option. Prefill context parallelism now works with data parallelism, and a low-SM multimem reduce-scatter targets SM100 and SM103. DeepEPv2 adds sequence parallelism, EPLB includes shared-expert overlap, and a sharding-aware NCCL M2N weight-transfer backend supports RL workloads. KV offloading now has back-pressure detection.

Fast restart and scheduling controls

The new vllm preload CLI launches a weight-cache daemon that keeps post-quantized weights resident in GPU memory across engine restarts. It now supports data parallelism, MTP draft models, a health endpoint, and a readiness wait. Experimental initialized-engine snapshots use CRIU to restore a fully initialized TP1 engine.

Scheduling changes include a max-num-active-seqs option that caps RUNNING admission independently of max_num_seqs, and a long-prefill-token-threshold that adapts to the number of waiting prefills instead of chunking a lone request. The waiting queue was reworked so requests already holding KV blocks are scheduled first, and a deadlock between the KV connector and MTP under KV pressure was fixed.

Model Runner V2 and speculative decoding

Draft-model speculative decoding and custom logits processors now work on Model Runner V2. The new LiLiCorr drafter is added, DFlash gets async scheduling with context K/V precompute captured in the draft CUDA graph, and DSpark adds adaptive verification for Gemma4 and variable-length decode for Kimi-K3. A DCP target can use a non-DCP DSpark draft. MoE memory is counted during MRV2 profiling to avoid OOMs on WideEP deployments.

Security and breaking changes

Per-request mm_processor_kwargs and media_io_kwargs are rejected unless the trust-request-mm-kwargs flag is set. Prefix-cache extra keys are tagged by source so a LoRA name and a cache_salt cannot collide, and the LoRA path is part of the block hash. Stale multimodal receiver-cache entries can no longer replace fresh payloads.

Breaking changes include the removal of tokenizer_mode slow, the rename of enable-mamba-fine-grained-prefix-cache to enable-mamba-shared-prefix-checkpoint, and online quantization through quantization fp8 replaced by the fp8_per_tensor shorthand. The AllSpark INT8 W8A16 backend is removed, enforce-eager now also disables JIT kernel warmup, and XPU graphs are enabled by default with the VLLM_XPU_ENABLE_XPU_GRAPH variable removed.

Model support additions include DiffusionGemma structured generation mode, MiMo V2 MXFP4 MoE, Cohere2MoE auxiliary hidden states for EAGLE3 and DFlash drafters, GLM-5.2-MXFP4 on the ROCm DeepSeek-V3.2 path, and a built-in granite_thinking_parser for Granite 4.2. Multimodal fixes cover Triton mm_input_norm, Qwen2.5-VL video fps for temporal M-RoPE, and clips longer than 30 seconds for Whisper and Qwen2-Audio. LoRA support adds Nemotron VL, ModernBert, VoyageQwen3 embeddings, and RoBERTa sequence classification.