vLLM v0.26.0 adds Inkling models, DeepSeek-V4 optimizations, per-KV attention backends

vLLM v0.26.0 ships with 411 commits from 212 contributors, including 61 first-time contributors. The release adds the Inkling model family with a complete support stack: base modeling, piecewise CUDA graph support, Hopper FA4 relative attention, MTP=1 speculative decoding, LoRA, and standard ModelOpt NVFP4 quantization. New models also include BertForMaskedLM, RobertaForTokenClassification, XLMRobertaForTokenClassification, LongCat-Flash-Lite n-gram embedding, Cosmos3 Edge Reasoner, Cosmos3-Super registration, and TranslateGemma-12b-it. TeleChat, Persimmon, and Fuyu were removed.

DeepSeek-V4 and kernel work

The release targets DeepSeek-V4 performance across vendors. A specialized routing kernel delivers a 2.94% end-to-end TPOT improvement, fused_topk_bias speeds up the kernel by 1.5 to 2 times, and redundant repeat and copy removal yields a 1.8% end-to-end TPOT gain. ROCm gets a two-stage compressor for HCA prefill, sparse decode and prefill optimizations, and DSpark speculative decoding on AMD and XPU. MoE router GEMMs include BF16x3 router GEMM, FP32 router GEMV, and a generic CuteDSL LL BF16 router GEMM. Qwen work fuses more RMSNorm with all-reduce in Qwen3.5, replaces MoE all-reduce with reduce-scatter, and adds an H20 optimization. MLA gets a dense MHA path for short sparse-MLA sequences and a long-context decode indexer for MiniMax-M3 on sm100. The sampler stops upcasting logits to fp32.

Attention and KV cache

The attention backend can now be selected per KV-cache group, and sliding-window support is an explicit backend capability, improving hybrid model support. A KV-cache layout refactor packs K and V into the content dim across backends. KV offloading matured with metrics, tier-owned event handling, an object-store secondary tier with workload identity, DP-replica-aware tiering, and a blocks_per_chunk config for heterogeneous KV groups. Encoder-cache connectors include CPU offloading. Partial prefix-cache hits work for hybrid models, with selective hybrid cache retention and reporting of prefix-cache-reused blocks in full report mode.

Precision and frontend

Generation models can use fp32 lm_head via head_dtype, extended to the LoRA path with a ROCm torch.mm fast path. Speculative decoding gains runtime draft weight updates, hybrid DFlash drafters, SWA support for qwen-eagle3, a Gemma4-12B DSpark draft model, and a separate kv_cache_dtype for speculative_config. The Rust frontend adds multimodal video and audio, a Seed-OSS tool parser, and a native vllm-bench port. Transformers 5.13.0 brings more models to the Transformers modeling backend: Olmo and Olmo2, MistralLarge3, and HunyuanVL. OpenAI compatibility adds bad_words in completions, logprob_token_ids on Python endpoints, and include_reasoning for non-Harmony models. Security fixes replace diskcache to eliminate pickle deserialization, fix a concurrent sparse-invariant race, and add resource-bounds validation to derender endpoints. Dependencies include FlashInfer 0.6.14, NIXL 1.3.1, and nvidia-cutlass-dsl 4.6.0.