vLLM v0.27.0 adds Kimi K3, DeepSeek-V4 optimizations, and FlashAttention 4 SM100 support

vLLM v0.27.0 ships 561 commits from 242 contributors, with Kimi K3 landing as a complete stack: model files, kernels, Python and Rust frontends, AttnRes kernels, DeepGEMM support, compressed-tensors quantized checkpoints, DSpark AR fusion, and an option to shard the shared expert instead of replicating it. The release also adds Qwen3.5 text-only dense and MoE models with EVS video token pruning, K-EXAONE-2.0-750B-A37B, VaultGemma via the Transformers modeling backend, and jina-embeddings-v5-text-nano.

Hardware and kernel work

PyTorch 2.13.0 arrives with torchvision 0.28.0 and Triton 3.7.1, a breaking environment change that also moved XPU and CPU builds to torch 2.13. FlashAttention 4 integration deepens on SM100 with FP8 KV cache and headdim-256 support, backed by new JIT warmup infrastructure and runner-owned Triton kernel warmup that remove first-request compilation stalls. Early next-gen hardware enablement includes an sm_107 target for NVIDIA Rubin with NVLink all-reduce paths, and ROCm gfx1250 architecture support.

DeepSeek-V4 gets a performance push with sequence parallelism, a roughly 2x kernel improvement from skipping empty c128 launches, 3.4% end-to-end TTFT from skipping unneeded topk and router work, 3.9% from workspace reuse, a 1.88x kernel from removing a redundant full kernel, adaptive topk width, 448 MiB GPU memory saved in the PP buffer, a compact MXFP4 indexer KV cache, and removal of sparse-MLA q-head padding on FlashInfer 0.6.14 and newer. Kernel work includes RMSNorm uncontiguous support with 1.2 to 3.1x improvement, a MoE reduce_scatter regression fix restoring 5% end-to-end throughput, and a tuned LL BF16 router GEMM.

Model Runner V2 and serving resilience

Model Runner V2 expands to non-generative workloads with encoder-only attention, sequence pooling for embedding and classification, encoder token classification and token embedding, BGE-M3 pooling, multimodal on CPU, a multi-layer MTP speculator, and PCP now selecting MRV2. A simplified fault tolerance framework for DP plus EP external load-balancer deployments arrives, along with async preparation for elastic EP scaling. Disaggregation work covers NIXL P/D for hybrid MLA plus SSM models, heterogeneous P/D block sizes for hybrid models, and MoRIIO heterogeneous TP to DP prefill and decode read routing.

The Rust frontend grows a gRPC control plane with engine-aware health reporting, abort control, server and model discovery, KV event source discovery, and vllm-bench integrated into the vllm CLI. KV offloading gains a generic P2P secondary tier with peer lookup and serving, per-request tier filtering, self-describing KV events, pluggable eviction policies, deduplicated replicated MLA KV in the shared CPU region, and a single-copy MLA layout for CPUOffloadingSpec.

Quantization and preprocessing

New quantization capabilities include FP4 Qutlass integration for compressed-tensors, CuTeDSL MoE for ReLU2 NVFP4, MXFP8 linear support in INC, AutoRound W4A16 MoE and MXFP4 linear and MoE on XPU, KV quant mode for TurboQuant, ModelOpt FP8 emulation on SM80, and the linear-backend option honored for ModelOpt W4A16. Compressed-tensors checkpoints now support DeepSeek-V4 and Kimi-K3. Preprocessing performance work offloads derender CPU work to the renderer thread pool, moves raw-prompt preprocessing off the AsyncLLM event loop, isolates multimodal preprocessing on its own executor, defers MM embed loading, and parallelizes preprocessing within a request for online pooling. Memory and robustness fixes include CuMem slept-L1 fragmentation accounting, cgroup memory limits respected on all platforms, fail fast when /dev/shm is too small, and zero-copy tensor pickling in shm_broadcast.