SGLang v0.5.15 tunes GLM-5.2 NVFP4, adds Spec V2 and DeepSeek-V4 gains

SGLang v0.5.15, published July 10, 2026, makes speculative decoding V2 the default path and tunes GLM-5.2 NVFP4 for production serving on Blackwell. The release reports GLM-5.2 NVFP4 running at over 500 tokens per second per user on 8x B300 and 450 on 4x GB300 at batch size 1, with a cookbook covering the setup.

Speculative decoding and kernel changes

Spec V2 becomes the default scheduling path. The release notes describe zero-overhead scheduling through CUDA-graphable DSA draft-extend, removal of device-to-host and host-to-device syncs, and fused metadata operations, with an 11% end-to-end throughput gain. IndexShare MTP reuses the indexer top-k across draft steps, cutting draft-step cost by up to 1.9x at long context. TopK V2 fuses top-k selection with the page-table transform and supports runtime k up to 2048. Indexer prologue fusion reduces 12 kernels to 4, giving roughly 8% faster decode at batch size 1. GEMM improvements include a shape-specialized JIT router GEMM and a CuteDSL BF16 GEMM for Blackwell. FlashInfer autotune now covers draft-model graphs.

Breakable CUDA Graph is now the default capture path, reducing per-step kernel-launch overhead, with full CUDA Graph support for the prefill phase landing as experimental. Linear-attention kernels get a new FlashKDA prefill backend for safe-gate KDA linear attention, plus ReplaySSM buffered output-only decode. FlashInfer all-to-all is added for routed MoE via the flashinfer_trtllm_routed runner.

DeepSeek-V4 and new models

DeepSeek-V4 receives several optimizations. FlashMLA sparse prefill is enabled by default, with the release reporting over 10% throughput gain on long context. A non-paged indexer for long-context prefill is enabled by default for large prefill chunks, with over 5% end-to-end throughput gain. C128 state-pool allocation is optimized using the request state pool. Decode context parallelism lands for MLA models, including DeepSeek V3 and Kimi K2 series.

New model support includes Hunyuan 3 (Hy3), Hierarchical Reasoning Model (HRM-Text), NVIDIA LocateAnything-3B, Baidu Unlimited-OCR, JoyEcho multi-shot audio-video, and Qwen3.6 NVFP4 support. Native web search backed by Exa is built in. Dependencies are upgraded, with transformers bumped to 5.12.1 and tvm-ffi, sgl-deep-gemm, and tilelang updated.

AMD and Intel GPU work continues. GLM-5.2 cookbooks add AMD MI300X, MI325X, and MI355X support, with EAGLE verified on MI300X and MI325X. DeepSeek-V4 gets AMD reduce-scatter decode, removal of per-batch device-to-host syncs in MTP, and a Triton FP8 sparse-MLA prefill kernel for gfx950. Speculative decoding gains XPU support, with a quarantine for EAGLE3 parity on that platform.