SGLang v0.5.19 adds beam search, DeepEP v2, and new model support

SGLang v0.5.19, published September 5, 2026, includes 786 pull requests from 214 contributors. The release adds beam search, a new DeepEP backend, LayerNorm sequence parallelism, and support for several new models, including Qwen3.8, Qwen3.8-27B, dots3.note, Ling-3.0-flash, Ling-3.0-tiny, Spark2.5, MiniCPM-SALA, Granite 4.2, and LongCat-Image-Edit and Edit-Turbo. Cookbook guides were updated for GLM-5.3, PaddleOCR-VL, Kimi-K3 on Ascend A3, Kimi-K2.7-Code-MXFP4 on MI355X, Qwen3.5 MXFP4 on MI355X with FP8 KV cache or HiCache, MiniMax-H3 on 24 GB GPUs or DGX Spark, Ling-3.0-flash on DGX Spark, and Qwen3.8-27B on RTX 5090, RTX PRO 6000, and DGX Spark.

New serving features

Beam search is now available by passing a beam width in the request, returning the best sequences instead of a single sample. It works alongside regular requests but does not yet combine with speculative decoding, disaggregation, DP attention, or HiCache. DeepEP v2 introduces the ElasticBuffer engine, selectable with the MoE all-to-all backend option for DeepSeek-V3/V4 and Qwen3-MoE in FP8. Its fixed-size buffers allow decode to run under CUDA graphs across nodes, with performance on par with the classic backend. LayerNorm sequence parallelism, enabled with a flag, lets each tensor-parallel rank normalize only its share of prefill tokens, cutting prefill time by 3.5% on Qwen3-8B on H100 and 5.6% on B200, with savings growing as TP degree increases. This applies to dense Qwen3 models only.

Performance updates

For MXFP4 experts on Hopper, activation quantization to FP8 is now possible with a FlashInfer precision option, giving DeepSeek-V4-Flash about 12% more output throughput with no change in GSM8K accuracy, requiring FlashInfer 0.6.18. Decode context parallelism now runs on the default Blackwell MLA backend, paying off at long context: at 128K input, plain TP stops scaling around 680 tokens per second on eight B200s, while DCP continues as concurrency grows. Speculative decoding kernels got faster: DSA prefill top-k moves to a v2 kernel, 1.3 to 1.8 times faster on B200, and KDA models can opt into a fused accept path that cuts MTP verify-and-commit time by 45% to 63% on Kimi-Linear shapes with bit-identical output. The unified radix tree is now the default cache for all models, and PD decode workers can reuse cached prefixes for SWA hybrid models like gpt-oss. L3 storage can be attached or detached on a running server, and pipeline parallelism with HiCache L3 stays consistent across ranks.

On AMD, a persistent Lean attention kernel spreads work across compute units on MI300X and MI355X, giving up to 1.52x more throughput and up to 3.62x lower inter-token latency on MI355X for long or uneven decode batches. It enables itself where helpful, with an environment variable to disable. DSA models on ROCm see improvements: disaggregated GLM-5.2 serving uses a fused top-k seed remap, cutting decode TPOT from 23 ms to 8 ms on eight MI355Xs, DeepSeek-V4 gets a v2 top-k kernel up to three times faster, and a shared-experts gate fix gives GLM-5.2 up to 16% better TPOT.

Dependency and platform updates

FlashInfer moves to 0.6.18, sgl-deep-ep to 0.1.2, sgl-deep-gemm to 0.1.7, and mooncake to 0.3.13. New container images include a CUDA 13.4 preview for Rubin and ROCm 10 images for gfx942, gfx950, and gfx1250. The Rust server gains a process-local in-memory KV indexer, Router integration, e2e latency metadata, configurable HTTP/2 connection window, and a fix for Sarashina import. Speculative decoding adds DFlash2 with local convolution and candidate selector, LFM2 and LFM2-MoE DSpark support, Nemotron 3.5 Lightning support, multi-adapter LoRA with EAGLE/NEXTN/DFLASH/DSPARK, and a fix for Gemma-4 FP8 MTP acceptance rising from 0% to 60%. Piecewise and breakable CUDA graphs now support pipeline parallelism in full prefill, coordinate prefill across DP-attention ranks, and reduce idle DP work with median TTFT down 18.9% and QPS up 12.9%. A new fmha_v2 attention backend for SM90/120 is about 15% faster than FA3 at the kernel level.