SGLang v0.5.17 adds Kimi K3 and MiniMax-H3 day-0 support

SGLang v0.5.17, published August 8, 2026, merges 582 pull requests from 194 contributors. The headline addition is day-0 support for Kimi K3, a 2.8-trillion-parameter multimodal LatentMoE model. Kimi K3 uses 896 experts with top-16 routing, a 3584-dim latent space, a 1M-token context, and the MoonViT3d vision tower. Serving features include KDA-aware prefix caching, HiCache L2 over DCP, LoRA on quantized weights, and an OpenAI-compatible API. The model runs on NVIDIA GB300 and AMD MI35x GPUs with DCP, DSpark speculative decoding, and TP decode.

Video generation and new models

MiniMax-H3, a video-generation model, also receives day-0 support. It handles video plus synchronized stereo audio in a single request and is available in three task profiles: t2va, fl2va, and ref2va, including video-to-video. SGLang verified MiniMax-H3 on B200 with TP2 plus Ulysses4, H100 with TP2 plus Ulysses2, AMD MI300X and MI355X, and 2x RTX 5090 with layerwise offload.

The release adds the EmbeddingGemma and LFM2.5 embedding models, plus an nvidia/MiniMax-M3-NVFP4 checkpoint. Cookbook recipes now cover the Poolside Laguna-S-2.1 family and the Inkling-Small model.

Rust frontend and communication changes

An initial Rust frontend migrates server ingress to tokenized request handling, replacing Python for that portion. The Rust server includes a tokenizer manager, request validation, an OpenAI-compatible API, and PD disaggregation support.

DCP communication backends are now pluggable, with a2a, fi_a2a, and ag_rs selectable via the dcp-comm-backend flag. A new flag, dcp-replicate-q-proj, projects full-head Q locally and skips per-layer Q all-gather.

DWDP prefill parallelism now prefetches expert weights over NVLink, removing EP all-to-all token dispatch. On 4x B200 with gpt-oss-120b, DWDP4 achieves a 1.92x speedup over DEP4 at 32K context.

Memory, loading, and caching improvements

A session-reference-aware Unified Radix Cache is introduced, enabled with the enable-session-radix-cache flag and closed with the close_session endpoint. The SM90 FP8 MegaMoE path for DeepSeek-V4 is added, guarded by the SGLANG_OPT_USE_DEEPGEMM_MEGA_MOE environment variable.

Model loading speed improves by copying non-contiguous CPU weight views to contiguous storage before H2D transfer. With this enabled, DeepSeek-V4-Pro TP8 loading time drops from about 35 minutes to 6 minutes 20 seconds, a 5.6x reduction. GPU memory for the MI355X FP4 MoE model falls from 159.07 GB to 112.36 GB by removing expert padding. The HIP compress-state pool moves into the memory_saver KV_CACHE region, cutting training-phase footprint from about 143 GiB to 87 GiB per GPU. Engine restart time is lowered via a weight-cache daemon that allows large-model recovery from GPU cache instead of storage. Hybrid-linear MTP decode host-side overhead is trimmed by reducing seam work in spec-v2 overlap scheduling.

Dependencies updated include flashinfer 0.6.15.post1, sgl-deep-gemm 0.1.5.post1, helion 1.4, mooncake 0.3.12.post1, and dynamo-tokenizers 1.7.0. PyTorch remains at 2.11.0, and the CUDA base image stays at 13.0.1.