Find reporting and practical references by subject.
Red Hat compares vLLM's PagedAttention and SGLang's RadixAttention for serving open weight models, with independent benchmarks showing similar throughput and latency across workload shapes.
Read the article ↗
NVIDIAInferenceNewsAlibaba's 125B-parameter MoE model with recurrent and sparse attention layers reaches over 16,000 tokens per second per GPU on NVIDIA GB300 NVL72, with day-0 support in SGLang, vLLM, and TensorRT LLM.
Read the article ↗NVIDIAInferenceNewsAlibaba's 2.4T-parameter MoE model with 95B active parameters runs over 4K tokens per second per GPU on NVIDIA GB300 NVL72 in FP8. NVIDIA provides recipes for SGLang, vLLM, Dynamo, and NIM, plus NeMo AutoModel for fine-tuning.
Read the article ↗SGLang v0.5.15 ships GLM-5.2 NVFP4 production tuning on Blackwell, makes Spec V2 and Breakable CUDA Graph defaults, and adds DeepSeek-V4 prefill optimizations plus new model support.
Read the article ↗NVIDIAInferenceNewsNVIDIA reports that enabling Confidential Computing on HGX B300 with a Qwen 3.5 model costs roughly 1% to 8% in throughput and time per output token, with optimizations in FlashInfer and SGLang reducing launch overhead.
Read the article ↗SGLang v0.5.19 brings beam search, DeepEP v2 with ElasticBuffer, LayerNorm sequence parallelism, and support for new models like Qwen3.8 and Granite 4.2. Performance gains target Hopper, Blackwell, and AMD GPUs.
Read the article ↗SGLang v0.5.18 brings new diffusion and autoregressive models, overlapped checkpoint staging for faster startup, a TP LMHead all-to-all optimization, and NVFP4 checkpoint support on AMD GPUs.
Read the article ↗llm-d v0.9.0 ships new batch gateway and latency predictor components, adds an Intel XPU SGLang image, and updates vLLM to v0.26.0 across CPU, ROCm, and TPU targets.
Read the article ↗SGLang v0.5.17, released August 8, 2026, brings day-0 support for Kimi K3 and MiniMax-H3, a Rust frontend, and faster model loading. The release merges 582 pull requests from 194 contributors.
Read the article ↗SGLang v0.5.16 introduces DSpark confidence-driven speculative decoding, adds support for the 975B-parameter Inkling model, and splits GLM-5.2 DSA cache layers under prefill CP to cut per-rank KV memory by about 74%.
Read the article ↗We count visits on the server, cookie-free. You can also let this browser tell us how long you read, how far you scroll and which links you follow. It stays with us.
