Find reporting and practical references by subject.
Red Hat compares vLLM's PagedAttention and SGLang's RadixAttention for serving open weight models, with independent benchmarks showing similar throughput and latency across workload shapes.
Read the article ↗Red Hat AI Inference 3.5 brings the MaaS governance layer to Azure Kubernetes Service, adding API key validation, rate limiting, and subscription control to models served through vLLM and KServe on non-OpenShift clusters.
Read the article ↗vLLM v0.31.0 introduces a weight-cache daemon for fast engine restarts, DeepSeek-V4.1-Flash performance work on SM100, new scheduling controls, and per-request multimodal kwargs gated behind a trust flag.
Read the article ↗The llm-d v0.10.0 release shifts inference image builds to upstream vLLM, deprecates its CUDA and AWS images, and introduces a two-registry system with signed production images and unsigned PR builds.
Read the article ↗
NVIDIAInferenceNewsAlibaba's 125B-parameter MoE model with recurrent and sparse attention layers reaches over 16,000 tokens per second per GPU on NVIDIA GB300 NVL72, with day-0 support in SGLang, vLLM, and TensorRT LLM.
Read the article ↗NVIDIAInferenceNewsNVIDIA's shadow engine recovery in Dynamo restores LLM serving in 7.3 seconds versus 283 for a cold restart, using a parked engine and shared GPU memory. It's a preview feature for Kubernetes 1.34+ with vLLM support.
Read the article ↗NVIDIAInferenceNewsAlibaba's 2.4T-parameter MoE model with 95B active parameters runs over 4K tokens per second per GPU on NVIDIA GB300 NVL72 in FP8. NVIDIA provides recipes for SGLang, vLLM, Dynamo, and NIM, plus NeMo AutoModel for fine-tuning.
Read the article ↗vLLM v0.26.0 introduces the Inkling model family with full support, DeepSeek-V4 performance kernels across vendors, fp32 lm_head via head_dtype, and per-KV-cache-group attention backend selection.
Read the article ↗The PyTorch Foundation's six hosted projects reported quarterly progress: PyTorch 2.13 with Apple Silicon FlexAttention, vLLM's Model Runner V2 redesign, DeepSpeed's Ulysses integration, Ray's GB200 support, Helion's cross-hardware kernels, and Safetensors' GIL-free serialization.
Read the article ↗vLLM v0.29.0 sets Model Runner V2 as the default for all models, adds support for Hy4-preview, Qwen3.8-Flash-Next, and Kimi K3 NVFP4, and introduces per-request speculative decoding acceptance stats in OpenAI API responses.
Read the article ↗vLLM v0.28.0 ships 584 commits with major Kimi-K3 performance work, end-to-end sparse MLA for DeepSeek V4, speculative decoding advances, and disk-based KV cache offloading. New defaults raise batched token limits and Blackwell CUDA graph capture.
Read the article ↗llm-d v0.9.0 ships new batch gateway and latency predictor components, adds an Intel XPU SGLang image, and updates vLLM to v0.26.0 across CPU, ROCm, and TPU targets.
Read the article ↗Dynamo v1.4.0, the 17th feature release, introduces experimental cross-datacenter prefix routing, a vLLM-compatible generate endpoint, and a 22.2% throughput gain in the Mocker. The update also brings NIXL RDMA transport for multi-node disaggregation and a rebuilt documentation site.
Read the article ↗The release lands Kimi K3 with a full stack, DeepSeek-V4 performance work, FlashAttention 4 FP8 KV cache on SM100, and Model Runner V2 expansion to non-generative workloads.
Read the article ↗vLLM v0.25.0 sets Model Runner V2 as the default for all dense models, deletes PagedAttention, and adds a streaming parser engine plus new speculative decoding drafters.
Read the article ↗The v0.24.0 release brings new model support, a streaming parser engine, DeepEP v2 integration, and a device selection change. It also expands Model Runner V2 to quantized models by default.
Read the article ↗The release graduates multimodal serving, batch gateway, and flow control to production, adds non-Kubernetes routing for RL workloads, and shifts GPU and ROCm images to upstream vLLM builds.
Read the article ↗llm-d v0.8.0 moves CUDA, ROCm, and XPU images to upstream vLLM builds, graduates multimodal, batch, and flow control to production, and adds multi-tier KV offloading with HMA support.
Read the article ↗We count visits on the server, cookie-free. You can also let this browser tell us how long you read, how far you scroll and which links you follow. It stays with us.
