Find reporting and practical references by subject.
PyTorch and NVIDIA Dynamo introduce session-level identifiers for agentic workloads, enabling session-aware routing, admission control, and KV cache reuse across multi-turn reasoning chains.
Read the article ↗
NVIDIAInferenceAnalysisNVIDIA Dynamo's encode-prefill-decode disaggregation separates vision encoding from LLM stages, delivering up to 5x faster time to first token and 7x faster end-to-end response for image-heavy workloads. Gains depend on media load, output length, model size, and traffic mix.
Read the article ↗NVIDIAInferenceNewsNVIDIA's shadow engine recovery in Dynamo restores LLM serving in 7.3 seconds versus 283 for a cold restart, using a parked engine and shared GPU memory. It's a preview feature for Kubernetes 1.34+ with vLLM support.
Read the article ↗NVIDIAInferenceNewsNVIDIA's recsys-examples adds an end-to-end HSTU generative recommender workflow using PyTorch AOTI compilation and FlexKV-backed KV caching, reporting up to 5.93x lower latency on an RTX PRO 6000 Blackwell GPU.
Read the article ↗Dynamo v1.5.0, the 18th feature release, introduces pluggable router policies, session affinity modes, a standalone Sidecar for stock engines, and a new GPU Memory Service, among 658 merged pull requests.
Read the article ↗Dynamo v1.4.0, the 17th feature release, introduces experimental cross-datacenter prefix routing, a vLLM-compatible generate endpoint, and a 22.2% throughput gain in the Mocker. The update also brings NIXL RDMA transport for multi-node disaggregation and a rebuilt documentation site.
Read the article ↗We count visits on the server, cookie-free. You can also let this browser tell us how long you read, how far you scroll and which links you follow. It stays with us.
