NVIDIA Dynamo v1.4.0 Adds Cross-Datacenter Prefix Routing and vLLM-Compatible Endpoint

NVIDIA Dynamo v1.4.0, the 17th feature release, ships with 640 merged pull requests from 127 contributors. The headline addition is experimental cross-datacenter prefix routing in the Dynamo Router, enabled by a sequenced datacenter-scoped KV relay that extends prefix decisions across datacenters. An in-process selection cache lets peers replay booking decisions, and session affinity now synchronizes across Router replicas. Cache-salt-aware KV routing isolates tenant KV caches, while the event transport is endpoint-scoped for explicit aggregation.

Frontend and Tokenizer Updates

The frontend gains an experimental vLLM-compatible /inference/v1/generate endpoint, activated via the DYN_VLLM_ENABLE_INFERENCE_V1_GENERATE flag. A process-local L1 tokenizer prefix cache is now enabled by default with a 64 MiB budget, and it supports exact TikToken special-token handling. Chat templates accept deployment-level defaults for reasoning behavior. The dynamo-protocols wire crate is updated to version 5.0.1.

Multi-Modal and Realtime Support

vLLM-Omni pipelines now use NIXL RDMA tensor transport for true multi-node disaggregation. The unified vLLM backend serves image, video, and audio through a single pipeline, with a pluggable custom vision encoder interface for in-process encoder swaps. An experimental realtime worker streams transcripts and output audio. The Spica discrete-event simulation framework has moved into the repository under aisimulate, and the request_trace_to_satf CLI converts request logs to SATF 2.0 artifacts.

Performance and Operations

Dynamo Mocker now uses the native G1 block manager, preserving KV event order, and median output-token throughput increased by 22.2% in a balanced A/B test. Inter-token latency collection is buffered, reducing CPU cost by 71.9%. The Kubernetes DynamoGraphDeployment gains rollout control via spec.grove.updateStrategy, and the ready condition reports specific stall reasons like insufficient_capacity. The convert_api_version.py utility migrates v1alpha1 manifests to v1beta1. GPU Memory Service adds intra-pod shadow-engine failover counters with a Grafana dashboard. XPU serving receives ready-made DRA deployment YAMLs for aggregated and disaggregated vLLM. The inference-gateway EPP moves toward standalone operation, discovering workers from GAIE InferencePool. Volcano scheduling is added for Grove-backed deployments, and the operator no longer bundles CRDs in the Helm subchart. Prometheus histograms now report images, videos, and audio parts per request. Request payload auditing is unified into the dynamo.request.trace.v1 stream, with exports via standard OpenTelemetry Collector and captures of allowlisted HTTP headers.

Documentation and Ecosystem

The documentation website is rebuilt with tab-based navigation and zh-CN localization on top pages. A compatibility page includes a version picker showing backend, CUDA, driver, GPU, OS, and architecture details. The Nightly Releases page lists latest nightly containers and wheels, and the Model Early Access Builds catalog shows per-model early-access containers. The engine fleet is upgraded to vLLM v0.26.0, SGLang v0.5.16, TensorRT-LLM 1.3.0rc22, and NIXL v1.3.x. First-time contributors include 29 developers making initial merged contributions.