NVIDIA Vera Rubin NVL72 Shows 30x Agentic Throughput per Megawatt vs GB300

NVIDIA published preview results for its Vera Rubin NVL72 system on the SemiAnalysis AgentX benchmark, a workload that replays prerecorded agentic-coding sessions with interleaved reasoning and tool use. At 160 tokens per second per user on the AgentX DeepSeek V4-Pro workload, Vera Rubin NVL72 delivers up to 30x higher AI-factory throughput per megawatt than GB300 NVL72, according to NVIDIA. These results were measured by NVIDIA and are pending SemiAnalysis review.

AgentX measures agentic serving

AgentX is the agentic-coding benchmark within InferenceX, SemiAnalysis's open-source suite. It evaluates how efficiently accelerators serve the request patterns produced by real coding agents. Unlike static benchmarks with fixed sequence lengths, AgentX replays each session turn by turn using the AIPerf client, preserving context, input and output sequence lengths, reasoning time, and tool-call latency from the original trajectory. This reproduces KV cache-capacity pressure and variable request lengths that accumulate across a session.

The benchmark varies concurrency to map the trade-off between throughput and interactivity. It reports sustained throughput per provisioned megawatt against four user-experience values: E2E normalized interactivity, standard interactivity, E2E latency, and time to first token. The key metric for AI factories is tokens per megawatt.

GB300 NVL72 results

On AgentX, GB300 NVL72 delivers up to 15x higher AI-factory throughput per megawatt than H200 NVL8 for DeepSeek V4 Pro 1.6T. This translates to up to 10x lower cost per million tokens than H200 NVL8, meaning a fixed power and infrastructure budget can support more interactive agentic capacity or deliver the same capacity at lower operating cost.

The advantage grows with model scale. For Kimi K3 2.8T on the AgentX workload, GB300 NVL72 shows roughly 80x the throughput per megawatt of H200 NVL8 at comparable interactivity, and extends the interactivity frontier to about 215 tokens per second per user, beyond H200 NVL8's operating range.

These results reflect system-level work across serving runtimes, model kernels, and scale-up fabric. MoE serving frameworks including SGLang, TensorRT-LLM, and vLLM distribute expert execution across the NVL72 domain. DeepGEMM-based kernels with MXFP4 and MXFP8 formats, plus fused MoE execution paths, reduce data movement between expert stages. NVIDIA Dynamo separates prefill and decode into independently scaled worker pools and supports session-aware serving through session IDs, while its KV-cache-aware router reduces unnecessary prefill computation. NVLink connects the 72 GPUs in a high-bandwidth scale-up domain for coordinated rack-scale serving.

Vera Rubin platform direction

NVIDIA positions Vera Rubin NVL72 as tuned for the long-context, interactive, and distributed execution patterns of agentic inference. The broader Vera Rubin platform includes Rubin GPUs for large context and efficient decode, Vera CPUs for tool execution and KV-cache offload, and Groq 3 LPX for ultrafast interactivity. NVLink 6, ConnectX-9, BlueField-4, and Spectrum-X move tokens, context, and tool results across the AI factory. Dynamo, Attention-FFN Disaggregation, NVFP4, TensorRT-LLM WideEP, and speculative decoding coordinate execution across processors.

The goal is to reduce recomputation and waiting, sustain interactive performance as agent sessions grow, and convert more of a fixed power budget into useful agentic throughput. The legacy InferenceX static 8K/1K sequence length test for DeepSeek-R1-0528, where GB300 NVL72 leads H200 by up to 40x more tokens per megawatt, has been demoted to maintenance mode on the InferenceX suite as agentic AI becomes the leading workload.