NVIDIA Rubin GPU: 10x Agentic Throughput, HBM4, and NVLink 6 Details

NVIDIA introduced the Rubin GPU as the core of the Vera Rubin platform, a rack-scale system designed for agentic AI workloads that run sustained inference across many reasoning steps. The company claims the Rubin GPU delivers up to 10x more agentic throughput per unit of energy than its Blackwell predecessor, based on an internal 2T MoE workload. The chip packs 336 billion transistors, 224 streaming multiprocessors, and 896 Tensor Cores, with a third-generation Transformer Engine that provides up to 50 petaflops of NVFP4 performance.

Memory and connectivity

The Rubin GPU integrates up to 288 GB of HBM4 memory using 12-Hi stacks, delivering up to 22 TB/s of peak bandwidth. Dedicated HBM controllers manage the memory subsystem, while an enhanced Tensor Memory Accelerator (TMA) handles data movement across complex layouts. For scale-up, NVLink 6 provides 3,600 GB/s of all-to-all GPU communication through the NVLink Switch, and NVLink-C2C offers 1,800 GB/s for coherent CPU-GPU transfers. Host connectivity comes via x16 PCIe Gen 6 at up to 256 GB/s. The GPU also includes Confidential Computing with TEE-I/O to secure data at rest, in transit, and in use.

MoE and matrix operations

For mixture-of-experts models, Rubin improves TMA with inline descriptor update support. Instead of modifying descriptors in memory, kernels can keep one unified descriptor for tensors sharing the same layout and override fields like memory pointer and stride directly in the TMA instruction at runtime. This reduces metadata management and data movement overhead, helping MoE models scale as expert counts grow.

Rubin doubles Tensor Core throughput per clock by processing twice the data along the K dimension. This reduces K loop iterations, cutting loop overhead and improving Tensor Core utilization for both context and decode GEMMs at high tensor-parallel scale. The Transformer Engine also supports a 3-bit lookup-table format for matrix B, storing a 3-bit index into a small table of representative values instead of each weight directly. Rubin Tensor Cores resolve these indices inline, reducing weight storage and data movement while retaining up to MXFP8 accuracy.

Long-context attention

Rubin accelerates attention by combining activation sparsity with adaptive compression and improved softmax throughput. The attention pipeline starts with a dense QK^T computation, then loads intermediate data from Tensor Memory into a structured 2:4 sparse compressed form, generating nonzero values and metadata. This reduces write cost and storage for attention scores, allowing softmax and the second attention GEMM to operate on less data while preserving the dense output format.

Softmax throughput improves with increased exponential throughput: 2x FP32 and 4x BF16/FP16 versus the Blackwell baseline. This helps softmax keep pace with faster matrix operations, addressing a potential bottleneck as Tensor Core throughput rises.

Kernel execution efficiency

Rubin also targets kernel-to-kernel transitions, which matter in inference because activations sit on the critical path. A producer kernel writes activation data to memory, and the next kernel consumes it to generate the next token. Traditional producer-consumer execution can create bubbles in the GPU timeline, and the architecture includes features to reduce these overheads.

The Vera Rubin NVL72 integrates compute, networking, liquid cooling, and Intelligent Power Smoothing into a rack-scale execution domain, with DSX MaxLPS enabling operators to provision up to 40% more GPUs within the same power budget. NVIDIA positions the entire platform as a single unit of compute for agentic AI factories.