DOCA GPUNetIO unifies GPU-initiated networking across NVIDIA libraries

NVIDIA has consolidated GPU-initiated networking under a single foundation called DOCA GPUNetIO, which now underpins multiple communication libraries instead of each maintaining its own GDA-KI implementation. The framework lets CUDA kernels directly drive Ethernet, RDMA, Verbs, and DMA operations while keeping the CPU off the application critical path.

The unified umbrella matters because previously every communication library built its own separate GDA-KI-style RDMA path, each with independent assumptions, code, and maintenance burden. Now NCCL, NVSHMEM, UCX/NIXL, and Holoscan Sensor Bridge converge on one GPUNetIO implementation. This creates a shared investment point for features, optimizations, and bug fixes, reducing duplicated engineering and letting innovations spread faster across frameworks.

Open source and SDK variants

NVIDIA ships GPUNetIO in two forms. The full DOCA SDK version is the superset, spanning Verbs, Ethernet, DMA, and Comm Channel integration. The open-source project is a lighter-weight, RDMA-Verbs-focused implementation for frameworks that want fully open networking transport integration.

The split is not about divergent stacks. The open-source implementation can detect the DOCA SDK at runtime and call selected closed-source SDK functions via dlopen when an environment variable points to the SDK path. If the SDK is absent, it runs standalone. On the CUDA device side, both variants expose a similar device-facing API, keeping the GPU programming model aligned.

Programming model

Applications follow a two-phase structure. First, a host-side CPU configuration phase initializes devices, allocates memory, creates network transport objects, and exports them into GPU memory descriptors. Then a CUDA kernel launches with those descriptors as input parameters.

For DOCA Verbs, high-level functions like doca_gpu_verbs_create_qp_hl condense the steps to create and connect RDMA QPs. The data path runs on the GPU where CUDA threads post Work Queue Entries, ring the NIC doorbell, and optionally poll Completion Queue Entries.

GPUNetIO offers two API levels. The high-level API implements composite operations, such as doca_gpu_dev_verbs_put_signal() which combines an RDMA Write WQE with an RDMA Atomic Fetch & Add, handling concurrent submissions and doorbell ringing. The low-level API provides basic building blocks for custom operations but is not thread-safe, leaving synchronization to the application.

Doorbell ringing has multiple modes: regular MMIO writes, BlueFlame which writes the entire WQE into NIC registers for latency-sensitive small-queue workloads, and a CPU-assisted mode for cases where GPU writes are not optimal.

Integration status

NCCL GIN has integrated the open-source GPUNetIO Verbs path as a backend since version 2.27, letting device-side collective algorithms drive RDMA directly from the GPU. NVSHMEM 3.7 introduced a GPUNetIO-based transport that reduces implementation complexity while preserving IBGDA performance, with benchmarks showing better CTA and QP scaling for small messages. NVQLink uses the GPUNetIO-based GPU RoCE Transceiver operator to achieve approximately 2.6 microseconds minimum round-trip latency for quantum-classical workflows on IGX Thor with Blackwell GPU and ConnectX-7.

For inference engineers, the practical effect is less duplicated code across the stack and a more consistent path for GPU-initiated communication, which matters when building distributed serving systems where host-side networking adds latency on the critical path.