NVIDIA's BEVPoolV3 Cuts BEV Pooling Latency With Memory-Regime-Specific Kernels

NVIDIA published a case study on June 24, 2026, showing how BEVPoolV3, a new BEV pooling operator, cuts latency on two RTX GPUs by adapting the kernel to the memory regime. The operator is exposed as a TensorRT IPluginV3 plugin and targets autonomous vehicles, robotics, and spatial AI systems that use bird's-eye-view perception to project multicamera features into a shared top-down grid.

The canonical configuration, derived from real nuScenes samples, contains about 209K scatter points, 80 feature channels, and a 49 MB working set. That working set exceeds the 6 MB L2 cache on the RTX A6000, making the workload DRAM-bound. On the RTX PRO 6000 Blackwell Max-Q, which has a 128 MB L2 cache, the same working set fits in L2 after the initial fill, so the kernel is largely L2-resident.

Memory regime drives optimization strategy

The fit or no-fit decision changes the optimization target. On the RTX A6000, the small-L2 path prioritizes byte reduction and cache-streaming output stores. On the RTX PRO 6000 Blackwell Max-Q, the large-L2 path shifts toward instruction efficiency, occupancy, precomputed indices, vectorized loads, and FP8 specialization.

BEVPoolV3 builds on prior work. BEVPoolV2, referred to as V2, introduced an efficient deployment-oriented formulation. CUDA-BEVFusion includes bevpool_half_pack10_kernel, referred to as V2+DO, which uses depth-outer traversal to remove repeated tile-outer index loading. BEVPoolV3 adds four changes: reduced duplicate depth loads, a five-array INT32 scatter map, precomputed indices that remove runtime integer division, and interval-owned output writes that avoid atomic operations.

The five-array scatter map uses ranks_depth, ranks_feat, ranks_bev, interval_starts, and interval_lengths. Packing the first three into an int3 array gives a 12-byte record that is inconvenient for aligned memory transactions and does not map cleanly to a 16-byte load. Separate INT32 arrays let adjacent threads merge aligned loads and avoid field coupling.

Measured results on two GPUs

In the canonical configuration, the V2-style TensorRT plugin path takes 274.0 µs on the RTX PRO 6000 Blackwell Max-Q. BEVPoolV3 reduces that to 17.3 µs in FP16 and 16.4 µs in FP8. On the RTX A6000, the DRAM-adapted BEVPoolV3 FP16 path reaches 90.0 µs, versus 1,738.0 µs for V2 FP16. The speedup over V2 FP16 is 19.31x on the RTX A6000, 15.84x on the RTX PRO 6000 Blackwell Max-Q in FP16, and 16.71x in FP8.

The workflow is repeatable for other gather- or scatter-heavy operators such as sparse embeddings, voxelization, histograms, and segmented reductions. NVIDIA recommends profiling the operator with Nsight Compute to identify the active bottleneck and memory regime, reviewing TensorRT documentation for IPluginV3 integration, and consulting the CUDA C++ Programming Guide for kernel optimization techniques matching the GPU's memory hierarchy. The implementation is validated against an FP64 reference and V2 paths with atol=1e-2.