ONNX Runtime 1.29.0 adds PagedAttention, NVFP4 QMoE, and WebGPU deprecations

ONNX Runtime 1.29.0, published on August 12, 2026, introduces PagedAttention with quantized KV cache on the CUDA execution provider, NVFP4 QMoE support, and a range of WebGPU operator additions. The release also deprecates WebGL and JSEP in onnxruntime-web, directing users to the native WebGPU execution provider as the recommended path. Security fixes address path traversal in TensorRT engine refitting and add validation for numerous operators.

Attention and MoE on CUDA

The CUDA execution provider gains PagedAttention with quantized KV cache, XQA decode, MLA, QK-Norm, and head-sink support. Quantized KV-cache support now includes attention sinks, independent and per-channel scales, sliding-window cache, and a fused K/V dequantization launch. A cuDNN SDPA decode tier was added to the standard ONNX Attention kernel, and cuDNN SDPA is enabled for the contrib Attention operator. The GroupQueryAttention unfused path supports attention_bias, and LinearAttention and CausalConvWithState support state_window for multi-token prediction.

For mixture-of-experts models, NVFP4 QMoE includes native FP4xFP4 prefill on SM120, faster decode GEMV, fused routing and finalization paths, and reduced activation and weight-dequantization overhead. New MatMulBlockQuantizedFp4Weight and MatMulBlockQuantizedFp8Weight operators add block-scaled tensor-core and GEMV decode paths, packed FP4 decode, M-tiling, and folded W8A8 activation QDQ. MatMulNBits and QMoE robustness improved with optimized 8-bit dequantization and release of raw MXFP4 initializers after prepack.

New contrib operators include LinearAttentionGate, GatedRMSNorm, and GatedAdd. AllReduce, AllGather, and AllToAll now support bfloat16. The default zero point in CUDA GatherBlockQuantized was fixed.

WebGPU expansion and deprecations

WebGPU adds DFT, HardSwish, Max/Min, Trilu, GRU, PRelu, MatMulBnb4, and MRotaryEmbedding. Integer support expanded across Clip, Reshape, Cast, Add, Tile, Concat, Expand, Gather, CumSum, Max, and Min. An initial WebGPU PagedAttention implementation is included, and Softmax and non-flash Attention moved to online algorithms. MatMulNBits wide-tile accumulation precision improved. Intel subgroup-matrix MatMul and Gemm kernels gained f16, batched-B, and odd-N support. Cold-start and upload overhead reduced with deferred dispatch and staging-buffer improvements.

onnxruntime-web deprecates WebGL and JSEP, with the native WebGPU execution provider as the recommended path. WebGPU also supports device-free compile-only sessions for offline graph transformation.

Runtime and security updates

Default intra-op and inter-op thread-pool sizes can be set with ORT_INTRA_OP_NUM_THREADS and ORT_INTER_OP_NUM_THREADS, with explicit thread settings taking precedence. Weightless-model support covers all initializer types, zero-input EpContext nodes are allowed, and maximum-shape inference wires into workspace estimation. ONNX-domain rotary embedding and a fused MRotaryEmbedding contrib operator for Qwen mRoPE variants were added. onnxruntime_perf_test supports multi-shape profiling through data_shape.

Security fixes include unconditional external-data path validation for TensorRT and NvTensorRTRTX engine refitting, CPU MoE k attribute validation against expert count, and a CPU TensorScatter fix. Missing rank, shape, and parameter validation was added for pooling, LSTM, DynamicQuantizeLSTM, Sampling, FeatureVectorizer, SkipLayerNorm, QLinearConv, Whisper decoding, RNN activations, GridSample, contrib Range, and CropAndResize. CUDA indexing and buffer handling hardened in GridSample, transpose, GatherBlockQuantized, InstanceNormalization, LayerNorm and RMSNorm, BeamSearch, DeformConv, AveragePool, and MaxPool. Packed sub-byte tensor over-copying in OrtApi::GetValue was fixed, and DML constant tensor byte sizes validated. Supply chain updates include npm lockfile refreshes and an adm-zip upgrade for onnxruntime-node.

POSIX telemetry is now available on Linux, macOS, Android, and iOS when built with telemetry enabled, with ORT_DISABLE_TELEMETRY=1 disabling it per process. The unused internal TensorRT dashboard tooling was removed without affecting TensorRT execution provider APIs.