ONNX Runtime WebGPU Plugin EP v0.4.0 adds fused activations, int8 KV cache

The ONNX Runtime WebGPU Plugin EP v0.4.0 release focuses on kernel performance, broader operator coverage, and correctness fixes for WebGPU-based inference. The update targets developers running ONNX models in browsers or WASM environments, with several changes that affect how models are compiled and executed on GPU hardware.

Performance and operator updates

Performance work in this release includes optimized MatMulNBits wide-tile execution using subgroup shuffle, which reduces data movement for quantized matrix multiplications. The plugin now supplies fused activation parameters to Conv and MatMul as uniforms, and it fused eight more activations into Conv, cutting kernel launches. The im2col Conv path also gained fused activation support, and Split was vectorized when every output segment is vec4-aligned, improving memory throughput.

Subgroup matrix MatMul now shares with pointwise Conv, and the subgroup size is pinned to 32 for subgroup-matrix MatMul and Gemm, ensuring consistent behavior across devices. Pooling path selection now uses occupancy instead of output size, which can better utilize GPU resources. Conv weights are prepacked for the im2col-matmul path, reducing runtime overhead. Dawn pipeline compilation workers scale with CPU count, speeding up shader compilation on multi-core systems.

For attention workloads, the release adds PagedAttention metadata and GPT-OSS support, plus continued PagedAttention optimizations. It also introduces int8 KV cache block quantization, which can lower memory usage for long-context inference.

New operator support includes WebGPU subgroup-size-control infrastructure, enabling the subgroup matrix path for WASM builds. More kernels were added for Qwen-3.5 ops and DeepSeek Engram contrib ops, specifically EngramGate and NGramHashMapping, expanding model compatibility.

Reliability and packaging

Correctness fixes address a 1D-dispatch shader fast path bug, out-of-bounds subgroup-matrix loads for trailing partial tiles, and a pipeline cache key for MatMulNaiveProgram that now covers all WGSL inputs. The plugin now reports copy_tensors misuse instead of terminating the process, and it rejects foreign GPU handles in the built-in data transfer, improving robustness in multi-GPU or cross-context scenarios.

On the build side, Linux AArch64 WebGPU plugin EP packages are now available, extending deployment options to ARM-based servers. The onnxruntime_providers compilation was separated from its archive, and wgsl_template Python tests run in CI, which should catch shader template issues earlier.

For inference engineers, the fused activations and int8 KV cache quantization are the most impactful changes, as they can reduce kernel count and memory footprint for transformer models. The subgroup optimizations matter for WASM targets, where WebGPU support is still evolving. The Linux AArch64 packages open the door to edge devices and ARM servers running WebGPU inference.

The release notes list contributions from 12 developers and say they were drafted with AI assistance from commit history and PR metadata, then reviewed before publishing. They include no benchmark numbers.