ONNX Runtime 1.30.0 adds INT4 paged KV caches, Go bindings, and CPU FP16 fallback

ONNX Runtime 1.30.0 adds INT4 paged KV caches with per-channel scales to the CUDA PagedAttention operator, extends paged XQA with speculative decoding, and introduces variable-length causal convolution for continuous batching. The release also adds Go bindings for the C API and gates CPU FP16 Gemm and MatMul execution on hardware acceleration, with FP32 fallback for unsupported nodes.

Generative AI on CUDA

The CUDA execution provider now supports INT4 paged KV caches with per-channel scales, plus an is_causal attribute on PagedAttention. Paged XQA gains speculative decoding, query-to-KV head group size 6, head size 256 including FP16 caches, and native block tables for 128-token pages. Split-KV is enabled for paged FlashAttention decode, and PagedAttention dispatch diagnostics were improved.

A new VarlenCausalConvWithState operator supports continuous batching with compact variable-length causal-convolution state updates. The GatedDeltaNet operator gained a compact variant and BFloat16 support on CUDA. For MoE workloads, FP4 QMoE kernels are now enabled by default in CUDA builds, with Windows build support added. An opt-in FP8 DeepGEMM MoE decode path for fixed-shape QMoE workloads on Hopper GPUs is available via the ORT_QMOE_FP4_DEEPGEMM environment variable, default off and disabled on Windows.

CUDA fpA-intB builds now default to a compact kernel set for FP16 activations, INT4 and INT8 weights, scale-only quantization, and block size 32. Source builds can retain the full kernel set, including BF16, zero-point, bias, larger block sizes, and native Hopper variants, by setting the onnxruntime_USE_FPA_INTB_GEMM_FULL option to ON.

CPU and WebGPU changes

Fused CPU LinearAttention kernels were added for AVX-512, Arm64 NEON, and SVE, along with AVX2 acceleration for LayerNorm and RMSNorm. CPU FP16 Gemm and MatMul execution is now gated on hardware acceleration, and CPU-assigned FP16 nodes without a matching kernel fall back to FP32.

WebGPU PagedAttention was improved with GPT-OSS support and INT8 KV-cache block quantization, plus extended convolution optimizations. The WebGPU plugin EP packaging now supports Linux AArch64, with plugin versions advanced to WebGPU 0.4.0 and CUDA 0.2.

Reliability and tooling

The release hardens model loading by limiting nested model-graph depth and canonicalizing external-data locations. Shape, rank, and parameter validation was strengthened for Split, Scan, GatherND, ScatterND, SpaceToDepth and DepthToSpace, Crop, Conv, Normalizer, and pooling operators. Generation and attention input handling was hardened, including attention-attribute narrowing, BifurcationDetector inputs, generation subgraph shapes, and QEmbed segment inputs. BeamSearch buffer expansion now uses dynamic shape storage.

CUDA indexing and buffer-size arithmetic was hardened in MatMulNBits, RemovePadding, RotaryEmbedding, SparseAttention, Whisper beam search, NMS, QDQ, and GatherElements. Fixed overflow in CUDA reduction scans and Softmax offset arithmetic, and zero-sized outputs in CUDA random-generator kernels are handled. CUDA MultiHeadAttention shared-cache scratch lifetimes were fixed, and CudaAsyncBuffer staging storage stays alive across CUDA graph replay. WebGPU out-of-bounds subgroup-matrix loads for partial tiles were fixed, writable device-allocator buffers are zero-initialized, and foreign GPU handles are rejected in built-in data transfers.

An allowlist of safe LoRA adapter parameter data types was added, MatMulFpQ4 shape inputs are validated, and MLAS blockwise quantization and dequantization index ranges are checked. Protobuf was upgraded to 33.6, and JavaScript dependencies including js-yaml, joi, and fast-uri were updated. GitHub Actions are pinned to full-length commit SHAs, and packaging infrastructure uses authenticated package feeds and NPM network isolation.