llama.cpp b11623: SIMT implicit GEMM conv2d on AMD without WMMA

llama.cpp b11623 introduces a SIMT implicit GEMM convolution path for AMD GPUs that does not rely on WMMA instructions. The change, merged as pull request #30325, adds a fallback for convolution layers on AMD hardware where WMMA is unavailable or less efficient. A subsequent update skips this SIMT path on AMD for short reductions, presumably to avoid overhead when the reduction dimension is small.

The release notes also mention a comment update for the SIMT conv2d implementation, and the work was co-authored with thelittlefireman. The build matrix includes a notable change: the macOS Apple Silicon build with KleidiAI enabled is now marked as DISABLED, meaning that variant is no longer shipped in this release. Other assets remain available, including CPU builds for Ubuntu, Windows, and Android, plus GPU backends such as CUDA 12 and 13, Vulkan, ROCm 10.0, OpenVINO, SYCL, and Snapdragon-specific builds for CPU, Adreno GPU, and Hexagon NPU.

What this means for inference engineers

For those serving models with convolutional layers, such as vision models or hybrid architectures, the new SIMT path provides an alternative on AMD GPUs. Without WMMA, the implementation uses a SIMT approach that may improve performance or enable execution on hardware where WMMA is not supported. The skip for short reductions suggests the developers tuned the path to avoid cases where the overhead outweighs the benefit, so users on AMD should test with their specific workloads to see if the new path activates and whether it helps.

The disabled KleidiAI build on macOS is a regression for users who relied on that optimized ARM build. KleidiAI typically accelerates matrix multiplication on ARM CPUs, so its absence in this release may affect performance on Apple Silicon for CPU inference. Users on macOS should check if the standard arm64 build meets their needs or wait for a future release to re-enable KleidiAI.

Release details

The release includes a wide range of prebuilt binaries: Ubuntu x64 and arm64 for CPU, Vulkan, CUDA 12.8, CUDA 13.4, ROCm 10.0, OpenVINO, and SYCL FP32/FP16. Windows builds cover CPU, arm64 CPU, OpenCL Adreno, CUDA 12.4, CUDA 13.4, Vulkan, OpenVINO, SYCL, and ROCm 10.0. Android builds include arm64 CPU and Snapdragon variants. The openEuler builds for Ascend 310p and 910b are listed but marked as DISABLED, indicating they are not included in this release. A UI asset is also provided.

For inference engineers, the key takeaway is the new AMD convolution path and the macOS KleidiAI change. Testing on AMD hardware with convolutional models is recommended to assess any performance impact. The release does not include benchmark numbers, so empirical evaluation is necessary.