Modular integrates Qualcomm Cloud AI 100 into MAX and Mojo stack

Modular announced at its ModCon keynote on August 18, 2026, that Qualcomm Technologies' data center AI accelerators are now integrated into the Modular Platform. Developers can target the Qualcomm Cloud AI 100 Ultra NPU using the same MAX and Mojo code that runs on NVIDIA and AMD GPUs, with no rewrite. This is the first ASIC, or NPU, to be integrated into the Modular stack, and it follows Qualcomm's acquisition of Modular Inc.

The integration covers the Qualcomm Cloud AI 100, extending to the Dragonfly AI 200 and future generations. The same abstractions and developer experience apply across hardware. The companies share a thesis: developers should pick the best hardware for their workload without rewriting software. Qualcomm brings decades of power-efficient compute expertise to purpose-built inference silicon, while Modular provides a portable software layer.

Native bring-up in weeks

Engineers from both companies worked side by side to integrate the Qualcomm Cloud AI 100 Ultra into MAX and Mojo. Within weeks of getting matrix multiplication kernels running on the device, they had GPT-2 executing end to end through MAX, using the same high-level model definition and host code paths used for NVIDIA and AMD GPUs. From the first Gemma 4 kernel work to a live serving endpoint, the bring-up took under six months.

This is a native integration, not bolted on. Mojo compiles to the Qualcomm Hexagon LLVM NPU backend. Hardware capabilities map directly into the Mojo standard library through the same abstractions used for GPU programming. The Qualcomm Cloud AI 100 plugs into the graph compiler, serving framework, and runtime via MAX's device driver connector, just like NVIDIA, AMD, and Apple Metal GPUs.

The bring-up progressed in stages: foundation work to enable the Mojo compiler and graph compiler and write first kernels; a GPT-2 hello world pipeline with reproducible numbers; scaling to a Gemma 4 31B model running across four chips, served live via max serve; and a final phase where new models and features went from weeks to days. The team went from Mojo works to Gemma 4 serving in under a quarter.

Hardware and memory model

The Qualcomm Cloud AI 100 Ultra has 4 devices with 32 GB of RAM each, made up of 16 NSPs. The NSPs are multi-threaded, orchestrating vector (HVX), DMA, and matrix (HMX) units. The HVX vector engine handles wide SIMD vector and elementwise work, mapping to upstream LLVM Hexagon vector intrinsics. The HMX matrix engine does the heavy lifting for GEMM and attention.

The memory model differs from a GPU: there is no unified memory. The device has separate on-chip VTCM and off-chip DDR, with data staged explicitly via DMA. Good performance requires tiling work into VTCM and moving data deliberately, not relying on a cache hierarchy. The HMX unit requires a specific data layout called crouton layout, and the conversion has a cost, so developers must minimize conversion overhead. Sometimes it is better to stay on the HVX vector engine than pay the conversion cost.

Kernel dispatch and implementation

The Mojo compiler leverages upstream LLVM, using the existing Hexagon DSP backend for SIMD types and operations. HMX and DMA are exposed through bindings to the Mojo programming model. The DeviceContext API, which targets NVIDIA and AMD GPUs, now also targets the Qualcomm Cloud AI 100 JIT API. The QAIC JIT API creates command buffers on each NSP and enqueues operations for data copy, mapped to equivalent DeviceContext functionality. There were no fundamental changes to DeviceContext, except adapting to single-stream dispatch and converting GPU block/grids to NPU cores/threads.

A key challenge was kernel signatures. Mojo kernels use typed arguments, while the QAIC JIT API expects a contiguous pointer array. Instead of dedicated compiler passes, the team used Mojo's reflection and compile-time capabilities to convert kernel signatures purely in Mojo, simplifying implementation and improving compilation times.

Once basic kernel dispatch worked, a graph-compiler proof of concept mapped an elementwise op to a single kernel. The NPU is fundamentally different from a GPU: GPUs dispatch in grids and blocks, while the NPU works in cores and threads. The Qualcomm Cloud AI 100 is a SIMD device, not SIMT, so the kernel programming model is different. The integration is a proof point for a broader collaboration across Qualcomm's AI accelerator roadmap, including the Dragonfly AI200, AI250, and AI300.