---
title: 'MLIR-AIR: MLIR-Based Hardware Compilation'
url: https://www.emergentmind.com/topics/mlir-air
type: topic
---

# MLIR-AIR: MLIR-Based Hardware Compilation

Searching arXiv for recent MLIR-AIR papers and related verification/compiler work.
MLIR-AIR denotes a line of MLIR-based compiler infrastructures for hardware-oriented compilation in which high-level programs are progressively lowered into representations that expose scheduling, memory movement, hardware structure, and host–device interaction. In the published literature, the term spans both an end-to-end MLIR-to-ASIC/FPGA flow built from SYCL, CIRCT, Calyx, and SystemVerilog, and a later AMD compiler stack centered on the AIR dialect for mapping structured loop nests onto NPUs. Across these usages, the common design principle is to exploit MLIR’s multi-level, multi-dialect organization so that software-oriented IR, hardware-oriented IR, and explicit synchronization constructs coexist in one lowering framework [2401.10249; 2510.14871].

## 1. Historical scope and terminological usage

The earlier usage appears in "Building a Reusable and Extensible Automatic Compiler Infrastructure for Reconfigurable Devices," which presents a consolidated MLIR-to-ASIC/FPGA flow developed as an end-to-end framework that leverages open-source, cross-platform compilation technology to generate MLIR from SYCL, lowers customized MLIR to RTL through CIRCT and Calyx, and couples the generated hardware module with the host CPU using vendor-specific crossbars [2401.10249]. In that setting, the emphasis is on reconfigurable devices, RTL generation, and FPGA deployment.

A later usage appears in "From Loop Nests to Silicon: Mapping AI Workloads onto AMD NPUs with MLIR-AIR," which introduces MLIR-AIR as a novel, open-source compiler stack built on MLIR that bridges the semantic gap between high-level workloads and fine-grained spatial architectures such as AMD’s NPUs. Its AIR dialect provides structured representations for asynchronous and hierarchical operations across compute and memory resources, and the lowering path continues through MLIR-AIE to hardware binary plus runtime [2510.14871].

This suggests that MLIR-AIR is best understood not as a single fixed implementation, but as a family of MLIR-centric approaches organized around explicit hardware-aware IR design. A common misconception is to treat the term as naming only the AMD NPU stack; the published record also uses it for a SYCL-to-RTL compiler flow targeting FPGA-style deployment. Conversely, treating the earlier FPGA flow and the later AIR-dialect stack as unrelated would obscure their shared reliance on MLIR as the organizing substrate for multi-stage hardware compilation.

## 2. End-to-end compilation architecture

In the reconfigurable-device flow, the front end begins with a SYCL kernel written in C++, such as matrix multiply, compiled by the Intel DPC++ compiler, a SYCL-aware fork of Clang/LLVM. DPC++ emits MLIR targets that interoperate with the CIRCT stack. The pipeline then ingests DPC++-generated MLIR combining standard MLIR core dialects—`arith`, `memref`, `affine`, and `func`—with CIRCT hardware dialects. CIRCT passes convert the software dialects into a Calyx hardware dialect, after which the Calyx toolchain lowers its IR into synthesizable SystemVerilog modules. Those SystemVerilog files, together with a small testbench, are passed to vendor EDA, specifically Vivado on a ZCU106 board, which generates timing-accurate simulations and performs synthesis and place-and-route. A tiny C driver and an AXI-full wrapper are then used to boot the FPGA and evaluate end-to-end performance on the board [2401.10249].

The flow can therefore be summarized as:

`SYCL (*.cpp) → DPC++ → MLIR → (CIRCT passes) → Calyx IR → (Calyx) → SystemVerilog → Vivado → bitstream + simulation`

In the AMD NPU compiler stack, the architectural picture is broader at the front end and more spatial at the back end. Front-ends include PyTorch via Torch-MLIR, TensorFlow via TOSA, and Triton-Shared, all feeding MLIR SCF/Linalg. High-level MLIR is lowered into the AIR dialect through constructs such as `air.launch`, `air.segment`, `air.herd`, `air.memcpy`, `air.channel`, and async tokens. AIR then lowers to MLIR-AIE, where tile-local compute, DMA engines, locks, and streaming interconnect are made explicit, and MLIR-AIE lowers further to hardware binary plus runtime through XRT on AMD-NPU [2510.14871].

The architectural distinction between the two flows is consequential. The SYCL-to-RTL path is centered on generating synthesizable hardware modules and host attachment through AXI infrastructure. The AMD AIR path is centered on exposing explicit spatial scheduling, distributed data movement, and asynchronous dependencies so that a compiler can orchestrate an existing spatial architecture rather than synthesize a new RTL design from scratch.

## 3. Dialects, primitives, and operational semantics

The earlier flow emphasizes standard MLIR dialects and their mapping into hardware. The `memref` dialect is used for multi-dimensional buffers, with types of the form `memref<d₀×d₁×…×dₙ × element_type, memory_space>`. An explicit example is `memref<1024×i32, 1>`, which denotes a 1 KiB, 32-bit-wide buffer in `memory_space 1`. The `arith` dialect contributes operations such as `arith.addi` and `arith.muli`, with semantics `r := x + y` and `r := x · y`. The `affine` dialect provides loop nests with affine bounds through `affine.for`, including `iter_args` and `yield` semantics. On the CIRCT/Calyx side, a new Calyx hardware dialect defines components such as `CombCell`, `SeqCell`, `Memory`, and `Groups`, and each MLIR operation is mapped to one or more Calyx primitives [2401.10249].

The later AIR stack introduces a more specialized dialect organization. AIR primitives are divided into scheduling constructs, data-locality constructs, and synchronization constructs. `air.launch` offloads a parameterizable N-D iteration space to the device and returns an async token that resolves when all nested work completes. `air.segment` reserves a subset of compute resources such as cores or tiles for nested herds. `air.herd` defines a logical grid of workers, such as a `2×2` herd of cores, where all workers execute the body in parallel with local indices and are atomically scheduled. `air.memcpy` denotes a logical multi-dimensional tensor copy with layout and memory-space attributes, while paired `air.channel.put` and `air.channel.get` implement decoupled, stream-based DMA injection and consumption with back-pressure automatically synchronizing the two ends. Async regions produce `!air.async.token`, and dependencies are expressed either through explicit dependency lists or via `air.wait_all` [2510.14871].

A common metric in the AIR literature is compute efficiency for matrix multiplication of size $M\times N\times K$, defined using the total floating-point operations $2MNK$, execution time $T_{exec}$, and theoretical peak FLOPS [2510.14871]. The emphasis is not merely on representing computation, but on representing orchestration: placement, dependency structure, and communication–computation overlap are first-class IR concerns rather than implicit consequences of later backend scheduling.

## 4. Lowering methodology and hardware realization

In the reconfigurable-device flow, lowering is organized as a chain of transformations $T_{\text{mapping}}:\mathrm{IR}_{in}\rightarrow \mathrm{IR}_{out}$. `arith.addi` is lowered to a `CombCell` with two input ports and one output port, with `CombCell` semantics `r := x + y in one cycle`. `memref` allocations become Calyx `Memory` components with entries of `element_type` and read/write ports. `affine.for` with statically known bounds is lowered into a Calyx `Group` plus control, either as a purely sequential schedule or as an unrolled schedule by factor $U>1$ that generates parallel copies of the body and adjusts indices accordingly. The call graph is then wrapped into a top-level Calyx component `Top` with an AXI4-full slave interface. After these passes, the IR is purely in CIRCT’s Calyx dialect [2401.10249].

The Calyx compiler then flattens each hardware component into Verilog modules. `CombCell` becomes `always_comb`, `SeqCell` becomes `always_ff @(posedge clk)` with optional reset, `Memory` becomes inferred BRAM or register arrays, and the control FSM is encoded as either a `case` statement or a one-hot state machine. Two key optimizations are identified: loop unrolling, which reduces control overhead and exposes ILP at the cost of larger area, and resource sharing via time-division multiplexing, where a single `CombCell` serves all iterations when `unroll_factor = 1` [2401.10249].

In the AMD AIR stack, the lowering strategy is more explicitly spatial. MLIR’s generic tiling utilities lower a 2D Linalg matmul into nested `scf.for` and `scf.parallel` loops. A pass then converts outermost `scf.for_all` to `air.launch + air.herd`, parameterized by tile sizes $\{t_i,t_j,t_k\}$. Data movement is exposed by hoisting memcpy into `air.channel.put/get` around the herd, enabling the host and herd to run asynchronously in ping-pong style. Dependency constraints are stated explicitly, and AIR tokens encode RAW/WAR/WAW relationships that drive distributed scheduling [2510.14871].

Host attachment is also explicit in the FPGA-oriented flow. Once `Top.sv` is available, an IP core with an AXI4-Full slave interface is generated. On Xilinx ZCU106, the connectivity is `PS (ARM CPU) ↔ AXI Interconnect (Xilinx crossbar) ↔ PL (our “Top” IP)`. The register map conforms to Xilinx XRT conventions, with a control register at offset `0`, status at `0x04`, and argument buffers at `0x10…`. The host-side C driver `mmap()`s the AXI region, writes input operands into DRAM or streams them via DMA, starts execution by poking the control register, polls the status register, and reads back results. This uses vendor-provided crossbar IP to route AXI transactions transparently from the processing system to the accelerator [2401.10249].

## 5. Case studies and performance characteristics

The reconfigurable-device study uses GEMM, $C=A\cdot B$, for matrix sizes from `4×4` through `128×128`, comparing two variants: nested `for` loops with full resource sharing, and an inner-loop fully flattened or unrolled implementation with maximum parallelism. The reported clock cycle counts assume a `1 ns` clock [2401.10249].

| Matrix size | Nested cycles | Unrolled cycles |
|---|---:|---:|
| 4×4 | 1 498 | 1 114 |
| 8×8 | 10 762 | 7 946 |
| 16×16 | 81 802 | 60 298 |
| 32×32 | 867 594 | 470 282 |
| 64×64 | 5 042 698 | 3 527 115 |
| 128×128 | 38 324 504 | 26 806 047 |

For the `32×32` case on ZCU106, the nested time-division-multiplexed configuration uses approximately `~8 000` LUTs, `~5 000` DFFs, and `16` DSPs, while the inner-flattened configuration uses approximately `~32 000` LUTs, `~20 000` DFFs, and `256` DSPs. The paper states that full unrolling drastically reduces latency, up to `~45 %`, at the expense of area, whereas the time-sharing design is area-efficient but slower [2401.10249].

The AMD NPU work presents two case studies. For matrix multiplication in `bf16`, herd shapes of `2×2`, `2×4`, and `4×4` correspond respectively to `1 TOP/s`, `2 TOP/s`, and `4 TOP/s` peak configurations. The achieved throughputs are `0.482 TOP/s (48.2 %)` for `2×2`, `1.300 TOP/s (65.0 %)` for `2×4`, and `1.943 TOP/s (48.6 %)` for `4×4`. On Pareto fronts for a `4×4` herd with sweep `M=N=K up to 4096`, the reported efficiencies are `78.7 % (MLIR-AIR) vs. 77.7 % (MLIR-AIE) vs. 76.6 % (ARIES)` for `i16`, `48.6 % vs. 50.8 % vs. 49.1 %` for `bf16`, and `59.1 % vs. 60.6 % vs. 56.2 %` for `i8`. The comparison is summarized in the paper as MLIR-AIR tracking hand-tuned MLIR-AIE implementations within `5 percentage-points`, while starting from a generic loop nest [2510.14871].

The second case study is a fused multi-head attention block from LLaMA 2 expressed in approximately `150 lines of AIR code`, targeting a single AIE core with `head size = 48` and `seq-len = 256`. The mapped stages are Q/K/V projection, RoPE, `Q·Kᵀ` matmul plus `invsqrt`, softmax, and final `softmax·V` matmul, with the KV cache implemented via persistent shared memory in DDR using `air.channel`. The latency breakdown per head is `169 µs` for Q/K/V projection, `121 µs` for RoPE, `283 µs` for matmul plus `invsqrt`, `115 µs` for softmax, and `146 µs` for the final matmul, for a total of `834 µs` as five kernels. The fused single-kernel version is reported at `373 µs`, a `2.24×` speedup. The listed trade-offs are elimination of host dispatch overhead of approximately `10–20 µs per kernel`, improved data locality, and a single-kernel binary that is larger and slightly less modular [2510.14871].

## 6. Formal verification, correctness conditions, and stated limitations

Formal verification enters the MLIR-AIR literature through "Practical Formal Verification for MLIR Programs," which reports on verification of the AMD MLIR-AIR and MLIR-AIE toolchains using PEQC-MLIR [2605.01124]. The verifier is defined around a small, formally specified parallel proxy language with small-step operational semantics
$$
\sigma \xrightarrow[]{n} \sigma',\quad \sigma \to \sigma'
$$
over states $\sigma=(M,G)$, where $M$ is memory and $G$ is a happens-before graph, or over `error(G)`. Program termination is written as
$$
P \Downarrow \sigma \Longleftrightarrow (M_0^P,G_0^P)\xrightarrow{*}\sigma \quad\text{and }\sigma\not\to\sigma'.
$$
Two programs are strictly equivalent if they terminate in final states with equal memories, although in practice the requirement is equality only on non-local live-out variables.

PEQC-MLIR accepts a restricted MLIR subset consisting of `func`, `arith`, `math`, `memref`, `scf`, `async`, and a minimal `verif` dialect with `verif.semaphore` and operations `wait`, `set`, `acquire`, and `release`. The restrictions are explicit: no recursion, no dynamic memory, no symbolic loop bounds, and no array subscripts that evaluate to a symbolic expression. These constraints are used to preserve a deterministic small-step semantics, a decidable next enabled statement per task, and a terminating construction of the global happens-before graph. Confluence is stated as Theorem 3.7 and determinism as Theorem 3.9 [2605.01124].

The interpreter is hybrid concrete–symbolic. Each executed statement produces either a concrete computation, a symbolic AST node represented in a Computation DAG, or an error if a condition such as a loop guard or array index fails to evaluate concretely. A global validity test `graph(G)` checks race-freedom and synchronization correctness for `set/wait` and `acquire/release`. Because each executed statement generates at most one new CDAG node, with hash-consing for repeated subexpressions, the stated complexity is linear in the number of executed statements: time $O(N)$ to build the final memory and graph, space $O(N)$ for the CDAG, and equivalence checking again $O(N)$ by comparing sorted lists of CDAG node hashes at each live-out cell [2605.01124].

Integration into the AMD AIR pipeline is implemented through three converter passes in the `verif-opt` plugin: `air.execute → async.execute`, `air.herd → scf.parallel`, and `air.channel → memref+verif`. In the C-AST generator, AIR async and synchronization constructs become calls into the PEQC-MLIR API. On a `32×32` matrix multiply, four intermediate AIR outputs—`linalg` at `8 LoC`, `air.input` at `51 LoC`, `air.tiled` at `133 LoC`, and `air.placed` at `902 LoC`—expand after MLIR→C→PIR translation to `201 870 → 1 320 799` statements, with up to `77815` synchronization operations and a happens-before graph ratio of at most `0.39`. Full interpretation times range from `1.6 s` to `9.0 s`, and equivalence checks remain at approximately `0.04 s`. The verifier also found a subtle bug in MLIR-AIE lowering, flagging the non-determinism in `5 s`, compared with `31 s` for the corrected version [2605.01124].

The stated limitations and future directions are correspondingly concrete. In the verification work, ongoing directions include CDAG compaction via affine folding, support for parametric loop bounds using symbolic control predicates, integration with a formally verified lowerer for `AIR→std→C`, and a richer rewrite-rule library or use of `egg` [2605.01124]. In the compiler flows themselves, stated future extensions include support for dynamic memory, floating-point types, richer MLIR tensor dialects such as `linalg`, replacement of Xilinx’s XRT with a lightweight portable runtime for low-power FPGAs, multi-NPU or global scheduling, AIR→ROCr lowering for heterogeneous dispatch, dynamic multi-device `air.launch` semantics, and performance-guided auto-tiling with cost-model–driven schedule search [2401.10249; 2510.14871].

Source: https://www.emergentmind.com/topics/mlir-air