MLIR-AIR: MLIR-Based Hardware Compilation

• MLIR-AIR is a family of MLIR-based compiler infrastructures that lower high-level programs into detailed hardware representations for FPGA and NPU targets.
• It integrates multi-level dialects and specialized constructs like air.launch and air.herd to handle asynchronous scheduling and explicit spatial mapping.
• Case studies show significant latency reductions and trade-offs between area and performance, with formal verification ensuring synchronization and resource correctness.

Searching arXiv for recent MLIR-AIR papers and related verification/compiler work. MLIR-AIR denotes a line of MLIR-based compiler infrastructures for hardware-oriented compilation in which high-level programs are progressively lowered into representations that expose scheduling, memory movement, hardware structure, and host–device interaction. In the published literature, the term spans both an end-to-end MLIR-to-ASIC/FPGA flow built from SYCL, CIRCT, Calyx, and SystemVerilog, and a later AMD compiler stack centered on the AIR dialect for mapping structured loop nests onto NPUs. Across these usages, the common design principle is to exploit MLIR’s multi-level, multi-dialect organization so that software-oriented IR, hardware-oriented IR, and explicit synchronization constructs coexist in one lowering framework (Zang et al., 2023, Wang et al., 16 Oct 2025).

1. Historical scope and terminological usage

The earlier usage appears in "Building a Reusable and Extensible Automatic Compiler Infrastructure for Reconfigurable Devices," which presents a consolidated MLIR-to-ASIC/FPGA flow developed as an end-to-end framework that leverages open-source, cross-platform compilation technology to generate MLIR from SYCL, lowers customized MLIR to RTL through CIRCT and Calyx, and couples the generated hardware module with the host CPU using vendor-specific crossbars (Zang et al., 2023). In that setting, the emphasis is on reconfigurable devices, RTL generation, and FPGA deployment.

A later usage appears in "From Loop Nests to Silicon: Mapping AI Workloads onto AMD NPUs with MLIR-AIR," which introduces MLIR-AIR as a novel, open-source compiler stack built on MLIR that bridges the semantic gap between high-level workloads and fine-grained spatial architectures such as AMD’s NPUs. Its AIR dialect provides structured representations for asynchronous and hierarchical operations across compute and memory resources, and the lowering path continues through MLIR-AIE to hardware binary plus runtime (Wang et al., 16 Oct 2025).

This suggests that MLIR-AIR is best understood not as a single fixed implementation, but as a family of MLIR-centric approaches organized around explicit hardware-aware IR design. A common misconception is to treat the term as naming only the AMD NPU stack; the published record also uses it for a SYCL-to-RTL compiler flow targeting FPGA-style deployment. Conversely, treating the earlier FPGA flow and the later AIR-dialect stack as unrelated would obscure their shared reliance on MLIR as the organizing substrate for multi-stage hardware compilation.

2. End-to-end compilation architecture

In the reconfigurable-device flow, the front end begins with a SYCL kernel written in C++, such as matrix multiply, compiled by the Intel DPC++ compiler, a SYCL-aware fork of Clang/LLVM. DPC++ emits MLIR targets that interoperate with the CIRCT stack. The pipeline then ingests DPC++-generated MLIR combining standard MLIR core dialects—arith, memref, affine, and func—with CIRCT hardware dialects. CIRCT passes convert the software dialects into a Calyx hardware dialect, after which the Calyx toolchain lowers its IR into synthesizable SystemVerilog modules. Those SystemVerilog files, together with a small testbench, are passed to vendor EDA, specifically Vivado on a ZCU106 board, which generates timing-accurate simulations and performs synthesis and place-and-route. A tiny C driver and an AXI-full wrapper are then used to boot the FPGA and evaluate end-to-end performance on the board (Zang et al., 2023).

The flow can therefore be summarized as:

SYCL (*.cpp) → DPC++ → MLIR → (CIRCT passes) → Calyx IR → (Calyx) → SystemVerilog → Vivado → bitstream + simulation

In the AMD NPU compiler stack, the architectural picture is broader at the front end and more spatial at the back end. Front-ends include PyTorch via Torch-MLIR, TensorFlow via TOSA, and Triton-Shared, all feeding MLIR SCF/Linalg. High-level MLIR is lowered into the AIR dialect through constructs such as air.launch, air.segment, air.herd, air.memcpy, air.channel, and async tokens. AIR then lowers to MLIR-AIE, where tile-local compute, DMA engines, locks, and streaming interconnect are made explicit, and MLIR-AIE lowers further to hardware binary plus runtime through XRT on AMD-NPU (Wang et al., 16 Oct 2025).

The architectural distinction between the two flows is consequential. The SYCL-to-RTL path is centered on generating synthesizable hardware modules and host attachment through AXI infrastructure. The AMD AIR path is centered on exposing explicit spatial scheduling, distributed data movement, and asynchronous dependencies so that a compiler can orchestrate an existing spatial architecture rather than synthesize a new RTL design from scratch.

3. Dialects, primitives, and operational semantics

The earlier flow emphasizes standard MLIR dialects and their mapping into hardware. The memref dialect is used for multi-dimensional buffers, with types of the form memref<d₀×d₁×…×dₙ × element_type, memory_space>. An explicit example is memref\<1024×i32, 1>, which denotes a 1 KiB, 32-bit-wide buffer in memory_space 1. The arith dialect contributes operations such as arith.addi and arith.muli, with semantics r := x + y and r := x * y. The affine dialect provides loop nests with affine bounds through affine.for, including iter_args and yield semantics. On the CIRCT/Calyx side, a new Calyx hardware dialect defines components such as CombCell, SeqCell, Memory, and Groups, and each MLIR operation is mapped to one or more Calyx primitives (Zang et al., 2023).

The later AIR stack introduces a more specialized dialect organization. AIR primitives are divided into scheduling constructs, data-locality constructs, and synchronization constructs. air.launch offloads a parameterizable N-D iteration space to the device and returns an async token that resolves when all nested work completes. air.segment reserves a subset of compute resources such as cores or tiles for nested herds. air.herd defines a logical grid of workers, such as a 2×2 herd of cores, where all workers execute the body in parallel with local indices and are atomically scheduled. air.memcpy denotes a logical multi-dimensional tensor copy with layout and memory-space attributes, while paired air.channel.put and air.channel.get implement decoupled, stream-based DMA injection and consumption with back-pressure automatically synchronizing the two ends. Async regions produce !air.async.token, and dependencies are expressed either through explicit dependency lists or via air.wait_all (Wang et al., 16 Oct 2025).

A common metric in the AIR literature is compute efficiency for matrix multiplication of size $M\times N\times K$, defined using the total floating-point operations $2MNK$, execution time $T_{exec}$, and theoretical peak FLOPS (Wang et al., 16 Oct 2025). The emphasis is not merely on representing computation, but on representing orchestration: placement, dependency structure, and communication–computation overlap are first-class IR concerns rather than implicit consequences of later backend scheduling.

4. Lowering methodology and hardware realization

In the reconfigurable-device flow, lowering is organized as a chain of transformations $$T_{\text{mapping}}:\mathrm{IR}_{in}\rightarrow \mathrm{IR}_{out}$$. arith.addi is lowered to a CombCell with two input ports and one output port, with CombCell semantics r := x + y in one cycle. memref allocations become Calyx Memory components with entries of element_type and read/write ports. affine.for with statically known bounds is lowered into a Calyx Group plus control, either as a purely sequential schedule or as an unrolled schedule by factor $U>1$ that generates parallel copies of the body and adjusts indices accordingly. The call graph is then wrapped into a top-level Calyx component Top with an AXI4-full slave interface. After these passes, the IR is purely in CIRCT’s Calyx dialect (Zang et al., 2023).

The Calyx compiler then flattens each hardware component into Verilog modules. CombCell becomes always_comb, SeqCell becomes always_ff @(posedge clk) with optional reset, Memory becomes inferred BRAM or register arrays, and the control FSM is encoded as either a case statement or a one-hot state machine. Two key optimizations are identified: loop unrolling, which reduces control overhead and exposes ILP at the cost of larger area, and resource sharing via time-division multiplexing, where a single CombCell serves all iterations when unroll_factor = 1 (Zang et al., 2023).

In the AMD AIR stack, the lowering strategy is more explicitly spatial. MLIR’s generic tiling utilities lower a 2D Linalg matmul into nested scf.for and scf.parallel loops. A pass then converts outermost scf.for_all to air.launch + air.herd, parameterized by tile sizes $\{t_i,t_j,t_k\}$. Data movement is exposed by hoisting memcpy into air.channel.put/get around the herd, enabling the host and herd to run asynchronously in ping-pong style. Dependency constraints are stated explicitly, and AIR tokens encode RAW/WAR/WAW relationships that drive distributed scheduling (Wang et al., 16 Oct 2025).

Host attachment is also explicit in the FPGA-oriented flow. Once Top.sv is available, an IP core with an AXI4-Full slave interface is generated. On Xilinx ZCU106, the connectivity is PS (ARM CPU) ↔ AXI Interconnect (Xilinx crossbar) ↔ PL (our “Top” IP). The register map conforms to Xilinx XRT conventions, with a control register at offset 0, status at 0x04, and argument buffers at 0x10…. The host-side C driver mmap()s the AXI region, writes input operands into DRAM or streams them via DMA, starts execution by poking the control register, polls the status register, and reads back results. This uses vendor-provided crossbar IP to route AXI transactions transparently from the processing system to the accelerator (Zang et al., 2023).

5. Case studies and performance characteristics

The reconfigurable-device study uses GEMM, $C=A\cdot B$, for matrix sizes from 4×4 through 128×128, comparing two variants: nested for loops with full resource sharing, and an inner-loop fully flattened or unrolled implementation with maximum parallelism. The reported clock cycle counts assume a 1 ns clock (Zang et al., 2023).

Matrix size	Nested cycles	Unrolled cycles
4×4	1 498	1 114
8×8	10 762	7 946
16×16	81 802	60 298
32×32	867 594	470 282
64×64	5 042 698	3 527 115
128×128	38 324 504	26 806 047

For the 32×32 case on ZCU106, the nested time-division-multiplexed configuration uses approximately ~8 000 LUTs, ~5 000 DFFs, and 16 DSPs, while the inner-flattened configuration uses approximately ~32 000 LUTs, ~20 000 DFFs, and 256 DSPs. The paper states that full unrolling drastically reduces latency, up to ~45 %, at the expense of area, whereas the time-sharing design is area-efficient but slower (Zang et al., 2023).

The AMD NPU work presents two case studies. For matrix multiplication in bf16, herd shapes of 2×2, 2×4, and 4×4 correspond respectively to 1 TOP/s, 2 TOP/s, and 4 TOP/s peak configurations. The achieved throughputs are 0.482 TOP/s (48.2 %) for 2×2, 1.300 TOP/s (65.0 %) for 2×4, and 1.943 TOP/s (48.6 %) for 4×4. On Pareto fronts for a 4×4 herd with sweep M=N=K up to 4096, the reported efficiencies are 78.7 % (MLIR-AIR) vs. 77.7 % (MLIR-AIE) vs. 76.6 % (ARIES) for i16, 48.6 % vs. 50.8 % vs. 49.1 % for bf16, and 59.1 % vs. 60.6 % vs. 56.2 % for i8. The comparison is summarized in the paper as MLIR-AIR tracking hand-tuned MLIR-AIE implementations within 5 percentage-points, while starting from a generic loop nest (Wang et al., 16 Oct 2025).

The second case study is a fused multi-head attention block from LLaMA 2 expressed in approximately 150 lines of AIR code, targeting a single AIE core with head size = 48 and seq-len = 256. The mapped stages are Q/K/V projection, RoPE, Q·Kᵀ matmul plus invsqrt, softmax, and final softmax·V matmul, with the KV cache implemented via persistent shared memory in DDR using air.channel. The latency breakdown per head is 169 µs for Q/K/V projection, 121 µs for RoPE, 283 µs for matmul plus invsqrt, 115 µs for softmax, and 146 µs for the final matmul, for a total of 834 µs as five kernels. The fused single-kernel version is reported at 373 µs, a 2.24× speedup. The listed trade-offs are elimination of host dispatch overhead of approximately 10–20 µs per kernel, improved data locality, and a single-kernel binary that is larger and slightly less modular (Wang et al., 16 Oct 2025).

6. Formal verification, correctness conditions, and stated limitations

Formal verification enters the MLIR-AIR literature through "Practical Formal Verification for MLIR Programs," which reports on verification of the AMD MLIR-AIR and MLIR-AIE toolchains using PEQC-MLIR (Tucker et al., 1 May 2026). The verifier is defined around a small, formally specified parallel proxy language with small-step operational semantics

$$\sigma \xrightarrow[]{n} \sigma',\quad \sigma \to \sigma'$$

over states $\sigma=(M,G)$, where $M$ is memory and $2MNK$0 is a happens-before graph, or over error(G). Program termination is written as

$2MNK$1

Two programs are strictly equivalent if they terminate in final states with equal memories, although in practice the requirement is equality only on non-local live-out variables.

PEQC-MLIR accepts a restricted MLIR subset consisting of func, arith, math, memref, scf, async, and a minimal verif dialect with verif.semaphore and operations wait, set, acquire, and release. The restrictions are explicit: no recursion, no dynamic memory, no symbolic loop bounds, and no array subscripts that evaluate to a symbolic expression. These constraints are used to preserve a deterministic small-step semantics, a decidable next enabled statement per task, and a terminating construction of the global happens-before graph. Confluence is stated as Theorem 3.7 and determinism as Theorem 3.9 (Tucker et al., 1 May 2026).

The interpreter is hybrid concrete–symbolic. Each executed statement produces either a concrete computation, a symbolic AST node represented in a Computation DAG, or an error if a condition such as a loop guard or array index fails to evaluate concretely. A global validity test graph(G) checks race-freedom and synchronization correctness for set/wait and acquire/release. Because each executed statement generates at most one new CDAG node, with hash-consing for repeated subexpressions, the stated complexity is linear in the number of executed statements: time $2MNK$2 to build the final memory and graph, space $2MNK$3 for the CDAG, and equivalence checking again $2MNK$4 by comparing sorted lists of CDAG node hashes at each live-out cell (Tucker et al., 1 May 2026).

Integration into the AMD AIR pipeline is implemented through three converter passes in the verif-opt plugin: air.execute → async.execute, air.herd → scf.parallel, and air.channel → memref+verif. In the C-AST generator, AIR async and synchronization constructs become calls into the PEQC-MLIR API. On a 32×32 matrix multiply, four intermediate AIR outputs—linalg at 8 LoC, air.input at 51 LoC, air.tiled at 133 LoC, and air.placed at 902 LoC—expand after MLIR→C→PIR translation to 201 870 → 1 320 799 statements, with up to 77815 synchronization operations and a happens-before graph ratio of at most 0.39. Full interpretation times range from 1.6 s to 9.0 s, and equivalence checks remain at approximately 0.04 s. The verifier also found a subtle bug in MLIR-AIE lowering, flagging the non-determinism in 5 s, compared with 31 s for the corrected version (Tucker et al., 1 May 2026).

The stated limitations and future directions are correspondingly concrete. In the verification work, ongoing directions include CDAG compaction via affine folding, support for parametric loop bounds using symbolic control predicates, integration with a formally verified lowerer for AIR→std→C, and a richer rewrite-rule library or use of egg (Tucker et al., 1 May 2026). In the compiler flows themselves, stated future extensions include support for dynamic memory, floating-point types, richer MLIR tensor dialects such as linalg, replacement of Xilinx’s XRT with a lightweight portable runtime for low-power FPGAs, multi-NPU or global scheduling, AIR→ROCr lowering for heterogeneous dispatch, dynamic multi-device air.launch semantics, and performance-guided auto-tiling with cost-model–driven schedule search (Zang et al., 2023, Wang et al., 16 Oct 2025).