Azul Accelerator: Sparse Iterative Solver Hardware

• Azul Accelerator is a spatial, all-SRAM hardware accelerator designed for iterative solvers on large sparse matrices in numerical linear algebra.
• It features a 2D grid of RISC-V-like tiles with local SRAM and a NoC for efficient execution of key kernels like SpMV and SpTRSV.
• The FPGA realization validates its compute-bound performance through high inter-iteration reuse and fine-grained task synchronization.

Searching arXiv for the specified paper and closely related Azul accelerator work. {"^^^^^^^^2query2 OR \2^^^^^^^^"SuperUROP: An FPGA-Based Spatial Accelerator for Sparse Matrix Operations\"","max_results":5} Azul Accelerator is a spatial, all-SRAM hardware accelerator specifically designed for sparse iterative solvers in numerical linear algebra. In the formulation emphasized by "SuperUROP: An FPGA-Based Spatial Accelerator for Sparse Matrix Operations" (&&&^^^^2query2^^^^&&&), Azul targets large sparse systems of linear equations of the form PRESERVED_PLACEHOLDER_^^^^2query2^^^^, where PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^ is sparse, often with nonzero density as low as $0.01\%$, and the solution is obtained through iterative methods such as preconditioned conjugate gradient rather than by computing $A^{-1}$ explicitly. Its central architectural premise is to store the sparse matrix entirely in distributed on-chip SRAM near compute, thereby exploiting inter-iteration reuse across thousands of iterations, maximizing on-chip data reuse, exploiting irregular, fine-grained parallelism, and eliminating the main-memory bandwidth bottleneck (&&&^^^^2query2^^^^&&&).

^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^. Computational target and architectural motivation

Azul is designed around the observation that sparse iterative solvers decompose into a small number of core sparse kernels, notably Sparse Matrix-Vector Multiplication (SpMV) and Sparse Triangular Solve (SpTRSV). For a sparse matrix $M$ and vector $v$, SpMV is expressed as

$y_i=\sum_{j \in nz(M_i)} M_{ij}\, v_j,$

where $nz(M_i)$ returns the indices of nonzeros in row $i$. SpTRSV solves systems such as $Lx=b$, where PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2query2^^^^ is sparse and typically lower triangular. These kernels recur within iterative methods that generate a sequence of iterates PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^ until convergence.

The architectural motivation arises from a mismatch between sparse numerical kernels and conventional CPUs and GPUs. The data characterize three main impediments. First, sparse iterative solvers exhibit poor short-term data reuse: each iteration scans the full sparse matrix and uses each entry almost exactly once per iteration, so large matrices cannot remain cache-resident across iterations. Second, sparse formats induce irregular memory access patterns, including pointer-chasing and indirect indexing, which lead to cache misses and TLB pressure on CPUs and to misaligned and scattered memory accesses on GPUs. Third, sparse kernels expose complex data dependencies and low regular parallelism: SpMV has row-level parallelism but little reuse of matrix entries, whereas SpTRSV includes explicit triangular dependencies that constrain execution order.

The data further report that, empirically, when running PCG on characteristic sparse matrices, GPUs achieve <^^^^2query2^^^^.5% of peak throughput, illustrating that conventional architectures leave most of their potential performance untapped. Azul’s response is to exploit inter-iteration reuse rather than within-iteration locality: if the full matrix remains resident in distributed on-chip SRAM for the duration of the solver, off-chip DRAM traffic for matrix entries is removed and the memory bottleneck shifts to high-bandwidth, low-latency local SRAM. This suggests that Azul is best understood not merely as a sparse-kernel accelerator, but as an attempt to reframe sparse iterative solving as a locality-preserving spatial computation.

2. Tile-grid organization and on-chip communication

Azul is structured as a 2D grid of tiles, exemplified in the evaluation by a ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6×^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6 organization. Each tile contains a processing element (PE), a local SRAM scratchpad, and network input/output queues connected to a Network-on-Chip (NoC). The PE is described as a small integer core, RISC-V-like in character, with a floating-point multiplier. The local SRAM holds instruction memory, data memory, lookup table memory, and the per-PE register file.

The memory subsystem is physically partitioned. The bottom 64 kB is instruction memory; the next 64 kB is data memory; and “higher hanging” SRAM-backed structures include a 32×32-bit register file and a ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6-entry task lookup table. A defining property of this organization is that each PE can perform single-cycle random access to any word in its local SRAM, allowing the assigned matrix block to remain in on-chip storage with perfect residency across iterations.

Tiles communicate through a NoC. In both the FPGA implementation and the architectural simulation, the network topology is a 2D torus over the ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6×^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6 tile grid. Each tile has BRAM-based input and output FIFOs. Messages contain metadata and a data payload. The metadata includes destination tile coordinates, task type, and target address; the row and column fields are 6 bits each, supporting up to 64×64 = 4^^^^2query2^^^^96 tiles. The task type specifies whether the message writes instruction memory, writes data memory, writes a lookup-table entry, or starts a task.

This message transport is used for three distinct purposes: distributing matrix blocks and control tasks during initialization, passing partial results during SpMV and SpTRSV, and synchronizing tasks via message passing rather than global barriers. In performance terms, the architecture is explicitly designed for high effective memory bandwidth and high arithmetic intensity. Once loaded, matrix blocks are reused across iterations without further off-chip fetches, and the traffic per iteration is largely confined to local SRAM accesses and modest inter-tile messaging.

The associated performance model relates iteration time to compute and communication. For PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \22^^^^ tiles with per-tile SRAM bandwidth PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^3 and PE throughput PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^4, the time per iteration is approximated as

PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^5

where PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6 is local sparse-kernel compute time and PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^7 is inter-tile messaging time. Azul is designed so that for realistic sparse matrices PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^8 dominates, rendering the system compute-bound rather than memory- or network-bound.

3. Task-based execution model and ISA extensions

Azul does not use SIMD or globally synchronized threads. Instead, it employs a task-based programming model in which each PE executes short instruction sequences implementing portions of a sparse computation. These tasks are independent per PE and synchronize only through message passing. A PE without active work idles; it is reactivated by network messages that install or modify instructions, write data, write lookup-table entries, or trigger execution of a specific task.

The execution model comprises two phases. In the network-reading phase, the PE repeatedly reads messages from its input queue and uses them to fill instruction memory, data memory, and the lookup table. In the task-execution phase, the PE enters an idle loop with program counter set to ^^^^2query2^^^^. When a START_TASK message arrives, the PE uses the lookup table to find the starting PC for the task, jumps to that PC, executes the instruction sequence, and then returns to idle with PC reset to ^^^^2query2^^^^. The data characterize this behavior as closely resembling a dataflow engine, because task firing depends on message arrival rather than a global time-step scheduler (&&&^^^^2query2^^^^&&&).

To support this execution style, Azul extends the base 32-bit RISC-V ISA with two instructions: send and recv. A message is logically composed of a 64-bit metadata register and a data register whose width is 64 bits or 32 bits depending on the payload. The metadata encodes destination tile coordinates, a 4-bit task type, and a ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6-bit address in the target memory segment. The instruction forms are send r_meta, r_data, which enqueues a network message into the output queue, and recv r_meta, r_data, which dequeues a message from the input queue and places its contents into registers.

These instructions are exposed through inline assembly in C/C++, enabling explicit programmer control over tile selection, operation type at the destination, and memory address placement. The ISA augmentation supports distributed task deployment and fine-grained synchronization via partial-result exchange and dependent-task triggering. The design deliberately does not enforce deadlock freedom or data-tiling correctness in hardware; those guarantees are left to the compiler and programmer. A plausible implication is that Azul prioritizes low-level expressiveness and hardware simplicity over protective execution semantics.

4. FPGA realization in SuperUROP

The SuperUROP implementation maps Azul onto Xilinx FPGAs as a concrete validation of the architecture. Each PE is realized as a simple 5-stage pipelined RISC-V processor with Instruction Fetch, Decode, Execute, Memory, and Writeback stages. The Execute stage contains ALU functionality for address computation, control, and arithmetic, plus a floating-point multiplier for SpMV and SpTRSV arithmetic. The rationale for this minimalist core is that sparse kernels require integer address computation, simple control, and floating-point multiply/add rather than wide vector units or complex instruction sets.

In ASIC terms, the simple pipeline is described as allowing each stage to fit comfortably into a ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^ ns clock period; on FPGA, the design is synthesized to a 2^^^^2query2query2^^^^ MHz clock. The all-SRAM design is approximated using FPGA on-chip memory primitives. Instruction and data memories are implemented using URAM, preserving the 64 kB instruction plus 64 kB data allocation per PE. The register file is implemented with Xilinx RAM32M primitives, the ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6-entry lookup table with LUTRAM, and the network FIFOs with BRAM. This mapping is intended to preserve the single-cycle access assumption central to Azul’s local-memory model.

Tiles are instantiated in the FPGA fabric and connected with a NoC implemented in RTL that emulates the 2D torus topology. The send and recv instructions are wired to per-tile BRAM FIFOs, and routing logic forwards messages hop-by-hop according to destination coordinates. The data emphasize several implementation constraints and trade-offs. URAM and BRAM capacities limit the number of tiles and per-tile memory, so the work uses a ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6×^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6 grid in simulation and a subset of tiles on FPGA for prototyping. Timing closure is affected by long NoC paths, requiring careful floorplanning and pipelining. Network bandwidth is lower than in a custom ASIC, but the implementation seeks to preserve the compute-bound character of Azul rather than becoming NoC-limited.

This FPGA mapping is significant because it shows that the architectural abstraction of distributed SRAM-backed tiles can be approximated with existing reconfigurable-memory resources while retaining the latency assumptions needed by the original design.

5. Sparse-kernel mapping and solver support

The FPGA implementation exercises Azul on SpMV and SpTRSV as the primary kernels. For SpMV, each tile owns a block of matrix rows and the corresponding vector segments. Local SRAM stores nonzeros and column indices. The PE iterates over local nonzeros and performs multiply-add operations using the floating-point multiplier and ALU. Output segments may remain local or be transmitted to other tiles if subsequent computation requires them elsewhere.

For SpTRSV, the lower-triangular dependency structure implies that each PRESERVED_PLACEHOLDER_^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^9 depends on previously computed values: $0.01\%$^^^^2query2^^^^ Although a naive implementation is strongly sequential, the data state that substantial parallelism can be exposed through block structure and sparsity. Azul assigns blocks of $0.01\%$^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^ to tiles, encodes tile responsibility in tasks, and uses recv to obtain prerequisite $0.01\%$2 values from other tiles while using send to propagate newly computed components. Because tasks can be triggered as dependencies are satisfied, the architecture can proceed without global barriers.

The implementation does not realize a full PCG loop end-to-end, but it is described as being designed to support standard iterative formulations such as

$0.01\%$3

where each iteration requires SpMV, SpTRSV, and vector operations including dot products and saxpys. Matrix $0.01\%$4 is partitioned into blocks and assigned to tiles as a one-time expense that can be done offline, and vectors are partitioned consistently with matrix blocks so that each tile stores the portion aligned with its rows and columns.

Communication patterns differ by kernel. In SpMV, tiles may need remote vector entries $0.01\%$5, which are exchanged via send and recv. In SpTRSV, tiles exchange components of $0.01\%$6 immediately after computation so that dependent tasks can proceed. The architecture thereby mitigates irregular access by replacing cache-based indirect memory behavior with local pointer arithmetic over directly indexable SRAM, and it mitigates limited reuse by exploiting matrix residency across repeated iterations. This suggests that Azul’s efficiency derives from the combination of static data placement and dynamically triggered task activation.

6. Validation, performance characteristics, and limitations

The FPGA implementation is validated against three baselines: a cycle-accurate architectural simulation of Azul, a Python script modeling expected numerical results and performance, and a set of simple distributed test cases together with SuiteSparse matrices (&&&^^^^2query2^^^^&&&). The architectural simulation assumes a ^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6×^^^^2id:(Parthasarathy, 15 Sep 2025) OR \2^^^^6 tiled architecture, a 2 GHz clock frequency, the same per-PE memory sizes, and the same 2D torus NoC topology. The validation setup is intended to align memory latency assumptions between the simulation and FPGA realization, while also checking correctness of the task model and the send/recv ISA.

The simple distributed test cases verify dataflow patterns that interleave multiply/add tasks with communication, liveness and deadlock absence under typical SpMV and SpTRSV communication patterns, and network semantics such as ordering and correct destination writes. These tests are particularly important because the task model itself does not enforce correctness properties.

The subsequent evaluation uses matrices from the SuiteSparse benchmark, spanning multiple matrix sizes and sparsity patterns. The central questions are whether the FPGA implementation remains compute-bound and whether its performance matches the architectural simulation after accounting for a 2^^^^2query2query2^^^^ MHz FPGA vs. 2 GHz simulation clock difference and memory-latency differences between URAM and ideal SRAM. The reported results are that the FPGA variant remains compute-bound rather than memory- or NoC-bound, that measured performance matches simulation when scaled by clock frequency and memory latency, and that numeric outputs match the Python reference results. The phrase “equivalent performance to an architectural simulation of the Azul framework” is explicitly defined to mean that the number of cycles on FPGA matches the cycle-accurate simulator up to minor implementation overheads and that normalized throughput is consistent with

$0.01\%$7

assuming identical microarchitectural pipeline and memory latencies.

The broader implications are presented cautiously. The data argue that distributed SRAM architectures like Azul can be realized on mainstream reconfigurable hardware, that the bottleneck for sparse iterative solvers can shift from off-chip DRAM bandwidth to on-chip compute throughput, and that Azul can be a viable competitor to GPUs for users with access to FPGAs, particularly for scientific computing workloads dominated by large sparse systems. At the same time, several limitations are explicit: FPGA resource constraints restrict tile count and local-memory capacity; NoC bandwidth and latency on FPGA are inferior to specialized on-chip fabrics; the task programming model lacks hardware support for deadlock avoidance and correctness guarantees; and the prototype’s 32-bit floating point and 2^^^^2query2query2^^^^ MHz clock may limit absolute performance relative to high-end GPUs or ASICs.

Potential extensions identified in the data include multi-FPGA deployments with hierarchical NoCs, finer-granularity tiles with more local memory per tile, further pipeline optimization and DSP usage to reach higher clock frequencies, vector or fused multiply-add units tailored to sparse kernels, automated matrix partitioning and static analysis for deadlock-free task graphs, high-level languages or libraries for expressing iterative solvers in Azul’s task/dataflow terms, and co-processing arrangements with CPUs and GPUs, including offloading of PCG, GMRES, and related kernels from PETSc or SuiteSparse-based frameworks. Taken together, these directions position Azul as a research architecture whose defining contribution is the systematic exploitation of inter-iteration reuse and fine-grained task synchronization in sparse linear algebra rather than a narrow optimization of any single sparse kernel.