Compiler and Hardware Co-Design for Accelerator Architectures

Abstract: Heterogeneous accelerator architectures offer an efficient path to performance for compute-intensive workloads. However, full-stack integration remains difficult. We present EAAC (Extensible Accelerator Architecture), a flexible and extensible compiler and hardware architecture designed to lower the overhead of hardware-compiler co-development for rapid prototyping of hardware accelerators. EAAC targets static data-flow workloads with predictable memory access patterns. By using MLIR, we enable possible integration with a range of different frontends that emit MLIR. And by providing a minimal set of compiler functionality that enable necessary data-orchestration we lower the effort needed to get a simple implementation of a hardware acceleration unit up and running, while providing a relatively blank and un-opinionated starting point for further work. We validate this approach with a prototype GEMM accelerator combining a systolic array unit with an embedded RISC-V core, compiled end-to-end through the EAAC MLIR pipeline. On a synthetic fully-connected-layer workload, the accelerator achieves a 28x speedup in execution time over a RISC-V-only baseline, with the GEMM operation itself accounting for only 1.64\% of total execution time. We further characterize the compiler's hardware-semaphore allocation, showing that the number of semaphores required scales linearly with instruction count in the worst case, and identify this as a concrete target for future optimization.

• The paper presents EAAC, a compiler-hardware framework designed to reduce the integration cost of custom accelerators by addressing memory orchestration, synchronization, and code generation challenges in accelerator development.
• EAAC uses an extensible hardware interface paired with an MLIR-based compiler to support memory, synchronization, and control mechanisms for custom hardware units. Specifically, the prototypes aim to validate this approach through an eight-bit systolic GEMM unit implementation reaching a 28x speedup over a RISC-V-only model.
• EAAC's memory-to-memory accelerator model decouples access and execution using a tiered scratchpad hierarchy and specialized DMA engines, which improves data flow and enhances specific tasks, such as matrix operations, activation functions, and pooling.

Research problem and design objective

“Compiler and Hardware Co-Design for Accelerator Architectures” presents EAAC, an extensible compiler–hardware framework intended to reduce the integration cost of custom accelerators into a complete execution stack (2609.30099). The paper addresses a specific gap in accelerator-development infrastructure: HLS and accelerator design languages reduce the cost of implementing hardware, but they generally produce application-specific hardware and do not, by themselves, solve memory orchestration, synchronization, code generation, runtime configuration, or integration with heterogeneous functional units. Conversely, conventional compiler infrastructures provide mature software abstractions but offer limited support for novel spatial architectures with explicit scratchpad management and distributed execution.

EAAC deliberately does not attempt to synthesize hardware automatically. Its objective is narrower and more pragmatic: provide a minimal, extensible hardware interface and a corresponding MLIR-based compiler pipeline through which independently developed accelerator units can share memory, synchronization, and control mechanisms. The target workload class consists of statically analyzable data-flow applications with regular memory accesses, limited control flow, and predictable buffer lifetimes. GEMM, DSP kernels, and image-processing pipelines are representative examples.

The design occupies a distinct point in the accelerator design space. It is more flexible than an HLS-generated accelerator specialized for one application, but less general than a CGRA or a conventional processor. The cost of this flexibility is that hardware developers remain responsible for implementing domain-specific functional units and their code-generation support. EAAC therefore provides infrastructure for co-development rather than an automated hardware-generation methodology.

EAAC architecture

EAAC implements a memory-to-memory accelerator model based on decoupled access and execute. Instead of exposing scalar registers as the primary instruction operands, instructions manipulate statically allocated memory buffers. Functional units interpret those buffers according to their operation semantics. This makes coarse-grained operations such as GEMM, activation functions, and pooling natural instruction units.

The architecture contains an arbitrary number of functional units, a tiered scratchpad hierarchy, programmable DMA engines, a load/store interface to the host, and a hardware semaphore array. Tier 0 is directly visible to accelerator units and acts as the shared working memory. Lower tiers provide additional capacity, with DMA operations explicitly moving buffers between tiers. The system has no program counter, branches, or loops in the conventional sense. Instead, execution is driven by an instruction stream and by data availability.

Figure 1: EAAC’s abstract architecture, including functional units, tiered memory, DMA interfaces, and hardware semaphores.

The central control mechanism is data-driven synchronization through binary or counting semaphore pairs. A semaphore represents the availability state of a buffer or buffer segment. Producers and consumers acquire and release the corresponding semaphore state, allowing functional units to operate asynchronously while preventing data hazards. Counting semaphores additionally support streaming access, in which a buffer is produced and consumed incrementally rather than as a single indivisible object.

EAAC extends this mechanism with semaphore chaining, event predicates, broadcasting, and generation tags. Chaining allows one semaphore transition to trigger another, enabling fences and producer-to-multiple-consumer communication. Generation tags distinguish successive logical allocations that reuse the same physical semaphore address. This is analogous to register renaming: a finite physical resource can represent multiple logically distinct synchronization instances without permitting stale operations to alias a newer allocation.

The use of dedicated hardware semaphore units, rather than memory-based atomic operations, is intended to avoid synchronization traffic competing with ordinary data traffic. The resulting design assumes that synchronization latency and access behavior should remain predictable. The paper nevertheless acknowledges that explicit synchronization introduces overhead, which motivates compiler-side semaphore pruning.

MLIR compilation pipeline

EAAC uses MLIR as the compiler substrate, enabling frontends that already lower into MLIR dialects to target the framework. The pipeline begins after tensor bufferization, when operations manipulate statically allocated memrefs with explicit lifetimes. This choice provides the compiler with the information required for memory placement, lifetime analysis, DMA insertion, and dependency construction. MLIR’s extensibility is particularly important because EAAC must support both generic data movement and target-specific code generation.

The compiler consists of three principal phases: static memory allocation and data planning, dependency and semaphore management, and code generation. MiniMalloc supplies the underlying static allocation mechanism. EAAC overlays a tier-aware spilling algorithm on top of MiniMalloc rather than modifying the allocator itself. Buffers are processed according to lifetime start points; when a tier lacks sufficient capacity, the compiler spills a buffer whose next use is farthest away, inserts a transfer to a lower tier, and reloads the buffer before its next use.

Figure 2: Tier-aware buffer spilling based on buffer lifetimes and next-use information.

This strategy is a form of linear-scan allocation extended to a multi-tier scratchpad hierarchy. Buffers are normally single-assignment and exist in only one tier at a time, except for constants that are statically placed in lower memory. Once the allocator determines offsets and spill points, subsequent passes materialize those decisions as explicit DMA operations.

The dependency phase initially constructs virtual semaphores for producer–consumer edges in the data-flow graph. This conservative strategy guarantees correctness under arbitrary partial out-of-order execution, but it can generate many synchronization objects. The compiler subsequently eliminates semaphores when it can establish that operations are already serialized by hardware behavior, by sufficient graph distance, or by known execution ordering between functional units. It also identifies potential read-after-write hazards caused by scratchpad address reuse and chains the relevant semaphore instances to preserve ordering.

Finally, virtual semaphores are mapped to a finite set of physical semaphore addresses. When physical addresses are reused, generation tags prevent aliasing between logically distinct instances. The allocator selects addresses based on an estimated release time, attempting to minimize serialization introduced by resource recycling. This design makes the compiler’s quality dependent on hardware-specific knowledge: the more accurately the compiler models unit ordering and run-ahead behavior, the more aggressively it can remove synchronization.

Code generation is split between accelerator-specific lowering and RISC-V lowering. Users provide the code generation needed for application-specific hardware units, while EAAC supplies common passes for memory orchestration and synchronization. Unsupported operations can be offloaded to an embedded RISC-V processor. The final accelerator program is serialized as a FlatBuffer, and the runtime configures the hardware and loads any generated RISC-V code.

Prototype accelerator

The prototype validates the framework with an 8-bit, $16 \times 16$ GEMM unit implemented as a systolic array. It also integrates a small five-stage, single-issue RISC-V core, load/store units connected through AXI4-Stream interfaces, a tiered memory system, and a bank of 16 hardware semaphores.

Figure 3: Prototype EAAC accelerator integrating a systolic GEMM unit, RISC-V control core, memory system, and semaphore subsystem.

The GEMM unit performs the matrix-intensive portion of a synthetic dense-layer workload. The complete workload comprises 24 independent $16 \times 16$ int8 matrix multiplications, accumulation into int32, bias addition, ReLU, and requantization to int8. The GEMM unit and the RISC-V core therefore divide the workload: the systolic array handles matrix multiplication, while the RISC-V core performs integer operations and residual control-oriented computation.

The prototype uses a 512-bit memory-transfer bus and a separate 16-bit semaphore bus. It contains 64 kB of Tier 0 memory and 128 kB each of Tier 1 and Tier 2 memory. The RISC-V core has 64 kB of instruction memory and 4 kB of data memory. The implementation reaches a maximum reported clock frequency of 104.602 MHz, with the critical path located in the RISC-V core rather than in the GEMM unit or semaphore subsystem.

Resource utilization is dominated by the systolic array. The complete design consumes 64,414 LUTs, 71,891 flip-flops, 16 block-RAM units of the reported B36 category, and 266 DSPs. The GEMM unit alone accounts for 40,384 LUTs and 258 DSPs, or 97% of the DSP resources. This distribution is consistent with the paper’s positioning of the prototype as a proof-of-concept integration vehicle rather than an optimized accelerator implementation. It also indicates that the infrastructure overhead is not negligible: the load/store units, semaphore system, memory system, and RISC-V core collectively consume substantial logic even though the systolic array remains the principal computational resource.

Evaluation of end-to-end performance

The principal performance comparison is between execution entirely on the embedded RISC-V baseline and execution on the EAAC configuration combining the RISC-V core with the systolic GEMM unit.

Configuration	Measured cycles
RISC-V-only baseline	3,591,489
EAAC accelerator total	127,348
GEMM computation within EAAC	2,092
EAAC RISC-V boot	24,711
EAAC RISC-V computation	102,057

The integrated accelerator reduces total execution time from 3,591,489 to 127,348 cycles, corresponding to a reported 28x speedup. The result demonstrates successful end-to-end compilation and execution through the EAAC stack rather than merely measuring an isolated GEMM kernel.

The detailed breakdown also qualifies the result. GEMM consumes only 2,092 cycles, or 1.64% of total accelerator execution time. The embedded RISC-V computation accounts for 102,057 cycles, while binary-transfer and boot overhead account for an additional 24,711 cycles. Thus, the measured speedup is not primarily a consequence of the GEMM unit dominating total execution time; it reflects the combination of hardware offload, workload partitioning, and the weakness of the intentionally simple RISC-V baseline.

This distinction is important for interpreting the result. The 28x improvement establishes that EAAC can connect an MLIR program, static memory planning, DMA orchestration, semaphore synchronization, generated RISC-V code, and a custom accelerator into a functioning system. It does not establish that the current architecture provides high utilization of the systolic array. In fact, the low GEMM fraction shows that the prototype is substantially limited by non-GEMM work and software initialization overhead.

The execution-time comparison also includes instruction-memory transfer costs for both configurations. The accelerator’s binary transfer is longer because the internal EAAC-to-LLVM lowering path is not optimized. Consequently, the reported result includes a compiler/runtime overhead that is unfavorable to the accelerator and may decrease as the lowering pipeline improves. At the same time, the evaluation uses a small synthetic workload and a weak scalar baseline, so the result should not be interpreted as a comparison against a high-performance CPU, vector processor, GPU, or production accelerator.

Semaphore scalability

The paper’s second evaluation examines compiler behavior under increasingly large programs with intentionally unfavorable buffer lifetimes. In the worst case, the number of distinct hardware semaphores grows linearly with the number of operations.

Figure 4: Worst-case semaphore demand as a function of total operation count, before and after hardware-aware pruning.

This result exposes the primary current scalability concern in EAAC. A virtual semaphore is initially associated with each data-flow edge, and long-lived buffers preserve synchronization state across many operations. Without sufficiently aggressive pruning, larger programs can exhaust the finite semaphore bank and become infeasible even when the underlying computation is structurally suitable for acceleration.

The paper makes a deliberately strong observation: for the prototype, much of this semaphore demand is unnecessary because only one GEMM unit is present, so GEMM operations are intrinsically serialized. Hardware-aware analysis breaks the linear growth observed in the conservative configuration by removing semaphores between operations whose order is already guaranteed. Additional pruning is possible when different units have known run-ahead or ordering properties.

This result supports the paper’s co-design thesis. Semaphore allocation cannot be optimized effectively using compiler information alone; it depends on the number and behavior of functional units, their execution ordering, and the degree of internal serialization. Conversely, the hardware exposes enough structure for the compiler to remove synchronization that would otherwise be required by a conservative data-flow model. The remaining question is whether this approach remains effective for architectures with multiple instances of the same unit, dynamic resource contention, or genuinely concurrent streaming pipelines.

Limitations and open questions

EAAC is explicitly restricted to static data-flow workloads. Programs with dynamic allocation, irregular memory accesses, substantial control flow, or data-dependent iteration are outside the demonstrated compilation model. The architecture’s absence of a program counter and its reliance on statically analyzable lifetimes simplify hardware control but constrain applicability.

The prototype evaluation is also narrow. It uses one synthetic dense-layer workload, one systolic GEMM unit, a small RISC-V core, and an FPGA implementation whose critical path lies in the control processor. The 28x speedup is therefore a validation of integration rather than a broad architectural comparison. No energy measurements, area-normalized performance analysis, comparison with an optimized CPU or GPU, or evaluation across multiple real workloads is reported.

The compiler remains dependent on target-specific hardware knowledge. Its initial virtual-semaphore construction can scale poorly, and the selected workload was reduced because larger programs produced infeasible semaphore requirements under the available optimization. This is a direct limitation rather than a peripheral implementation detail: semaphore allocation currently constrains the size of compilable workloads.

Memory planning also assumes well-defined static buffer lifetimes and single-tier residency. The spilling heuristic selects the buffer with the latest next use, but the paper does not provide a comparative evaluation against alternative multi-tier allocation algorithms, nor does it quantify DMA traffic, spill-induced latency, or memory-energy costs. Similarly, the correctness and performance behavior of semaphore chaining and generation-tag reuse are demonstrated conceptually and through the prototype, but not evaluated under a broad range of concurrent producer–consumer topologies.

The paper leaves open how EAAC would scale when the hardware contains multiple parallel instances of a functional unit, when synchronization latency becomes significant relative to coarse-grained computation, and when the compiler must jointly optimize tiling, memory placement, DMA scheduling, and accelerator occupancy. It also leaves open whether a more expressive MLIR dialect could make hardware-unit integration less dependent on manually implemented code-generation passes.

Conclusion

EAAC presents a deliberately minimal framework for compiler–hardware co-development. Its contribution is the integration of MLIR-based compilation, static multi-tier memory planning, explicit DMA orchestration, distributed hardware semaphores, and heterogeneous functional units under a common memory-mapped interface. The prototype demonstrates a complete compilation-to-execution path and reports a 28x speedup over a small RISC-V-only baseline.

The evaluation simultaneously identifies the framework’s principal constraint: synchronization-resource demand can grow linearly with instruction count under conservative allocation. Hardware-aware semaphore pruning mitigates this behavior, but the result depends on detailed knowledge of accelerator-unit ordering and remains an important compiler optimization target. EAAC therefore offers a credible infrastructure layer for statically scheduled accelerator systems, while leaving scalability across larger, more concurrent, and less regular workloads as an open empirical question.