Gemmini Accelerator Overview

• Gemmini is an open-source deep neural network accelerator generator featuring a configurable spatial array, explicit on-chip memories, and tight RISC-V integration.
• Its design addresses methodological challenges by enabling evaluations under realistic conditions that include shared caches, DRAM bandwidth, and operating-system overheads.
• Gemmini serves as a reusable reference for hardware-software co-design, supporting research in FPGA deployment, compiler integration, performance modeling, and LLM-driven accelerator generation.

Gemmini is an open-source deep neural network accelerator generator and full-stack evaluation platform whose architectural template centers on a spatial array of processing elements, explicit on-chip memories, and tight integration with RISC-V system-on-chip infrastructure. Rather than defining one fixed accelerator instance, Gemmini was introduced to enable systematic deep-learning architecture evaluation under realistic conditions that include shared-resource contention, operating-system overheads, and programming-stack effects; later work has repeatedly reused it as a standardized template for compiler research, FPGA deployment, performance modeling, semantic extraction, and LLM-driven accelerator generation (Genc et al., 2019, Vungarala et al., 2024).

1. Origins, scope, and research role

Gemmini was created in response to a methodological problem in accelerator research: many DNN accelerators are evaluated largely in isolation, which obscures the effect of shared caches, buses, DRAM bandwidth, virtual memory, host-CPU overheads, and software/runtime choices on realized performance and energy efficiency. Its defining contribution is therefore dual. At the hardware level, it is a configurable accelerator generator for dense DNN workloads; at the systems level, it is a full-stack platform that generates not only accelerator RTL but also software support and Linux-capable RISC-V SoCs suitable for end-to-end study (Genc et al., 2019).

This full-stack emphasis became central to Gemmini’s later reception. Subsequent work describes it as “a comprehensive and well-packaged generator with open-source infrastructure tailored to designing full-stack DNN accelerators,” with “a versatile hardware framework, a multi-layered software stack, and an integrated System-on-Chip (SoC) environment” (Vungarala et al., 2024). In that sense, Gemmini occupies two positions in the literature simultaneously: it is both a concrete accelerator family and a reference methodology for cross-stack co-design.

A persistent point of clarification in later research is that Gemmini is not primarily presented as a one-off ASIC macro. It is instead used as a reusable template for generating accelerator instances, integrating them into Chipyard-based systems, and studying how hardware parameters, software mappings, and surrounding system resources interact. This broader role explains why it appears not only in architecture papers, but also in compiler integration, automatic modeling, formal semantic lifting, and dataset construction efforts (Genc et al., 2019).

2. Architectural template and execution model

Gemmini’s architectural template is based on a spatial array of processing elements performing multiply-accumulate operations. The original description emphasizes a two-level hierarchy: the array is organized into tiles, and within each tile the PEs are connected combinationally, while communication between tiles goes through explicit pipeline registers. This permits designs ranging from more vector-like organizations to more fully systolic meshes, while retaining nearest-neighbor structure (Genc et al., 2019).

Around the compute array, Gemmini provides an explicitly managed local memory system. The main structures described across the literature are a banked scratchpad SRAM for operands, a banked accumulator SRAM for partial sums and outputs, a DMA path connected into the SoC memory system, and decoupled controllers for data movement and execution. A later compiler-integration study describes Gemmini as a $DIM \times DIM$ systolic array with a banked scratchpad SRAM, a banked accumulator SRAM with adders, a DMA engine connected directly to L2 cache, decoupled controllers for load, execute, and store, and a reorder buffer that enforces hazards and issues instructions to those controllers (Peccia et al., 2022). FPGA-oriented work restates the same model in operational terms: a load controller DMA-transfers tiles from external memory into local memories, an execute controller feeds the systolic array from local storage, and a store controller writes results back; the CPU orchestrates execution by issuing Gemmini instructions (Peccia et al., 2024).

Gemmini supports both output-stationary and weight-stationary dataflows. Each PE performs one multiply-accumulate per cycle and can be configured for either dataflow, while the generator also exposes design-time and runtime configurability over dataflow, array shape, memory sizing, datatypes, and optional helper blocks (Genc et al., 2019). In later descriptions, the baseline compute kernel remains conventional matrix multiplication, for example

$C_{i,j} = \sum_k A_{i,k} B_{k,j},$

with GEMM offloading often written as

$$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$

These formulations are used not as new abstractions, but as the mathematical form to which Gemmini’s tiled systolic execution is repeatedly mapped (Peccia et al., 2024, Peccia et al., 2022).

Gemmini’s programmability is correspondingly explicit. One line of work partitions its ISA into configuration instructions, move instructions for internal SRAM transfers, execution instructions that dispatch a $DIM \times DIM$ GEMM, flush/fence instructions, and higher-level loop instructions that behave as CISC-style commands implemented by internal finite-state machines (Peccia et al., 2022). FPGA deployment work further distinguishes CISC-style commands for tiled matrix multiplication and convolution from lower-level RISC-style instructions that explicitly control scratchpad and accumulator movement and launch fine-grained matrix operations (Peccia et al., 2024). This explicit local-memory model is important: Gemmini is not characterized as a free-running streaming engine, but as a programmable accelerator whose efficiency depends on tiling, scheduling, and reuse.

3. Full-stack SoC integration and system-level evaluation

Gemmini’s most distinctive feature relative to many accelerator templates is its SoC integration. It is generated inside the Chipyard ecosystem and can be attached to Rocket or BOOM RISC-V cores, with support for multiple CPUs, multiple Gemmini instances, shared caches, shared buses, and Linux-capable execution. The original evaluation emphasizes that this enables studies that would be invisible in isolated datapath analysis, including cache interference, TLB behavior, context switches, physical-memory permission bugs, and non-deterministic deadlocks triggered by OS scheduling (Genc et al., 2019).

The paper reports concrete implementation results for one synthesized configuration: a system with a $16 \times 16$ spatial array, a 256 KB scratchpad, a 64 KB accumulator, and one Rocket CPU has an area breakdown of 116 K $\mu m^2$ for the spatial array, 544 K $\mu m^2$ for the scratchpad, 146 K $\mu m^2$ for the accumulator, and 171 K $\mu m^2$ for the CPU, for a total of 1,029 K $\mu m^2$. SRAM dominates area, with the paper explicitly stating that SRAMs consume 67.1% of total accelerator area while the spatial array itself is 11.3% (Genc et al., 2019). This distribution became one of Gemmini’s recurring architectural lessons: local memory sizing is often at least as consequential as PE count.

System-level experiments in the same work further show that end-to-end performance depends strongly on what remains on the host CPU. Without on-the-fly im2col support in the accelerator, moving from an in-order Rocket host to an out-of-order BOOM host increases CNN performance by 2.0× across the evaluated CNNs because the CPU performs convolution lowering. With on-the-fly im2col in the accelerator and a simple Rocket CPU, the reported performance at 1 GHz includes 22.8 FPS on ResNet50, 79.3 FPS on AlexNet, and 18.7 FPS on MobileNetV2, together with speedups such as 2,670× over the in-order Rocket CPU for ResNet50 and 144× for BERT (Genc et al., 2019). The same study explicitly notes that MobileNetV2 maps less efficiently because depthwise convolutions have low data reuse.

Gemmini was also used to study virtual memory and shared-memory provisioning. In a low-power $C_{i,j} = \sum_k A_{i,k} B_{k,j},$0 systolic design with 256 KB scratchpad, increasing a private accelerator TLB from 4 to 16 entries improves performance by up to 11%, whereas adding 512 shared L2 TLB entries improves performance by no more than 8%. Two TLB filter registers caching the last read and last write TLB hits reduce hit latency to 0 cycles for consecutive accesses to the same page; with these filter registers, a 4-entry private TLB and no shared L2 TLB achieve only 2% below maximum performance, aided by the observation that 87% of consecutive read TLB requests and 83% of consecutive write TLB requests target the same page (Genc et al., 2019).

A related cache-partitioning study compares allocating an extra 1 MB SRAM either to Gemmini’s private memories or to a shared L2. In the single-core case, larger scratchpads help convolution layers more than larger L2, but in the dual-core case running two ResNet50 workloads in parallel, a larger shared L2 is better overall: the dual-core BigL2 design reduces L2 miss rate by 7.1% and improves overall performance by 8.0%, while the BigSP design improves overall performance by 4.2% (Genc et al., 2019). These results are representative of Gemmini’s stated purpose: evaluating accelerators as parts of real systems rather than as isolated kernels.

4. Software stack, compiler interfaces, and verified translation

Gemmini is accompanied by software interfaces at multiple abstraction levels. The original work describes a high-level push-button flow from ONNX, low-level C/C++ APIs, and tuned functions for common kernels such as tiled matrix multiplication, convolution, and residual addition, with generated header files exposing instance-specific parameters such as array dimensions, supported dataflows, and included helper units (Genc et al., 2019). Later work treats these interfaces not as fixed endpoints, but as entry points for compiler research.

One important extension is the integration of Gemmini into TVM and AutoTVM. In this line of work, quantized TensorFlow Lite models are imported into Relay IR, Gemmini-compatible patterns are identified and replaced by custom operators, and TVM schedule transformations attach pragmas and tensorization points that lower loops into Gemmini intrinsics. The exposed schedule space includes two-level tiling over $C_{i,j} = \sum_k A_{i,k} B_{k,j},$1, $C_{i,j} = \sum_k A_{i,k} B_{k,j},$2, and $C_{i,j} = \sum_k A_{i,k} B_{k,j},$3, together with parallel_accumulations, apply_double_buffer, exchange_axis, WS/OS, and mvout_big_block as tunable parameters. Using an XGB model-based AutoTVM tuner with early stopping at 500 iterations, the generated code achieves a peak throughput of 46 GOP/s at 100 MHz on a Xilinx ZCU102 FPGA, and the resulting schedules surpass Gemmini’s default handcrafted schedules on most evaluated workloads (Peccia et al., 2022).

The same study also clarifies Gemmini’s execution model from the compiler perspective. GEMM offloading is formulated as explicit scheduling over DRAM, DMA, scratchpad SRAM, accumulator SRAM, and the $C_{i,j} = \sum_k A_{i,k} B_{k,j},$4 systolic array. Larger GEMMs must be decomposed into tiles and subtiles that fit local memories, align with array dimensions, and manage overlap between load, execute, and store. Throughput is computed using

$C_{i,j} = \sum_k A_{i,k} B_{k,j},$5

which counts multiply-add work together with the additive term $C_{i,j} = \sum_k A_{i,k} B_{k,j},$6 in the offloaded GEMM formulation (Peccia et al., 2022).

A different software path connects Gemmini to Metalift. Here, ordinary Python or C++ array-processing code is analyzed symbolically, lifted into a Gemmini-oriented DSL, and verified by SMT-based synthesis rather than by handwritten pattern matching. The prototype models matrix multiplication and convolution from Gemmini’s ISA; the paper’s worked example recognizes a sliding-window sum loop as a 1D convolution with kernel $C_{i,j} = \sum_k A_{i,k} B_{k,j},$7 and stride 1, synthesizes an equivalent Gemmini-level expression, and then emits code in Gemmini’s ISA. The prototype compiler is reported as implemented in less than 100 lines of code, and the example translation takes less than one minute (Nishida et al., 2023). This suggests a different interpretation of Gemmini’s programmability: not only as a target for conventional ML compilers, but also as a semantically modeled accelerator vocabulary that can be reached by synthesis and proof.

5. Reuse as a research substrate: FPGA deployment, datasets, modeling, and semantic extraction

Later work demonstrates that Gemmini is not only a baseline accelerator, but also a reusable substrate for derivative methodologies. One prominent example reworks Gemmini into an FPGA-oriented overlay for edge CNN inference. In that study, the baseline $C_{i,j} = \sum_k A_{i,k} B_{k,j},$8 “Both”-dataflow configuration with 256 KiB scratchpad and 64 KiB accumulator is modified into a $C_{i,j} = \sum_k A_{i,k} B_{k,j},$9 weight-stationary design with 512 KiB scratchpad, 128 KiB accumulator, two scratchpad ports, scratchpad read delay 8, spatial array output bits reduced from 20 to 18, and maximum in-flight memory requests increased from 16 to 32. The authors further implement DSP packing on Xilinx DSP48E2 slices, disable features deemed unnecessary for the target YOLO-family CNNs, and reduce output-scaling precision from float32 to float16. The resulting system deploys YOLOv7 on a Xilinx ZCU102 FPGA and reports real-time performance with an energy efficiency of 36.5 GOP/s/W (Peccia et al., 2024). This does not redefine Gemmini’s original role, but it shows that the generator can be retargeted substantially when the implementation substrate changes.

Gemmini also became the basis of SA-DS, a dataset for LLM-driven accelerator design generation. SA-DS is explicitly centered on “Berkeley’s Gemmini accelerator generator template” and represents each sample as a natural-language description paired with a Chisel implementation. The dataset varies Gemmini-based spatial array structure, function-unit combinations, and dataflow settings while keeping memory parameters fixed. Its counting formulas state that each category has $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$0 possible function-unit combinations, “input type” yields $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$1 datapoints per dataflow type, and additional dataflow variations such as Output Stationary and Weight Stationary raise each category to $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$2 datapoints (Vungarala et al., 2024). In multi-shot prompting, the paper reports 100% pass rates for GPT-4o, Gemini Advanced, and Claude-3.5-sonnet on its Gemmini-style code-generation task, whereas GPT-3.5 remains weak (Vungarala et al., 2024). The significance here is not a new Gemmini microarchitecture, but Gemmini’s emergence as a standardized accelerator template legible both to humans and to code-generating models.

Gemmini has likewise been used as a target for automatic performance estimation. An ACADL/AIDG-based modeling framework instantiates Gemmini with $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$3, models its scratchpad, accumulator, DMA engine, RoCC interface, decoupled access-execute architecture, and reorder-buffer-like hazard handling, and then evaluates instruction/object occupancy graphs rather than cycle-accurate RTL for every loop iteration. Convolutional layers are converted to GEMM by im2col, tiled into $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$4 blocks, and executed using public Gemmini instructions such as gemmini_mvin, gemmini_preload, and gemmini_compute_accumulated. Reported results on a $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$5 Gemmini include 3.67% MAPE for TC-ResNet8 in 0.5 s versus 527 s for Verilator, 9.78% MAPE for AlexNet in 37.9 s versus 43.5 h, and 7.51% MAPE for EfficientNet in 17.3 s versus 11.9 h (Lübeck et al., 2024). This line of work treats Gemmini as complex enough to stress performance-modeling methodology, especially because of its decoupled access-execute architecture.

A still more formal reuse appears in ATLAAS, which starts from Gemmini RTL and automatically lifts bit-level semantics into tensor ISA specifications. The evaluation covers all hardware instruction semantics across 127 Gemmini MLIR files and targets the PE, ExecuteController, LoadController, and StoreController. For the PE, the pipeline reduces one semantic function from 686 low-level MLIR lines to 49 lifted lines, a 92.9% reduction; controller modules see 23.9% to 33.5% reductions, reflecting the greater amount of irreducible control logic (Gao et al., 15 Apr 2026). ATLAAS recovers hardware details omitted from the hand-written Gemmini reference, including three independent DMA banks with 15 configuration registers, 12 pooling configuration registers, and nine im2col output ports, then feeds the recovered specification into ACT to generate a backend whose geometric-mean performance is 1.014× relative to hand-written Gemmini kernels (Gao et al., 15 Apr 2026). This reinforces Gemmini’s status as a canonical open RTL target for semantic reconstruction and automated software-stack generation.

6. Comparative assessments, misconceptions, and limitations

Gemmini is frequently used as a strong open-source baseline, but comparison papers also expose recurring limitations. OpenGeMM presents Gemmini as an adjacent open-source, generator-based, RISC-V-integrated platform, while arguing that Gemmini’s practical throughput is constrained by heavy control and memory-system inefficiencies. In OpenGeMM’s measurements, normalized throughput speedups over Gemmini range from 3.58× to 16.40× across GeMM workloads, and Gemmini’s average temporal utilization on the tested workloads is reported as 6.25%, attributed to intensive memory stalls (Yi et al., 2024). This is not a claim that Gemmini lacks architectural sophistication; rather, it suggests that array peak performance and sustained utilization can diverge sharply in a full system.

LEGO adopts a different critique. It treats Gemmini as the “state-of-the-art open-source NN accelerator generator” and a major end-to-end baseline, but classifies it as a template-based RTL generator with a design space limited to two templates. Under matched resources of 256 MACs, 256 KB on-chip buffer, a 128-bit memory bus, and 16 GB/s bandwidth, LEGO reports 3.2× speedup and 2.4× energy efficiency improvement over Gemmini, attributing the gain to a more accurate performance model and to hardware flexibility from generated function-unit interconnects and dynamic spatial dataflow switching (Lin et al., 15 Sep 2025). In this framing, Gemmini remains a strong baseline precisely because it is open, end-to-end, and competitive; its limitation is template rigidity rather than lack of practical value.

FEATHER offers a third comparative perspective. It treats Gemmini as an end-to-end practical baseline but groups it with accelerators that use a single fixed dataflow and lack on-chip layout reordering. In FEATHER’s evaluation methodology, Gemmini is characterized as “(fix, T, none)” with 1024 PEs, 512 bit/cycle on-chip bandwidth, int8 datatype, and FireSim evaluation on AWS F1 at 50 MHz with latency scaled to 100 MHz. FEATHER then reports a 3.91× geomean normalized throughput improvement over Gemmini, attributing the gap to Gemmini’s fixed weight-stationary dataflow with degree of parallelism 16 in both $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$6 and $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$7, which underutilizes the array when $$C_{[M,N]} = A_{[M,K]} \times B_{[K,N]} + D_{[M,N]}.$$8 is not divisible by 16 (Tong et al., 2024). Because the platforms and implementation flows differ, this comparison is best read as directional rather than as a strict iso-platform ranking.

Several misconceptions are clarified by this broader literature. One is that Gemmini should be understood as a single accelerator instance; the evidence instead consistently presents it as a generator and full-stack infrastructure (Genc et al., 2019). Another is that openness or configurability implies universal optimality. Gemmini’s own evaluations already show weaker mapping efficiency on low-reuse operators such as MobileNetV2 depthwise convolutions (Genc et al., 2019), and later FPGA work argues that baseline Gemmini is not FPGA-optimal without substantial retuning, feature pruning, and DSP-aware redesign (Peccia et al., 2024). A third misconception is that Gemmini’s public documentation fully captures its hardware/software contract; ATLAAS shows that important hardware-visible features such as multi-bank DMA configuration, pooling semantics, and im2col support were absent from the hand-written reference specification (Gao et al., 15 Apr 2026).

Taken together, these comparisons position Gemmini as a durable reference point in accelerator research. Its enduring importance lies less in claiming a universally best microarchitecture than in providing an open, configurable, full-stack platform against which new ideas in architecture, compilation, modeling, verification, and automation can be stated precisely and tested reproducibly.