Voltra: 16nm DNN Accelerator SoC for Edge AI

• Voltra is a 16 nm digital DNN accelerator SoC designed with a 3D spatial GEMM array and unified shared memory enabling high utilization across various AI workloads.
• Its design integrates flexible data streamers, mixed-grained prefetching, and programmable dynamic memory allocation to reduce latency and memory traffic.
• Benchmark evaluations show 69.71–100% spatial utilization and up to 2.94× improved temporal performance over conventional designs, optimizing energy and area efficiency.

Voltra is a 16 nm digital DNN accelerator SoC designed around a highly utilized General Matrix Multiplication (GEMM) core plus an aggressively optimized on-chip memory system. It is aimed at edge-class designs and seeks to provide high energy efficiency, high area efficiency, and high utilization across a wide variety of AI workloads, including CNNs, point cloud networks, RNNs, transformers, and LLM inference. Its defining architectural elements are a 3D spatial GEMM array, a unified shared memory accessed through flexible data streamers, mixed-grained data prefetching, and programmable dynamic memory allocation (Yi et al., 11 Feb 2026).

1. System scope and design objectives

Voltra is intended to sustain high utilization across diverse layer shapes rather than optimize only for a narrow workload family. The reported workload set includes ResNet-50, MobileNetV2, ViT, PointNeXt, LSTM, BERT-Base, and both prefill and decode stages of LLaMA3.2-3B. The chip is squarely positioned as an edge-class design: it has a die area of 0.654 mm², operates over 300–800 MHz, and integrates 128 KB of on-chip SRAM with DRAM behind a DMA engine.

The central design goals are fourfold. First, the architecture targets high spatial utilization of the MAC array across diverse shapes, including CNN, Transformer, GEMM, and GEMV cases. Second, it targets high temporal utilization despite using banked shared memory rather than fixed operand-specific buffers. Third, it aims to preserve flexibility through a programmable yet lightweight interface. Fourth, it seeks system-level energy and area efficiency under realistic tiled, off-chip workloads rather than only under idealized on-chip-resident cases.

These goals are reflected in the system decomposition. Three main compute blocks are attached to a multi-bank shared SRAM: a GEMM core, a quantization SIMD unit, and a data reshuffler. A 32-bit integer RISC-V Snitch core orchestrates operation through configuration registers, while a DMA engine handles off-chip transfers. This suggests a design philosophy in which utilization, rather than raw peak MAC count alone, is treated as the primary determinant of effective performance.

2. 3D spatial GEMM architecture

The GEMM core implements matrix multiplication as

$C_{M\times N} = A_{M\times K} \cdot B_{K\times N}.$

Its compute fabric contains 512 MACs organized as an 8×8×8 structure. Programmatically, the core can be understood as an 8×8 array of Dot-ProdUs, with each Dot-ProdU computing an 8-element dot product,

$\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$

This yields a three-dimensional unrolling structure: two spatial dimensions across the Dot-ProdU array and an internal vector dimension of width 8 along the reduction axis.

The paper refers to this organization as 3D spatial data reuse. One dimension corresponds to one spatial dimension of the Dot-ProdU array, a second dimension corresponds to the other spatial dimension, and the third dimension corresponds to the internal vector dot-product dimension. Vectors from $A$ and $B$ are broadcast horizontally and vertically across the 8×8 Dot-ProdU grid, while each Dot-ProdU computes and accumulates 8 partial products combinationally. As a result, the architecture reuses rows of $A$ across multiple $N$-columns, reuses columns of $B$ across multiple $M$-rows, and also reuses data along an 8-element segment of $K$.

The dataflow is output-stationary. Partial sums for $C$ remain in local registers inside the array; inputs and weights are streamed in and reused while partial sums stay local until completion. A hardware loop controller, programmed by the RISC-V core, determines when accumulators are cleared based on matrix dimensions.

The principal motivation for the 3D organization is the utilization loss observed in conventional 2D arrays when tensor dimensions do not align well with an 8×8 PE topology. By splitting $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$0 spatially as well and balancing unrolling across three dimensions, Voltra fits more combinations of $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$1 to the physical 8×8×8 structure, reduces idle PEs when dimensions are small or irregular, and distributes reduction work between spatial and temporal domains more effectively. Across eight diverse workloads, spatial utilization is reported as 69.71–100%, with up to 2.0× improvement over a 2D spatial array baseline; the notably lower case is LLM decode, where many GEMV-like operations yield 69.71% spatial utilization (Yi et al., 11 Feb 2026).

3. Unified shared memory and streamer subsystem

Voltra’s on-chip SRAM is a shared, multi-bank memory rather than a set of dedicated input, weight, and output buffers. The memory comprises 32 banks, each 64 bits wide, for a total capacity of 128 KB. Inputs, weights, partial sums, outputs, and intermediate tensors all reside in the same address space. This organization improves capacity utilization and enables dynamic allocation, but it also creates a central temporal-utilization challenge: simultaneous accesses by multiple functional units can induce heavy bank contention.

The architecture addresses that challenge with data streamers placed between the compute blocks and the shared memory. These streamers include multi-dimensional address generation units, memory interface controllers, and per-channel FIFOs. The input streamer has a 6-D AGU, while the weight streamer has a 3-D AGU. Their addresses are programmable affine functions whose base addresses, loop bounds, and strides are written via CSR by the RISC-V core. The 6-D AGU is explicitly used to support strided and blocked GEMM access as well as implicit im2col for all Conv2D variants, including arbitrary stride, kernel, and in/out channels.

A key mechanism is mixed-grained data prefetching (MGDP). Each streamer has multiple memory access channels, and when a FIFO is not full, the memory interface controller prefetches the next chunk from SRAM according to the AGU. Operand types use different granularity. The input streamer uses 64-bit channels, which are suitable for strided Conv2D accesses and avoid over-fetch. The weight streamer uses a 512-bit channel width formed by combining 8 banks into a super bank, matching the more regular and coalesced access patterns of weights. Partial-sum and output streamers have only 1-depth FIFOs because output-stationary GEMM already maximizes their temporal reuse and because those streamers are time-multiplexed.

MGDP is complemented by Programmable Dynamic Memory Allocation (PDMA). Because SRAM is not statically partitioned, software running on Snitch assigns regions of the 128 KB to current operands, programs streamer base addresses, and reuses regions when sub-computations complete. In the BERT-Base multi-head attention example, the same SRAM is reused across phases for input tokens, $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$2 matrices, intermediate $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$3 scores, and attention outputs. The weight streamer also includes an on-the-fly $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$4 transposer for attention and related GEMMs, avoiding a separate transpose unit and reducing memory traffic. In the reported MHA example, PDMA yields a 14.3% reduction in total memory access count, and across workloads PDMA plus MGDP yields a 1.15–2.36× total latency reduction relative to a separated-memory architecture. Temporal utilization with MGDP reaches 76.99–97.32%, corresponding to a 2.12–2.94× improvement over a non-prefetching shared-memory baseline (Yi et al., 11 Feb 2026).

4. Programming model and workload mapping

Control is exercised by a 32-bit integer RISC-V Snitch core that configures streamers, programs GEMM dimensions and tiling factors, orchestrates DMA transfers, and controls auxiliary units such as quantization, reshuffling, and pooling. The interface is low-level but systematic: nested-loop tensor operations are mapped to multi-dimensional affine address generators through CSR configuration sequences. The paper does not describe a full compiler, but it explicitly notes that predecessors such as OpenGeMM, DataMaestro, and ZigZag suggest a stack in which an algorithm is transformed by a high-level mapping tool into a tiling strategy and then into CSR configuration sequences.

GEMM, fully connected layers, and attention map directly to the GEMM core. The $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$5, $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$6, and $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$7 dimensions are tiled so that submatrices fit into the 128 KB memory, and $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$8, $\text{DotProdU}: \sum_{k=0}^{7} a_k \cdot b_k.$9, $A$0, and projection matrices in multi-head attention are all handled as GEMMs. For $A$1, the transpose is generated on the fly by the weight streamer.

Conv2D is implemented through implicit im2col. For a standard convolution,

$A$2

the input streamer’s 6-D AGU generates the access pattern corresponding to im2col rows without materializing the expanded matrix. The data reshuffler can convert layouts such as HWC to C/8HWC8 so that the resulting accesses are more bank-friendly. Depthwise convolution and pointwise convolution are also handled through implicit im2col plus GEMM mapping, with tiling selected to match the specific reuse structure.

RNN workloads, exemplified by LSTM, are treated as multiple GEMMs per time step. Tiling is chosen to reuse recurrent weights across multiple time steps while respecting the 128 KB memory limit. LLM prefill is reported as a good fit because it contains large batch or sequence GEMMs for attention and MLP blocks. LLM decode is less ideal because it contains many small GEMVs and small-$A$3 GEMMs, which explains the lower spatial utilization. Maxpool is implemented by a dedicated unit with eight comparison lanes and programmable window sizes.

The preferred operating region follows directly from the array structure and output-stationary dataflow. Utilization is higher when $A$4, $A$5, and $A$6, or their tiled versions, are multiples of 8. Larger $A$7 improves temporal reuse, while very small or highly irregular shapes cannot fully occupy the 8×8×8 structure. A plausible implication is that Voltra’s flexibility depends not only on the hardware itself but also on mapping quality, particularly for workloads that fall outside regular GEMM-like regimes.

5. Physical implementation and microarchitectural economy

Voltra is fabricated in 16 nm FinFET technology and operates over a 0.6–1.0 V supply range at 300–800 MHz. The following implementation parameters are explicitly reported.

Parameter	Value
Process	16 nm FinFET
Die area	0.654 mm²
Supply voltage range	0.6–1.0 V
Frequency range	300–800 MHz
On-chip SRAM	128 KB, 32 × 64-bit banks
MAC array	512 MACs, 8×8 Dot-ProdUs, 8-element dot product each
Control core	32-bit integer RISC-V Snitch
Compute precision	32-bit accumulations with 8-bit quantized output

Peak performance is reported on a fully dense GEMM with $A$8. The chip achieves 1.60 TOPS/W peak system energy efficiency at 0.6 V and 300 MHz, and 1.25 TOPS/mm² system area efficiency at 1.0 V and 800 MHz. A shmoo plot across 0.6–1.0 V shows stable operation up to 800 MHz. Energy efficiency increases with higher weight sparsity, lower input toggle rate, and larger matrix sizes, especially larger $A$9, due to increased temporal reuse under the output-stationary dataflow.

An important part of the area-efficiency result is the use of time-multiplexed auxiliary structures. The quantization SIMD unit has only 8 parallel lanes although the GEMM core produces 64 outputs; quantization is therefore spread across 8 cycles under a hardware loop unroller. This incurs a reported 0.7% performance loss on ResNet-50 while reducing SIMD area by 4.92× relative to a 64-lane design. Partial-sum and output streamers also share access ports to the memory crossbar in a time-multiplexed fashion, with partial-sum reads prioritized over output writes. This incurs only 0.02% performance loss on ResNet-50 while reducing crossbar area by 1.46×. These choices are explicitly identified as crucial to achieving the reported 1.25 TOPS/mm² at such a small die area (Yi et al., 11 Feb 2026).

6. Empirical evaluation and comparative position

Evaluation spans eight representative workloads: MobileNetV2, ResNet-50, ViT-B, PointNeXt, LSTM, BERT-Base with token size 512, and LLaMA3.2-3B prefill and decode with token size 256. Because 128 KB cannot hold full-layer operands, all experiments use realistic layer-wise tiling. Utilization metrics are measured within tiles, while total latency includes both GEMM core cycles and off-chip DMA cycles obtained through cycle-accurate RTL simulation.

The ablation methodology separates three architectural questions. Spatial utilization is evaluated by comparing the 3D array against a 2D array baseline. Temporal utilization is evaluated by comparing MGDP-enabled shared memory against a plain shared-memory design without prefetching. End-to-end latency is evaluated by comparing the shared-memory-plus-PDMA organization against a separated-memory architecture with distinct operand buffers and dispatchers.

The reported results attribute distinct benefits to each mechanism. The 3D array improves spatial utilization to 69.71–100% across workloads and yields up to 2.0× improvement relative to the 2D baseline. MGDP raises temporal utilization to 76.99–97.32% and yields a 2.12–2.94× improvement over non-prefetching shared memory. PDMA reduces off-chip communication by enabling larger tiles and in-place reuse, and together with MGDP and the 3D array yields a 1.15–2.36× reduction in overall execution time relative to a separated-memory system once tiling and off-chip transfers are included.

The comparative discussion places Voltra alongside DIANA, Marsellus / REB, Ayaka, Cygnus, and broader literature that includes NVDLA and RISC-V-based ML SoCs. Within that comparison set, Voltra is described as the only one among those chips that efficiently supports a wide set of workloads spanning CNNs, RNNs, transformers, and LLM inference, all with high utilization. On dense GEMMs, its system energy efficiency is reported as competitive with these chips, while it has the smallest chip area and the highest system area efficiency among the compared accelerators, at 1.25 TOPS/mm² (Yi et al., 11 Feb 2026).

7. Trade-offs, limitations, and broader implications

Voltra’s design makes several explicit trade-offs. The 3D spatial dataflow increases mapping complexity relative to a conventional 2D array, because $B$0, $B$1, and $B$2 must be partitioned across an 8×8×8 resource rather than mapped onto only two spatial dimensions. The benefit is measurable utilization gain, but the cost is a stronger dependence on mapping methodology and tools that understand 3D partitioning.

The shared-memory design with PDMA improves flexibility and SRAM utilization, reduces total storage needs for a given tiling strategy, and can reduce off-chip transfers. Its cost is increased susceptibility to bank conflicts and a more complex access subsystem. Voltra mitigates these costs through multi-dimensional AGUs, reshuffling, FIFO-based decoupling, and mixed-grained prefetching. The result is that a unified memory organization, which might otherwise be assumed to limit throughput, attains temporal utilization of up to 97.32%. A common misconception is that any shared SRAM architecture necessarily sacrifices utilization relative to separated operand buffers; the reported measurements show that this is not generally true when the access engines are sufficiently decoupled and programmable.

The area-reduction strategy based on time-multiplexing also carries an inherent trade-off. Quantization lanes and shared-memory ports are narrower or fewer than the peak output bandwidth of the main array, which could in principle reduce throughput. In the measured cases, however, the reported penalties are negligible—0.7% and 0.02% on ResNet-50—while the area reductions are 4.92× and 1.46×, respectively. This suggests that non-critical parallel units can be compressed aggressively when the principal bottleneck remains the main GEMM array and its data supply.

The paper also identifies clear limitations. Small or irregular workloads, especially GEMV-heavy decode phases in LLM inference, do not fully exploit the 3D array and therefore exhibit lower spatial utilization. There is no explicit sparse compute engine: although efficiency improves with higher weight sparsity and lower input toggle rate, the architecture does not skip MACs in the manner of dedicated sparse accelerators. The software burden is also non-trivial. The architecture is flexible, but practical deployment likely requires an automated mapping tool, and the paper does not detail that tooling in depth.

The broader implication is that high utilization across CNNs, RNNs, transformers, and LLM inference can be pursued through a combination of three specific ideas: three-dimensional spatial unrolling for GEMM-centric computation, unified shared memory with programmable streamers, and selective time-multiplexing of non-critical hardware. In that sense, Voltra’s main contribution is not a single isolated block, but the demonstration that these mechanisms can be combined in a very small 16 nm accelerator while maintaining competitive system-level energy and area efficiency.