Weight-Stationary Streaming Dataflow

• Weight-Stationary Streaming Dataflow is an architectural paradigm that fixes DNN weights locally in processing elements while streaming activations to maximize reuse and reduce off-chip memory traffic.
• It achieves significant energy-delay improvements, with quantitative gains such as up to 40% reduction in EDP and 47% lower latency in 3D systolic array implementations.
• The approach is realized in diverse platforms like FPGAs and programmable dataflow engines, offering practical insights for optimizing DNN acceleration in edge and high-density systems.

Weight-stationary streaming dataflow is an architectural and microarchitectural methodology central to efficient DNN acceleration, particularly in systolic arrays, FPGA implementations, and recent programmable engines. It minimizes movement of filter weights by pinning each weight locally in the processing element (PE), streaming activations through the array and accumulating partial sums, thereby maximizing weight reuse, reducing off-chip memory bandwidth requirements, and optimizing energy–throughput trade-offs. The term encompasses a spectrum of realizations—from constrained FPGA designs to monolithic 3D systolic arrays and decoupled access/execute streaming engines—each mapping the fundamental “weights-stationary, streaming activations” paradigm to different hardware fabrics and system goals (Shukla et al., 2024, Yi et al., 18 Apr 2025, Li, 13 May 2025).

1. Formal Model and Canonical Workflow

The weight-stationary (WS) streaming dataflow formally assigns each convolution weight to a local buffer in a PE for the duration of a workload tile. Weights are loaded once per tile and then remain fixed, while activations are successively streamed and multiplied, maximizing the reuse factor. Given a convolutional layer with $K \times K$ kernel, $C_\mathrm{in}$ input channels, $C_\mathrm{out}$ output channels, and output size $H_\mathrm{out} \times W_\mathrm{out}$:

• Each weight $w_{k, c_\mathrm{in}, c_\mathrm{out}}$ is loaded once and reused for all output spatial locations, yielding a reuse factor $R_\mathrm{reuse} = H_\mathrm{out} W_\mathrm{out}$
• Memory movement cost is minimized to

$$C_\mathrm{move} = \text{cost}_\mathrm{readW} |W| + \text{cost}_\mathrm{readA} (N_\mathrm{tile} \cdot |A|) + \text{cost}_\mathrm{writeP} |P|$$

where $|W| = K K C_\mathrm{in} C_\mathrm{out}$, $|A| = H_\mathrm{in} W_\mathrm{in} C_\mathrm{in}$, and $|P| = H_\mathrm{out} W_\mathrm{out} C_\mathrm{out}$.

A typical loop nest for a WS-Streaming $C_\mathrm{in}$0 convolution follows: $w_{k, c_\mathrm{in}, c_\mathrm{out}}$4 Inner MAC loops are typically fully unrolled/pipelined (II = 1) in FPGA HLS or hand-coded RTL flows (Li, 13 May 2025).

2. Weight-Stationary Streaming in Monolithic 3D Systolic Arrays

WS-MONO3D implements a WS streaming dataflow within a monolithic 3D systolic array, leveraging ultra-dense vertical interconnects and multi-tier memory (Shukla et al., 2024).

3D Integration Stack

• 6-tier flip-chip stack:
  • Tier 0: 256×256 PE array (MAC units, local pipeline)
  • Tier 1: 2 MB IFMAP/OFMAP SRAM banks (16 × 128 KB each)
  • Tiers 2–5: 8 MB RRAM per tier for weights (64 × 128 KB banks)
  • Dense monolithic inter-tier vias (MIVs): $C_\mathrm{in}$110 ps, $C_\mathrm{in}$20.05 fJ/bit vertical hop

Streaming MAC Sequence

Compared to classic 2D WS, which requires multiple cycles to preload weights ($C_\mathrm{in}$3), shift IFMAPs ($C_\mathrm{in}$4), and MAC/forward psums ($C_\mathrm{in}$5):

• WS-MONO3D enables all weights and all IFMAPs for each fold to be available in two cycles (one cycle for weights, one for IFMAPs), then $C_\mathrm{in}$6 steady-state MAC cycles:

$C_\mathrm{in}$7

• Results: Elimination of weight/input forwarding cycles, enabled by vertical RRAM/SRAM+SRAM banks and IFMAP multicasting

Quantitative Results

• Energy–delay product (EDP): $C_\mathrm{in}$840% (avg. 32%) at 85 °C thermal budget relative to 2D baseline
• Latency: $C_\mathrm{in}$947% (avg. 41%)
• IPS/W/footprint: $C_\mathrm{out}$010× (6× area, 1.7× power)
• Design supports high-throughput, area-efficient edge DNN acceleration (Shukla et al., 2024)

Limitations and Trade-Offs

• Higher thermal densities: frequency must be throttled under strict budgets, reducing EDP gain
• Fabrication complexity and yield risks due to MIVs
• Only WS explored for 3D; other dataflows (output-stationary, partial-sum stationary) remain open research topics

3. Programmable Weight-Stationary Streaming in Dataflow Engines

DataMaestro (Yi et al., 18 Apr 2025) demonstrates a programmable architectural realization of WS streaming dataflow within a decoupled access/execute (DAE) GEMM accelerator:

Access/Execute Separation

• GEMM datapath: 3D PE array (e.g., $C_\mathrm{out}$1)
• Five DataMaestro streaming engines (“access” side): manage address generation, memory interface, prefetch buffering, and on-the-fly data manipulation

Programmable Affine Access Patterns

• Address generation is fully programmable: $C_\mathrm{out}$2-dimensional affine loop-nest model for both weight and activation streaming

$C_\mathrm{out}$3

• Enables WS mapping by holding weight tiles fixed in PE registers while activations are streamed in, supporting variable tiling/unrolling strategies for both convolutional and matrix-multiply workloads

Run-Time Bank-Conflict Avoidance and Prefetch

• Fine-grained memory prefetch and addressing-mode switching (Fully-Interleaved, Grouped-Interleaved, Non-Interleaved) mitigate bank conflicts and maximize sustainable streaming bandwidth

On-the-Fly Data Manipulation

• In-stream transposer, broadcaster, and quantizer units: reshape, broadcast, or quantize streaming operands without extra scratchpads

Evaluation

• GEMM utilization: nearly 100%
• Throughput: 1.05–21.39× improvement over prior programmable dataflow solutions
• Area/power: All five DataMaestro units occupy 6.43% of total accelerator area, 15.06% of power (Yi et al., 18 Apr 2025)

4. FPGA-Optimized WS-Streaming: Design Patterns and Case Studies

Comprehensive reviews highlight the WS-Streaming pattern’s effectiveness for edge-AI FPGA accelerators (Li, 13 May 2025). Core attributes include:

• Each PE houses a local weight buffer (LUTRAM/BRAM/register), holding $C_\mathrm{out}$4 weights per output channel
• Streaming interconnect (e.g., AXI-Stream) provides high-throughput delivery of activations; partial sums either accumulate in situ (output-stationary variant) or propagate downstream

Real Examples

Accelerator	Platform	Dataflow	LUTs	DSPs	BRAM
FINN-conv3x3	Zynq-7020 (28nm)	WS-Streaming	12,500	0	18
FINN-R	VU9P (16nm)	WS-Streaming	56,000	0	40

• Binarized conv ($C_\mathrm{out}$5-bit weights/activations): $C_\mathrm{out}$6 G bit-ops/s at $C_\mathrm{out}$7 W ($C_\mathrm{out}$8 G ops/W), $C_\mathrm{out}$954 $H_\mathrm{out} \times W_\mathrm{out}$0s latency per inference (Li, 13 May 2025)

Design Guidelines

• Select tile sizes such that $H_\mathrm{out} \times W_\mathrm{out}$1, $H_\mathrm{out} \times W_\mathrm{out}$2 fit within physical memory tiers
• Double-buffer activations; partition weight arrays across PEs; fully unroll inner MAC loops (II = 1)
• Streaming matches fine-grained, pipelinable hardware flows and maximizes LUT/DSP efficiency

5. Analytical Performance and Energy Models

Both custom monolithic 3D and programmable dataflow engines present generalized analytical models for WS-Streaming:

Energy-Delay Product (EDP) Model (Shukla et al., 2024)

• EDP: $H_\mathrm{out} \times W_\mathrm{out}$3, $H_\mathrm{out} \times W_\mathrm{out}$4
• In 3D:

$H_\mathrm{out} \times W_\mathrm{out}$5

$H_\mathrm{out} \times W_\mathrm{out}$6

Utilization and Stall Models (Yi et al., 18 Apr 2025)

• GEMM core utilization:

$H_\mathrm{out} \times W_\mathrm{out}$7

• Bandwidth-limited stall term tied to effective reduction in memory accesses ($H_\mathrm{out} \times W_\mathrm{out}$8) and data-path optimizations:

$H_\mathrm{out} \times W_\mathrm{out}$9

• Overall energy model for GEMM operation:

$w_{k, c_\mathrm{in}, c_\mathrm{out}}$0

Temperature Effects (for 3D stacks)

• Leakage and wire-resistance increase with temperature $w_{k, c_\mathrm{in}, c_\mathrm{out}}$1:

$w_{k, c_\mathrm{in}, c_\mathrm{out}}$2

$w_{k, c_\mathrm{in}, c_\mathrm{out}}$3

• Monolithic 3D integration imposes thermal density constraints, requiring budget-aware DVFS or advanced cooling to sustain top performance (Shukla et al., 2024).

6. Comparative Dataflow Taxonomy and Trade-offs

WS-Streaming is situated alongside output-stationary (OS) and row-stationary (RS) dataflows:

Dataflow	Stationary Data	Movement Cost	Suited Applications
Weight-Stationary	Weights	Min weight fetch; high activation traffic	Convolution with high weight reuse, quantized nets
Output-Stationary	Partial Sums	Max weight/activation traffic	High precision or accumulation-intensive layers
Row-Stationary	Input activations/patches	Balance W/A reuse, complex buffers	Architectures optimizing overall bandwidth

WS-Streaming generally achieves the lowest off-chip weight-memory cost for typical 3×3 conv layers in quantized designs, making it preferable for edge FPGAs and emerging 3D DNN accelerators (Li, 13 May 2025).

7. Limitations and Research Trends

• Increased integration density (3D/stacked) exacerbates thermal management and manufacturing complexity; frequency scalars and EDP gains may taper under strict temperature budgets (Shukla et al., 2024)
• RRAM endurance and multi-tier fabrication cost are nontrivial at scale; trade-offs remain underexplored
• Programmable engines (e.g., DataMaestro) show generalized applicability but may require careful memory-architecture co-design to achieve sustained WS-Streaming bandwidths
• Hybrid or alternative dataflows (e.g., output-stationary in 3D, partial-sum stationary) are promising for future studies
• Tools such as MAESTRO and Timeloop facilitate simulation and optimization of WS-Streaming reuse, bandwidth, and tiling for various fabrics and workloads (Li, 13 May 2025)

---

References:

• (Shukla et al., 2024) "A New Dataflow Implementation to Improve Energy Efficiency of Monolithic 3D Systolic Arrays"
• (Yi et al., 18 Apr 2025) "DataMaestro: A Versatile and Efficient Data Streaming Engine Bringing Decoupled Memory Access To Dataflow Accelerators"
• (Li, 13 May 2025) "Dataflow & Tiling Strategies in Edge-AI FPGA Accelerators: A Comprehensive Literature Review"