ProDF: Programmable DataFlow for SSMs

• ProDF is a programmable dataflow architecture designed for mapping structured state-space models onto 2-D systolic arrays with reconfigurable inter-PE communication.
• It fuses recurrent and output computations in a single tiled sweep, employing weight-stationary caching and per-timestep diagonal data streaming to optimize performance.
• ProDF achieves substantial improvements in throughput, latency, and energy efficiency, making it adaptable for both SSM and standard deep learning workloads.

ProDF (Programmable DataFlow) is an advanced dataflow architecture developed for efficient execution of structured State-Space Models (SSMs) such as S4 and Liquid-S4 on 2-D systolic arrays, specifically as a core component of EpochCore's LIMA-PE accelerator. By introducing a reconfigurable inter-PE communication strategy—encompassing non-traditional diagonal flows together with standard east-west and north-south interconnects—ProDF enables highly efficient, high-utilization execution of SSM layers for long-sequence inference, while also generalizing to standard DNN workloads. Key innovations in ProDF include fusing recurrence and output stages in a single tiled sweep, weight-stationary per-PE caching, and per-timestep diagonal data streaming, yielding marked improvements in throughput, latency, and energy efficiency (Raja et al., 29 Jul 2025).

1. Architectural Definition and Core Objectives

ProDF is explicitly designed to address the peculiar requirements of mapping structured SSM layers, which follow the operations

• State update: $x_{k+1} = \overline A \odot x_k + \overline B u_k$
• Output computation: $y_k = C x_k + D u_k$

to systolic arrays in a manner that maximizes compute density and minimizes data movement. The architecture encompasses:

• A set of per-PE operating modes: FRI-MAC, TRI-MAC, BWS-MAC, TOS-MAC, Pass-Through, and Sleep, allowing each processing element (PE) to dynamically switch function according to the dataflow phase.
• A unified inter-PE interconnect: Supports classical north-south (N→S), west-east (W→E), and critically, a programmable northeast→southwest (NE→SW) data path, with no physical re-wiring necessary.
• A controller-programmed schedule: Orchestrates the sequence of compute, data-load, and result-read phases in three tightly pipelined stages.

The explicit objectives are to (1) fuse the SSM’s recurrence and output stages into a single execution sweep, (2) amortize weight loads through weight-stationary caching, (3) exploit diagonal dataflow for rapid, one-cycle-per-token state updates, and (4) maximize PE utilization and data reuse for both SSM and standard GEMM layers (Raja et al., 29 Jul 2025).

2. Mapping, Tiling, and Scheduling Strategy

SSM layers with state dimension $N$ (and typically $H = 1$ output head) are mapped onto an $(N+2) \times (N+1)$ tile of LIMA-PEs. ProDF employs a two-phase execution:

• Pre-Load Phase: Each PE's Stationary and Control Caches are loaded with required weights and control bits in $(N+2)\times(N+1)$ cycles. Specific assignments per tile row govern which weights and modes are held stationary.
• Compute Phase: At each timestep $k=0,\ldots,T-1$, the input $u_k$ is injected, activating a cascade of operations across diagonally, vertically, and horizontally interconnected PEs:
  • Row 0: BWS-MAC mode multiplies $u_k$ by $\overline B_j$ and dispatches partials diagonally NE→SW into Row 1.
  • Row 1: FRI-MAC (or TRI-MAC for Liquid-S4) computes the SSM recurrence.
  • Rows 2 to N+1: Standard weight-stationary MAC performs the output $y_k = C x_k + D u_k$0 aggregate.
  • Row N+1, column 0: Produces final $y_k = C x_k + D u_k$1 after adding $y_k = C x_k + D u_k$2, which is written back to output SRAM.

Pre-load of weights and controls is batched and conducted only once per tile, leveraging SRAM's efficiency over repeated DRAM accesses (Raja et al., 29 Jul 2025).

3. Data-Movement and Interconnect Patterns

ProDF reconfigures per-PE data exchange in real time to match the distinctive data dependency structure of SSMs. The key flows are:

• Row 0: Receives $y_k = C x_k + D u_k$3 (W→E), then streams NE→SW partials.
• Row 1: Receives NE→SW partials from Row 0, then produces $y_k = C x_k + D u_k$4, streaming results N→S to Rows 2 to N+1.
• Rows 2–N+1: Accept $y_k = C x_k + D u_k$5 (W→E) and partial sums (N→S) for output accumulation.
• Final Output: Row N+1, column 0 dispatches $y_k = C x_k + D u_k$6 directly to output SRAM.

Buffer management assigns 16 MB each for weight and input/output SRAMs, with double-buffering to pipeline $y_k = C x_k + D u_k$7 and $y_k = C x_k + D u_k$8 at one word per cycle. Stationary caches within PEs retain weights and controls, with moving caches holding $y_k = C x_k + D u_k$9 and partial sums. This arrangement supports high bandwidth amortization and near-peak bandwidth utilization (Raja et al., 29 Jul 2025).

4. Performance Models and Architectural Comparison

ProDF delivers marked improvements in PE utilization, memory bandwidth, and throughput over conventional dataflows (Output-Stationary/OS, Weight-Stationary/WS, and Input-Stationary/IS):

Dataflow	Weight Loads per Tile	Outputs per Cycle	Avg PE Utilization
OS	N+1	1/N	50%
WS	N+1	1/N	65%
IS	N+1	1/N	65%
ProDF	1	1	90–95%

Throughput for a long sequence ($N$0) approaches the clock rate $N$1 in elements/second; first output emerges after $N$2 cycles, followed by one output per cycle. Effective PE utilization ($N$3) is measured at $N$4, compared to $N$5 for typical OS/WS mappings. ProDF reduces weight SRAM bandwidth by approximately $N$6 relative to OS/WS, and halves the number of idle PEs (Raja et al., 29 Jul 2025).

5. Energy Efficiency and Empirical Results

Experimental evaluation across six LRA sequence modeling tasks (where $N$7 ranges from 1,000 to 65,000 and $N$8) demonstrates:

• Latency: ProDF/EpochCore achieves $N$9 speedup over Nvidia A100 GPUs, $H = 1$0 over Sparse-SA, and $H = 1$1 over 1-D FFT-SA baselines.
• Energy: Per-inference energy is $H = 1$2 lower than Sparse-SA, and $H = 1$3 lower than FFT-SA.
• System-level mixed workload: Integrating “TPU+EpochCore” (for DNN and SSM, respectively) yields about $H = 1$4 total system-wide energy reduction, although DNN layers on EpochCore add $H = 1$5 overhead offset by SSM gains.
• PE-level metrics: LIMA-PEs incur a $H = 1$6 area and $H = 1$7 power overhead versus traditional PEs, but dual clock-gating enables $H = 1$8 energy saving in Sleep versus Accumulation modes (Raja et al., 29 Jul 2025).

These results suggest that ProDF's dataflow strategy effectively exploits both the sparsity and banded structure of SSM recurrences, yielding orders-of-magnitude improvements in inference latency and energy.

6. Applicability, Generalizability, and Limitations

ProDF is explicitly architected to unlock high-performance SSM inference, particularly for structured S4 and Liquid-S4 models, but is also readily extensible to conventional GEMM-based DNN/CNN/Transformer layers, due to its programmable sequencing and PE mode flexibility. The architectural design achieves weight-stationary operation, minimal SRAM bandwidth, and linear per-token throughput for both structured and unstructured layers.

A plausible implication is that the principal limitation of ProDF lies in the $H = 1$9 area and moderate power growth per PE, demanded by its increased microarchitectural flexibility and interconnects. However, these are offset by the substantial throughput and energy advantages, especially for long-sequence, memory-intensive tasks (Raja et al., 29 Jul 2025).

7. Summary and Outlook

ProDF represents a generalizable, tiled, programmable dataflow that enables ultra-efficient execution of structured SSM layers and conventional deep learning operations within modern systolic arrays. By fusing recurrent and output computations, deploying diagonal NE→SW dataflows, and leveraging per-PE weight-stationarity, ProDF achieves over $(N+2) \times (N+1)$0 latency speedup, $(N+2) \times (N+1)$1 energy reduction, and $(N+2) \times (N+1)$2 higher utilization compared to prior dataflows, while marginally increasing area and power cost. Its integration in EpochCore's LIMA-PE stacks establishes a template for specialized inference hardware targeting both structured state-space and conventional neural architectures, thus extending the performance envelope of sequence modeling and temporal AI workloads (Raja et al., 29 Jul 2025).