EpochCore: Efficient Accelerator for State-Space Models

• EpochCore is a specialized hardware accelerator for state-space models and deep learning, featuring a programmable systolic array and custom processing elements.
• Its ProDF dataflow unifies weight preload and pipelined sequence processing to achieve up to 2,000× latency reduction and significant energy savings.
• Benchmark results show high throughput (~2.9 TOPS at 700 MHz) and scalability for long-sequence tasks, making it ideal for advanced SSM applications.

EpochCore is a specialized hardware accelerator designed for the efficient execution of state-space models (SSMs), particularly targeting long-range sequence modeling tasks. By integrating a programmable systolic array architecture with custom processing elements and a dedicated dataflow, EpochCore aims to vastly improve energy efficiency and throughput compared to conventional accelerators when operating on structured SSMs such as S4 and Liquid-S4, as well as standard deep neural network workloads (Raja et al., 29 Jul 2025).

1. System Architecture and Organization

EpochCore's architecture is organized around a PCIe-attached accelerator card, comprising the following key subsystems:

• Host CPU interface over PCIe
• On-chip static RAM: 16 MB for weights and 16 MB for I/O operations
• Controller unit responsible for instruction decoding and dataflow scheduling (ProDF)
• Two-dimensional systolic array (SA) composed of LIMA-PEs (processing elements), scalable in size (e.g., 64×64)
• Nonlinear/Normalization unit for activations and layer normalization

The systolic array is constructed with PEs arranged in rows and columns, supporting canonical West→East (row-wise) and North→South (column-wise) dataflows, with additional northeast→southwest diagonal links to enable banded-matrix and recurrent operations. The weight-stationary tiling scheme preloads weights into local PE SRAM before computation, eliminating off-chip reads during inference. Long input/output sequences are streamed through the array each cycle, supporting high-throughput for very long-sequence tasks (Raja et al., 29 Jul 2025).

2. LIMA Processing Element: Modes and Capabilities

Within the systolic array, each LIMA-PE features a microarchitecture that allows highly versatile operation:

• Dual gated clock domains: a load clock for bringing in weights and control bits, and a compute clock for operation execution.
• Local stationary cache (weights and control) and result cache (partial sums, $x(t)$).
• Four programmable MAC (multiply-accumulate) modes, set by control bits:
  1. Traditional GEMM (TOS-MAC): $Y_s \leftarrow Y_s + B_m \cdot C_m$
  2. Banded Weight-Stationary (BWS-MAC): $Y_m \leftarrow Y_m + B_s \cdot C_m$
  3. Fixed Recurrent Integration (FRI-MAC, for S4): $Y_s \leftarrow B_s \cdot Y_s + A_m$
  4. Time-Varying Recurrent Integration (TRI-MAC, for Liquid-S4): $Y_s \leftarrow (B_s + A_m) \cdot Y_s + A_m$

• Native support for real and complex datatypes (real and imaginary packed together).

• Pass-through and sleep modes for utilization and power optimization.

The LIMA-PE can thus natively implement both conventional neural network layers and the specialized computations required by SSMs. The underlying state-space models are discretized from their continuous ODE form, as exemplified by:

$\dot x(t) = A x(t) + B u(t)$

Zero-Order-Hold discretization gives: $$x_{k+1} = \overline{A} \odot x_k + \overline{B} u_k, \qquad \overline{A} = e^{-A \Delta t},\, \overline{B} = A^{-1}(I-e^{-A \Delta t}) B$$ And for Liquid-S4: $$x_{k+1} = (\overline{A}+\overline{B}u_k)\odot x_k+\overline{B}u_k$$ (Raja et al., 29 Jul 2025)

3. ProDF Dataflow: Programmable and Efficient

EpochCore implements the ProDF (Programmable DataFlow), which is specifically designed to:

• Stream long sequences through the systolic array, requiring zero off-chip bandwidth after weights are loaded
• Fuse S4’s three sub-steps—scalar-vector multiplication, recurrent update, linear projection—into a single pipelined pass through the SA
• Execute standard GEMM (WS/IS/OS) for DNN layers
• Leave all inter-PE wiring fixed, with all functional reconfiguration realized in PE microarchitecture

Typical SSM kernel scheduling in ProDF involves a preload phase for weights and control bits, followed by a compute phase where input sequences of length $T$ are processed in a pipelined series of SA rows, with each row specialized for a sub-step or passthrough operation. After the pipeline is filled, outputs are collected every cycle.

This pipeline is illustrated by the dataflow:

$Y_s \leftarrow Y_s + B_m \cdot C_m$6 The approach supports in-situ, streaming computation of SSMs, precluding the need for intermediate SRAM spills (Raja et al., 29 Jul 2025).

4. Performance, Energy, and Area Trade-offs

EpochCore demonstrates substantial accelerative gains over prior architectures:

• Peak performance for a 64×64 SA at 700 MHz: ~2.9 TOPS (32-bit)
• S4 token throughput approaches $T/(T+N+1) \times 700$M tokens/sec after warmup
• Latency:
  • Up to 250× faster than 2D sparse SAs
  • 25× faster than 1D FFT-SAs
  • ~2,000× faster than Nvidia A100 GPU kernels on LRA benchmarks, e.g., CIFAR-10 at 64K sequence length
• Energy efficiency:
  • 45× lower energy per inference vs. 2D sparse SAs
  • 10× lower vs. FFT-SAs
  • LIMA-PE dynamic power: 11.5 mW (compute), 6.7 mW (pass-through), 3.8 mW (sleep)
• Area:
  • LIMA-PE is 1.4–2.0× the size of a conventional PE
  • Overall systolic array has around 2× the area of an equivalently sized TPU-style SA (Raja et al., 29 Jul 2025)

5. Implementation and Scalability Aspects

The reference implementation is synthesized in Cadence Genus using FreePDK45 (45 nm), clocked at 700 MHz. The PE critical path incurs approximately a 5% slowdown due to clock gating. On-chip SRAM provides low-latency access (1.4 ns as per CACTI), and after initial weight preload there are no further off-chip bandwidth requirements during layer execution. Off-chip DRAM is accessed only between layers.

Scalability derives from the state-map size $Y_s \leftarrow Y_s + B_m \cdot C_m$0 and hidden size $Y_s \leftarrow Y_s + B_m \cdot C_m$1: SA dimensions grow as $Y_s \leftarrow Y_s + B_m \cdot C_m$2 to accommodate larger models, and multiple tiles can be deployed for batch or sequence partitioning. Typical utilization for practical $Y_s \leftarrow Y_s + B_m \cdot C_m$3 and $Y_s \leftarrow Y_s + B_m \cdot C_m$4 values remains above 80% (Raja et al., 29 Jul 2025).

6. Contribution, Limitations, and Prospects

EpochCore constitutes the first unified systolic array accelerator with:

• Native support for structured SSMs (S4, Liquid-S4) alongside DNN, CNN, and Transformer layers
• The LIMA-PE microarchitecture, offering four dynamically programmable MAC modes with real/complex support
• ProDF dataflow, enabling fully in-situ streaming SSM computation in a single SA pass
• Empirically demonstrated 250–2,000× latency reduction and 10–45× energy saving vs. prior SA and GPU baselines

Identified limitations include:

• Mamba-style SSMs, which require input-dependent recomputation of $Y_s \leftarrow Y_s + B_m \cdot C_m$5, with coefficient adaptation currently performed on the host; on-PE update units are a possible extension
• Support for gated SSMs would require new MAC modes implementing element-wise gating and variable time-steps
• Multi-dimensional SSMs (processing 2D input) would benefit from further reconfigurable inter-PE connections beyond the current NE→SW diagonal support

EpochCore provides a microarchitectural foundation for efficient hardware acceleration of long-range sequence models and compatibility with standard deep learning workloads, offering new avenues for SSM integration and hardware–algorithm co-design (Raja et al., 29 Jul 2025).