---
title: 'EpochCore: Efficient Accelerator for State-Space Models'
url: https://www.emergentmind.com/topics/epochcore
type: topic
---

# EpochCore: Efficient Accelerator for State-Space Models

EpochCore is a specialized hardware accelerator designed for the efficient execution of state-space models (SSMs), particularly targeting long-range sequence modeling tasks. By integrating a programmable systolic array architecture with custom processing elements and a dedicated dataflow, EpochCore aims to vastly improve energy efficiency and throughput compared to conventional accelerators when operating on structured SSMs such as S4 and Liquid-S4, as well as standard deep neural network workloads [2507.21394].

## 1. System Architecture and Organization

EpochCore's architecture is organized around a PCIe-attached accelerator card, comprising the following key subsystems:

- Host CPU interface over PCIe
- On-chip static RAM: 16 MB for weights and 16 MB for I/O operations
- Controller unit responsible for instruction decoding and dataflow scheduling (ProDF)
- Two-dimensional systolic array (SA) composed of LIMA-PEs (processing elements), scalable in size (e.g., 64×64)
- Nonlinear/Normalization unit for activations and layer normalization

The systolic array is constructed with PEs arranged in rows and columns, supporting canonical West→East (row-wise) and North→South (column-wise) dataflows, with additional northeast→southwest diagonal links to enable banded-matrix and recurrent operations. The weight-stationary tiling scheme preloads weights into local PE SRAM before computation, eliminating off-chip reads during inference. Long input/output sequences are streamed through the array each cycle, supporting high-throughput for very long-sequence tasks [2507.21394].

## 2. LIMA Processing Element: Modes and Capabilities

Within the systolic array, each LIMA-PE features a microarchitecture that allows highly versatile operation:

- Dual gated clock domains: a load clock for bringing in weights and control bits, and a compute clock for operation execution.
- Local stationary cache (weights and control) and result cache (partial sums, $x(t)$).
- Four programmable MAC (multiply-accumulate) modes, set by control bits:
    1. Traditional GEMM (TOS-MAC): $Y_s \leftarrow Y_s + B_m \cdot C_m$
    2. Banded Weight-Stationary (BWS-MAC): $Y_m \leftarrow Y_m + B_s \cdot C_m$
    3. Fixed Recurrent Integration (FRI-MAC, for S4): $Y_s \leftarrow B_s \cdot Y_s + A_m$
    4. Time-Varying Recurrent Integration (TRI-MAC, for Liquid-S4): $Y_s \leftarrow (B_s + A_m) \cdot Y_s + A_m$
- Native support for real and complex datatypes (real and imaginary packed together).
- Pass-through and sleep modes for utilization and power optimization.

The LIMA-PE can thus natively implement both conventional neural network layers and the specialized computations required by SSMs. The underlying state-space models are discretized from their continuous ODE form, as exemplified by:

\[
\dot x(t) = A x(t) + B u(t)
\]
Zero-Order-Hold discretization gives:
\[
x_{k+1} = \overline{A} \odot x_k + \overline{B} u_k, \qquad \overline{A} = e^{-A \Delta t},\, \overline{B} = A^{-1}(I-e^{-A \Delta t}) B
\]
And for Liquid-S4:
\[
x_{k+1} = (\overline{A}+\overline{B}u_k)\odot x_k+\overline{B}u_k
\]
[2507.21394]

## 3. ProDF Dataflow: Programmable and Efficient

EpochCore implements the ProDF (Programmable DataFlow), which is specifically designed to:

- Stream long sequences through the systolic array, requiring zero off-chip bandwidth after weights are loaded
- Fuse S4’s three sub-steps—scalar-vector multiplication, recurrent update, linear projection—into a single pipelined pass through the SA
- Execute standard GEMM (WS/IS/OS) for DNN layers
- Leave all inter-PE wiring fixed, with all functional reconfiguration realized in PE microarchitecture

Typical SSM kernel scheduling in ProDF involves a preload phase for weights and control bits, followed by a compute phase where input sequences of length $T$ are processed in a pipelined series of SA rows, with each row specialized for a sub-step or passthrough operation. After the pipeline is filled, outputs are collected every cycle.

This pipeline is illustrated by the dataflow:

```
Input → [Row 0: BWS] → [Row 1: FRI/TRI] → [Row 2…H: TOS/BWS] → Output
```
The approach supports in-situ, streaming computation of SSMs, precluding the need for intermediate SRAM spills [2507.21394].

## 4. Performance, Energy, and Area Trade-offs

EpochCore demonstrates substantial accelerative gains over prior architectures:

- Peak performance for a 64×64 SA at 700 MHz: ~2.9 TOPS (32-bit)
- S4 token throughput approaches $T/(T+N+1) \times 700$M tokens/sec after warmup
- Latency:
    - Up to 250× faster than 2D sparse SAs
    - 25× faster than 1D FFT-SAs
    - ~2,000× faster than Nvidia A100 GPU kernels on LRA benchmarks, e.g., CIFAR-10 at 64K sequence length
- Energy efficiency:
    - 45× lower energy per inference vs. 2D sparse SAs
    - 10× lower vs. FFT-SAs
    - LIMA-PE dynamic power: 11.5 mW (compute), 6.7 mW (pass-through), 3.8 mW (sleep)
- Area:
    - LIMA-PE is 1.4–2.0× the size of a conventional PE
    - Overall systolic array has around 2× the area of an equivalently sized TPU-style SA [2507.21394]

## 5. Implementation and Scalability Aspects

The reference implementation is synthesized in Cadence Genus using FreePDK45 (45 nm), clocked at 700 MHz. The PE critical path incurs approximately a 5% slowdown due to clock gating. On-chip SRAM provides low-latency access (1.4 ns as per CACTI), and after initial weight preload there are no further off-chip bandwidth requirements during layer execution. Off-chip DRAM is accessed only between layers.

Scalability derives from the state-map size $N$ and hidden size $H$: SA dimensions grow as $(N+2) \times (N+1)$ to accommodate larger models, and multiple tiles can be deployed for batch or sequence partitioning. Typical utilization for practical $N$ and $H$ values remains above 80% [2507.21394].

## 6. Contribution, Limitations, and Prospects

EpochCore constitutes the first unified systolic array accelerator with:

- Native support for structured SSMs (S4, Liquid-S4) alongside DNN, CNN, and Transformer layers
- The LIMA-PE microarchitecture, offering four dynamically programmable MAC modes with real/complex support
- ProDF dataflow, enabling fully in-situ streaming SSM computation in a single SA pass
- Empirically demonstrated 250–2,000× latency reduction and 10–45× energy saving vs. prior SA and GPU baselines

Identified limitations include:

- Mamba-style SSMs, which require input-dependent recomputation of $\overline{A},\overline{B}$, with coefficient adaptation currently performed on the host; on-PE update units are a possible extension
- Support for gated SSMs would require new MAC modes implementing element-wise gating and variable time-steps
- Multi-dimensional SSMs (processing 2D input) would benefit from further reconfigurable inter-PE connections beyond the current NE→SW diagonal support

EpochCore provides a microarchitectural foundation for efficient hardware acceleration of long-range sequence models and compatibility with standard deep learning workloads, offering new avenues for SSM integration and hardware–algorithm co-design [2507.21394].

Source: https://www.emergentmind.com/topics/epochcore