64-RV-Core Shared L1 Memory Cluster

• The 64-RV-core Shared L1 Memory Cluster is a parallel architecture that integrates 64 RISC-V cores with a multi-banked, low-latency L1 scratchpad for enhanced compute density.
• Hierarchical grouping and adaptive interconnects ensure efficient synchronization, low access latency, and scalable bandwidth for demanding applications.
• Performance metrics show high energy efficiency and throughput, making it suitable for AI, DSP, HPC, and other latency-sensitive workloads.

A 64-RV-core Shared-L1-memory Cluster is a parallel computer architecture in which 64 RISC-V processing elements (PEs)—typically RV32I or RV64I cores—are tightly coupled to a physically shared, multi-banked, low-latency Level-1 (L1) memory. This architectural pattern, exemplified in both FPGA and ASIC platforms, is designed to offer high compute density, high memory bandwidth, scalable synchronization, and efficient inter-core communication. Key research work in this area includes the GRVI Phalanx FPGA accelerator (Gray, 2016), shared-memory parallel LP solvers (Coutinho et al., 2018), hardware-accelerated synchronization (Glaser et al., 2020), and hierarchical scratchpad-based clusters such as MemPool and its specialized derivatives (Cavalcante et al., 2020, Riedel et al., 2023, Mazzola et al., 21 Mar 2025).

1. Cluster Composition and Shared L1 Memory Organization

A 64-RV-core Shared-L1-memory Cluster is typically organized hierarchically. Cores are grouped (e.g., in tiles of 4 PEs), and tiles are further aggregated into local groups, all attached to a globally addressable L1 scratchpad memory (SPM) implemented as a set of banks. Each core or tile is connected to one or more banks via a low-latency interconnect. For instance, in MemPool-type systems, each tile includes 16 SPM banks and 4 cores, providing single-cycle local access and a global memory view accessible across the cluster with average latencies of 3–5 cycles (Cavalcante et al., 2020, Riedel et al., 2023, Mazzola et al., 21 Mar 2025). In the GRVI Phalanx architecture, shared L1 memory (CRAM) is multi-ported and banked, e.g., an 8-core cluster uses 12 ports; for 64-core clusters, the number of banks and ports is increased to allow for parallel access and scaling (Gray, 2016).

Memory access bandwidth is given by:

$$B_\text{cluster} = N_\text{ports} \times \text{data width} \times f$$

Memory banks are generally fully accessible, but design decisions regarding port count and bank interleaving strongly affect access conflict rates and scalability. Hierarchical organizations (tiles and groups) limit the maximum interconnect diameter, preserve locality, and keep wiring congestion in check, permitting clusters to scale to and beyond 64 cores (Cavalcante et al., 2020, Mazzola et al., 21 Mar 2025).

2. Interconnects and Scalability Strategies

Efficient access to shared L1 memory becomes increasingly challenging with high core counts due to wiring congestion, access conflicts, and increasing load. Hierarchical and physical-aware interconnects—such as multi-stage crossbars, radix-4 butterfly networks, and directional group-level crossbars—are used to maintain low round-trip latency and high throughput.

• Hierarchical Topologies: In MemPool (Cavalcante et al., 2020, Riedel et al., 2023), cores are organized in local groups (e.g., 16–32 cores) with fast intra-group crossbars and hierarchical inter-group links, providing ≤5 cycles access latency under heavy load.
• Hybrid Addressing: Data locality is improved by “scrambling” or assigning certain memory regions to reside in local banks, minimizing remote accesses and related energy/latency (Cavalcante et al., 2020).
• Burst Access Support: For vector workloads, TCDM Burst Access aggregates many narrow requests into wide bursts dispatched to banks in parallel, enabling bandwidth utilization of up to 80% of the memory’s peak (Shen et al., 24 Jan 2025).
• Bank Partitioning: In large clusters, dynamic allocation schemes re-partition bank mappings at runtime to maximize data locality, avoid contention, and address NUMA effects (Wang et al., 2 Aug 2025).

Compressed summary table:

Mechanism	Purpose	Effect in 64-core Cluster
Hierarchical Interconnect	Wiring, scalability	≤5 cycles latency, manageable congestion
Multi-banked SPM	Parallel access	High aggregate bandwidth, reduced stalls
Hybrid/Adaptive Address	Locality, contention	1–2× bandwidth improvement, lower energy
Burst Vector Access	SIMD scaling	77–226% bandwidth gain in vector kernels

3. Core Microarchitecture and Processing Efficiency

Each RV core in the cluster is typically optimized for area efficiency and energy proportionality. Choices include simple, single-stage RV32IMA “Snitch” or similar scalar cores (Cavalcante et al., 2020, Riedel et al., 2023), augmented for selected SPM instructions. Variations include:

• Vector Processing Units (Spatz): Vector accelerators (implementing e.g., RVV Zve32x or Zve64d) directly attached to each tile/core, dramatically reduce instruction fetch overhead and power, achieving 266 GOPS/W and 70% higher throughput than scalar cores in typical kernels (Cavalcante et al., 2022, Perotti et al., 2023, Mazzola et al., 21 Mar 2025).
• Complex DSP/AI Units: For communications and AI (e.g., HeartStream), the instruction set includes fused multiply–accumulate, division/sqrt, and SIMD operations for (16/32/8-bit and complex) data types. Hardware-managed systolic execution and QLRs enable implicit, pipelined dataflow (Zhang et al., 10 Sep 2025).
• Synchronization Hardware: A hardware Synchronization and Communication Unit (SCU) per core or tile brings barriers and mutexes down to 6-cycle operation, with negligible per-barrier energy, improved power management, and area-efficient implementation (Glaser et al., 2020).

4. Systolic and Dataflow Execution Support

Shared-L1 clusters can overlay systolic execution models for regular, high-throughput kernels typical in signal processing, AI, and telecommunications. This is achieved by:

• Memory-mapped Queues: FIFO queues implemented in SPM provide inter-core pipelined data movement.
• RISC-V ISA Extensions: Xqueue and Queue-Linked Registers (QLRs) allow single-instruction or even implicit communication, drastically reducing control instruction overhead and improving utilization (doubling to 73% utilization in select DSP kernels) (Mazzola et al., 20 Feb 2024).
• Hybrid Execution Models: The system can dynamically configure the cluster for dataflow or shared-memory execution, maximizing efficiency for regular kernels while retaining full programmability for irregular workloads.

Systolic support yields up to a 1.89x improvement in energy efficiency and enables clusters to operate within tight latency/power budgets for 5G/6G and AI workloads (Zhang et al., 10 Sep 2025).

5. Programming, Synchronization, and Use Cases

64-core shared-L1-memory clusters support parallel and scalable execution models such as bulk-synchronous (OpenMP, Halide), dataflow programming, and direct hand-tuned SIMD code. Programmatic support includes:

• Multiple Runtimes: Bare-metal, OpenMP, and high-level DSLs are mapped efficiently via appropriate backends (Riedel et al., 2023).
• Synchronization Primitives: RISC-V atomics, low-overhead hardware barriers, and fine-grain sleep/wakeup, supported both in instruction set and hardware-SCU logic (Glaser et al., 2020).
• Dynamic, Adaptive Memory Allocation: Allocators can exploit hardware address remapping units (as in DAS) to maximize both cache locality and memory bandwidth for variable and data-heavy workloads such as transformers and attention models (Wang et al., 2 Aug 2025).

Typical high-efficiency applications include:

• Linear algebra (e.g., matrix multiplication, DCT),
• Signal processing (beamforming, FFT, channel estimation),
• AI inference (CNN/transformer execution),
• Telecommunications (baseband, MIMO),
• Large-scale LP (parallel Simplex (Coutinho et al., 2018)).

6. Performance Metrics, Energy, and Technology Scalability

Documented implementations report the following:

• Performance: Up to 410 GFLOP/s peak in 64-core baseband-optimized clusters (@800MHz), with 204.8 GB/s L1 bandwidth (Zhang et al., 10 Sep 2025). MemPool derivatives regularly reach ≥200 GFOPS for 64–256 cores at moderate power (Riedel et al., 2023, Mazzola et al., 21 Mar 2025).
• Energy Efficiency: Spatz-based clusters reach up to 266 GOPS/W (Cavalcante et al., 2022); HeartStream demonstrates up to 49.6 GFLOP/s/W in PUSCH baseband (Zhang et al., 10 Sep 2025).
• Area Overhead: Hardware extensions for synchronization, systolic queues, or burst vector access typically incur <8% area cost (Glaser et al., 2020, Shen et al., 24 Jan 2025, Mazzola et al., 20 Feb 2024).
• 3D Integration: Moving to 3D stacked die layouts (logic/memory partitioning) reduces wirelength, alleviates routing congestion, and increases SPM capacity; boosts performance by 9% and energy efficiency by up to 18% at identical footprint compared to planar designs (Cavalcante et al., 2021).
• Cache Coherence: Directory-based MOESIF (e.g., BP-BedRock (Wyse et al., 2022)) and programmable coherence engines allow extension to few-dozen-core coherent clusters with low area penalty.

7. Architectural Trade-offs, Specialization, and Future Directions

The 64-RV-core shared-L1-memory paradigm offers a scalable balance between energy efficiency, high bandwidth, and reconfigurability. Trade-offs and future research axes include:

• General-Purpose vs. Specialized (Vector/Systolic): Baseline MemPool supports broad programmability, while Systolic and Vectorial MemPool flavors achieve up to 7% and >33% higher utilization respectively for regular/well-vectorized kernels at modest area cost (Mazzola et al., 21 Mar 2025).
• Contention and NUMA Effects: At high core counts, bank contention and variable bank access latency (NUMA) can erode performance. Adaptive, hardware-accelerated address mapping (e.g., DAS) provides localized mapping, boosting core utilization and throughput for memory-bound workloads (Wang et al., 2 Aug 2025).
• Parallel Algorithm Limits: Linear programming and dense computation workloads show nearly ideal scaling up to cache/memory resource limits, with speedups up to 19× over sequential for LP Simplex (Coutinho et al., 2018), and 226% bandwidth improvement in large vector clusters using burst access (Shen et al., 24 Jan 2025).
• Application Flexibility: The architecture serves a spectrum from scientific HPC to AI inference and telecommunications; dynamic reconfiguration between general-purpose and dataflow/AI-optimized flavors addresses evolving demands (Mazzola et al., 21 Mar 2025, Zhang et al., 10 Sep 2025).

Summary Table: Key Performance Metrics Compared

System / Flavor	Cores	L1 BW (GB/s)	Peak Perf. (GFLOP/s)	Energy Eff. (GFLOP/s/W)	Notes
HeartStream	64	204.8	410	49.6 (PUSCH)	B5G/6G, systolic, complex AI+DSP
MemPool-Vectorial	64	n/a	192	266	4 Spatz VPU/tile, 94% utilization
MemPool-Baseline	64	n/a	192	128	Scalar Snitch cores, 59% utilization
MemPool-Systolic	64	n/a	~206	n/a	+7% over Baseline, +5% area

A plausible implication is that further enhancements—such as 3D stacking, burst vector accesses, adaptive memory remapping, and lightweight hardware-managed queues—are likely to remain central to the continued scaling, performance, and programmability of 64-core (and larger) shared-L1-memory clusters. As generalized building blocks in diverse domains, such clusters represent a matured and versatile architectural paradigm for massively parallel, energy-constrained, and latency-sensitive processing.