Processing-In-Memory (PIM) Architectures

• Processing-In-Memory (PIM) architectures are systems where computation occurs in or near memory, dramatically reducing data movement and energy costs.
• They leverage modern memory technologies like SRAM, DRAM, and emerging nonvolatile memory to exploit high internal parallelism and bandwidth for intensive workloads.
• Innovations in bit-serial processing, adaptive precision, and compiler co-design enable significant gains in compute density and overall system efficiency.

Processing-In-Memory (PIM) architectures designate systems wherein computation is performed directly in—or extremely close to—memory arrays, fundamentally shifting away from the classical von Neumann model by minimizing energy and bandwidth expended on data movement. This approach leverages the massive internal parallelism and aggregate bandwidth of modern memory technologies (e.g., SRAM, @@@@2@@@@, emerging nonvolatile memory), offering substantial acceleration for workloads characterized by high data-movement intensity. PIM designs are increasingly adopted for domains such as deep learning, high-throughput data analytics, and scientific computing, as they enable orders-of-magnitude savings in data transfer while achieving scalable in situ processing.

1. Fundamental Architectural Principles and Physical Organization

State-of-the-art PIM architectures are physically structured around large arrays of compute-enabled memory blocks embedded within hierarchical or mesh interconnect fabrics. For instance, the PIMSAB system comprises a two-dimensional mesh of tiles, each containing a 16×16 grid of dual-ported bit-serial SRAM blocks (CRAMs), where each CRAM is a 256×256 SRAM supporting per-bit ALU functionality and carry/mask latches (Arora et al., 2023). Within a tile, CRAMs are operated in lock-step under a SIMD controller, enabling 4096 processing elements per tile. Interconnect is realized via a static H-tree for intra-tile reductions and a mesh NoC (e.g., 12×10 topology, 1024 bits/cycle) for inter-tile communication; the top-row tiles are interfaced directly to high-bandwidth DRAM controllers. Area allocation in such designs is heavily skewed toward compute arrays (e.g., 72% for CRAMs in PIMSAB).

Computation is performed bit-serially inside memory, and hierarchical reduction/interconnect networks (static H-tree, circular shift rings, wormhole-routed mesh NoC) support efficient aggregation and broadcasting of intermediate results. Explicit shuffle hardware—peripheral crossbars indexed by programmable stride fields—allow operand folding, replication, or pattern broadcasting with zero software overhead.

2. Spatially-Aware Data Communication Networks

Highly-parallel PIM systems require efficient data movement mechanisms at both local (within tile/bank/subarray) and global (across tiles/chips/ranks) scales to prevent communication bottlenecks that offset compute speedups. PIMSAB implements a hierarchical reduction by wiring CRAM outputs into a multi-level H-tree, where a k-level tree reduces a 256-element vector in k cycles, latency given by $T_{\mathrm{reduce\_cram}(N) = k (ℓ_{\mathrm{HT}} + t_{\mathrm{switch}})$, $k = \lceil \log_2 N \rceil$. Systolic broadcasting is used for propagating constants and weights along rows in a wavefront pattern.

Off-chip bandwidth constraints ($BW_{\mathrm{DRAM}}$) and on-chip bandwidth ($BW_{\mathrm{NoC}}$) are modeled analytically for total data movement costs: $$T_{\mathrm{DRAM}} = \frac{S}{BW_{\mathrm{DRAM}}}, \quad T_{\mathrm{NoC}} = \frac{D_{\mathrm{tiles}}}{BW_{\mathrm{NoC}}}$$ where $S$ is tensor size and $D_{\mathrm{tiles}}$ is the inter-tile data volume. Hardware shuffle logic and explicit multicast paths facilitate aligned and pattern-based movement, reducing redundant operations (Arora et al., 2023).

3. Bit-Serial Computation Techniques and Adaptive Precision

Bit-serial processing permits high compute density at low area cost, but traditionally suffers from poor utilization for wide operands or constant multipliers. PIMSAB introduces adaptive precision and bit-slicing (instruction-level programmable bit-width $p$), allowing the compiler to partition multiplications and accumulations into parallel sub-operations mapped to different portions of the CRAM hierarchy. The computation time for $p \times q$ multiplication is: $C_{\mathrm{mul}}(p,q) = p\cdot q + 3\max(p,q) - 2$ This facilitates simultaneous computation on bit-sliced segments—a crucial optimization for workloads with variable or sparse precision requirements.

Multiplication with sparse constants leverages operand sparsity for cycle reduction: $$C_{\mathrm{const\_mul}}(p,k) = k(p + 3), \quad (k \leq p)$$ Peak throughput for $p$-bit operations is modeled as: $$\Omega(p) = \frac{\mathrm{CRAM}_{\mathrm{count}} \times p}{A_{\mathrm{CRAM}}(p)} \times f$$ For $p=8$, PIMSAB achieves compute densities exceeding 300 GOPS/mm²—comparable to NVIDIA A100 INT8 throughput.

4. Compiler Co-Design for Maximizing Utilization

PIM architectures demand sophisticated compiler frameworks to partition data-parallel workloads efficiently across hierarchical memory structures and exploit bit-level parallelism. PIMSAB extends the TVM tensor DSL, allowing explicit loop/data layout primitives (split, reorder, bind_to_tile). The compiler co-optimizes tiling, reduction scheduling (aligning partial sums with H-tree levels), bitline lifetimes, buffer packing, and shuffle patterns. Calls such as tile_bcast, cram_bcast, and tile_tx/rx are inserted with static shuffle configurations, exhaustively searching buffer/tile allocation spaces to maximize utilization.

The lowering of kernel IR into PIMSAB ISA replaces large numbers of DRAM loads and computation loops with singular memory-local broadcasts, substantially reducing total data movement and improving performance (Arora et al., 2023).

5. Quantitative Evaluation: Performance, Energy, and Comparative Analysis

Benchmark analysis shows PIMSAB consistently delivers geometric mean speedups of 3× over NVIDIA A100 GPUs (iso-area, iso-bandwidth), 3.70× over Duality Cache SRAM PIM, and 3.88× over SIMDRAM DRAM PIM across representative deep learning kernels (vecadd, gemm, conv2d, ResNet18). Energy usage is reduced by 4.2× relative to A100. Table below (extracted from (Arora et al., 2023)) provides normalized speedups:

Workload	vs A100	vs DC	vs SIMDRAM
vecadd	1.20×	2.10×	2.45×
gemm	3.08×	3.85×	4.02×
conv2d	2.75×	3.62×	3.98×
ResNet-18	3.30×	3.72×	3.80×
Geomean	3.00×	3.70×	3.88×

Energy breakdown: DRAM (40%), CRAM compute (15%), NoC (8%), DRAM transpose/RF/instr ctrl (37%). Massive block-level parallelism and spatially-aware communication, together with compiler-driven bit-level optimizations, unify to deliver these gains.

6. Reliability Enhancements in PIM Architectures

Reliability and fault-tolerance are mission-critical for large-scale PIM deployments. FAT-PIM (Zubair et al., 2022) exploits the inherent current-summation property of ReRAM crossbars, storing a homomorphic checksum as an extra column per row to detect both soft and analog errors. This error-detection detects 100% of injected faults with only 4.9% performance and 3.9% storage overhead (on 128×128 arrays), maintaining massive parallelism and energy efficiency without expensive ECC or TMR. Pipeline integration is minimally invasive, leveraging existing peripheral datapath, and can be extended to SRAM, MRAM, and DRAM PIM technologies.

7. Design Trends, Open Challenges, and Future Directions

The unification of spatially-aware hierarchical compute fabric, communication-optimized hardware, and deeply integrated compiler infrastructure marks the direction for scalable, high-density PIM. Key trends include:

• Further reduction in inter-tile/inter-bank data movement latency via explicit broadcast/systolic interconnects.
• Exploitation of sparsity, variable precision, and dynamic operand characteristics through compiler co-design.
• Integration of low-overhead error detection/correction mechanisms (e.g., FAT-PIM homomorphic checksum).
• Continued architectural co-optimization for compute density, area efficiency, and energy savings.
• Extension of reliability, security, and coherence protocols for mission-critical and multi-tenant environments.

The field advances toward heterogeneous, multi-technology PIM fabrics—SRAM, DRAM, ReRAM—with unified execution and programming models, leveraging both device-level innovations and system-level orchestration (Arora et al., 2023, Zubair et al., 2022).