---
title: Processing-in-Memory (PIM) Overview
url: https://www.emergentmind.com/topics/processing-in-memory-pim
type: topic
---

# Processing-in-Memory (PIM) Overview

Processing-in-Memory (PIM) is an architectural paradigm focused on mitigating the limitations of processor-centric computing by tightly integrating computation with memory resources. This data-centric approach primarily addresses the rapidly scaling overheads of data movement between memory and processors, which are now dominant in the energy and latency profiles of data-intensive applications. PIM enables computation either "in" memory (using the physical properties of memory devices for logic) or "near" memory (embedding programmable logic close to memory arrays), thereby collapsing the von Neumann bottleneck and exposing massive parallelism and internal memory bandwidth. The paradigm spans diverse substrates—commodity DRAM, 3D-stacked memory, SRAM macros, emerging resistive memories, and even quantum-dot cellular automata—with corresponding trade-offs in programmability, performance, system complexity, and workload mapping.

## 1. Motivation and Historical Foundations

The performance, energy, and scalability bottleneck in modern systems is primarily attributed to the separation of computation (CPU, GPU, accelerator) and data storage (DRAM, NVM), enforced by the narrow, high-latency off-chip memory bus. Off-chip data movement dominates both latency (tens–hundreds of nanoseconds per DRAM access vs. a few nanoseconds for in-core operations) and energy (e.g., a 64-byte DRAM fetch can consume ~640 pJ, >100× a 64-bit add) [2012.03112]. Large-scale workloads—ML/AI, graph analytics, database engines—consistently show that memory access and data movement account for 35–62% of total system energy and overwhelm cache or on-chip compute resources when arithmetic intensity is low [2110.01709].

The central thesis of PIM is to eliminate or drastically reduce this movement by enabling computation where the data resides. Early PIM concepts appeared in the 1960s and 1970s (e.g., logic-in-memory, Active Pages), but practical realization was limited by device, cost, and system design barriers. Today’s renewed interest leverages 3D-stacked integrations, industry prototypes (UPMEM, HBM-PIM, AxDIMM, AiM), nonvolatile technologies (RRAM/PCM), and new system software [2012.03112, 1903.03988, 1907.12947].

## 2. PIM Architectural Taxonomy: Approaches and Primitives

### 2.1. Processing-Using-Memory (PUM)

PUM architectures exploit the intrinsic analog or digital behavior of memory cell arrays to perform computation directly inside storage arrays, typically with minimal logic overhead. Canonical mechanisms include:

- **DRAM analog operations (Ambit, RowClone):** RowClone enables in-DRAM bulk copy/initialize by activating multiple rows sequentially for memcopy, yielding up to 11.6× speedup and 74.4× energy reduction for 4 KB transfers. Ambit leverages triple-row activation for bitwise MAJ, AND, OR, and NOT at subarray bandwidths up to hundreds of TB/s. Energy per DRAM operation (e.g., AND/OR) is of order 0.1 pJ/bit [2012.03112, 1905.04376].
- **Bulk bitwise processing with RRAM/memristors:** Bulk-bitwise primitives (e.g., IMPLY logic) in stateful resistive arrays support high-throughput operations for database analytics and filtering, and can surpass CPU-based systems by up to 608× for TPC-H queries with up to 18.6× energy savings [2203.10486].

### 2.2. Processing-Near-Memory (PNM)

PNM architectures embed programmable logic (simple in-order cores, vector units, or custom accelerators) in the logic die of 3D-stacked memories or at the bank periphery in conventional DIMMs [2012.03112, 2206.04218]:

- **3D-stacked HMC/HBM PIM:** The logic layer, interconnected via TSVs, hosts per-bank or per-vault microcontrollers. Internal stack bandwidths reach 400–500 GB/s, and the logic-layer budget allows up to ∼200 mm² area and 5–10 W [2012.03112]. Tesseract (ISCA’15) achieves a 13.8× speedup and 87% energy savings on graph analytics [2012.03112].
- **Digital SRAM-PIM:** Augments SRAM arrays with small logic (bitwise gates, adder trees), performing parallel dot-products, as in DB-PIM. This approach achieves up to 8.01× speedup and 85.28% energy reduction compared to dense digital PIM [2505.18954].

### 2.3. Commodity and Emerging PIM Substrates

- **Commercial PIM (UPMEM):** General-purpose DPUs (post-2021) with co-located DRAM, scratchpad, and low-power in-order cores. Aggregate bandwidth reaches >2 TB/s with >2,500 DPUs per server [2110.01709, 2105.03814].
- **Novel substrates:** QCA-based PIM, as realized in the Akers logic array, demonstrates nanoscale, ultra-low-power PIM concepts [1602.02249].

## 3. Programming Models, Offloading Methodologies, and Runtime Systems

PIM requires explicit or compiler-guided workload mapping, synchronization mechanisms, and consistency primitives:

- **Granularity:** Offloading ranges from single instructions (PEI), bulk primitives (RowClone, Ambit), function-level (pragma-annotated offload, e.g., TensorFlow Lite), to application-level (standalone PIM appliances) [1907.12947].
- **Static tools (A$^3$PIM):** A static code analyzer partitions code by memory intensity, arithmetic intensity, and inter-region connectivity, achieving up to 7.14× speedup over CPU-only execution [2402.18592].
- **Software transactional memory (PIM-STM):** Rich software transactional memory abstraction tailored to PIM DPUs demonstrates up to 14.5× speedup and up to 5× energy efficiency on UPMEM hardware compared to CPU-STM [2401.09281].
- **Distributed optimization/emulation frameworks:** PrIM and SimplePIM expose distributed arrays, iterator APIs, and support for reductions, gathers, and host–DPU communication, offering up to 1.43× speedup over expert-tuned baselines with up to sixfold reduction in code complexity [2310.01893].

## 4. Systems-Level Considerations: Data Movement, Locality, Memory Management

Effective use of PIM hinges on minimizing off-chip transfers, exploiting data locality, and managing memory coherency and address spaces:

- **Host-to-PIM/DRAM–PIM data movement:** In commercial systems, explicit data copy engines (PIM-MMU) dramatically accelerate DRAM↔PIM transfers, with average 4.1× throughput and energy efficiency improvements, resulting in 2.2× end-to-end speedup on real PrIM PIM workloads [2409.06204].
- **Locality-aware data migration:** DL-PIM augments 3D-stacked PIM with dynamic per-vault migration and indirection tables, reducing per-request average latency by 54% (HMC) and increasing IPC by up to 15% for high-reuse workloads. Adaptive controller policies cap bandwidth overhead and prevent performance degradation in low-locality scenarios [2510.07719].
- **Skew resistance and load balancing:** Architectural mechanisms such as PIM-tree achieve up to 69.7× throughput over prior PIM index designs by switching between push (PIM-executed) and pull (host-executed) queries depending on detected skew, thereby maintaining balanced load under worst-case key distributions [2211.10516].
- **Memory coherence and page management:** Optimistic, signature-based coherence (CoNDA) and region-based page tables (IMPICA) are necessary to maintain correctness without incurring massive coherence traffic [1907.12947].

## 5. Empirical Performance, Energy Efficiency, and Application Domains

PIM systems, across multiple substrates, consistently demonstrate order-of-magnitude energy savings and speedup for selected data-intensive kernels:

- **Large-scale ML and DNN Inference:** PIM-DRAM achieves up to 19.5× speedup over a Titan Xp GPU for convolutional layers by integrating bitwise multiplication primitives and accumulation in commodity DRAM with negligible (sub-1%) area overhead [2105.03736]. UPMEM PIM accelerates large-scale SGD, obtaining 1.94×–10.65× speedup (and substantial energy reduction) vs. CPU/GPU on memory-bound kernels [2404.07164].
- **Database analytics:** Bulk-bitwise PIM in RRAM crossbar arrays yields 56×–608× end-to-end speedup on TPC-H queries and up to 18.6× energy reduction [2203.10486].
- **General PrIM benchmarks:** On a 2,556-DPU UPMEM system, 10/16 PrIM kernels beat a Titan V GPU by an average of 2.54× (and Xeon by 23.2×) under memory-bound, low-arithmetic-precision workloads [2105.03814, 2110.01709].
- **Graph analytics and filtering:** Logic-layer PIM (Tesseract, HMC) sustains double-digit speedups and 80%+ energy cuts for BFS, PageRank, and filtering both in commercial and academic prototypes [2012.03112, 2012.03112].
- **Programming productivity:** SimplePIM demonstrates that high-level iterator APIs with runtime-optimized data movement and collective patterns can bring PIM performance in range (or above) expert manual implementations on modern hardware [2310.01893].

## 6. Ongoing Challenges, Limitations, and Future Directions

Despite demonstrated gains, PIM adoption and effectiveness are subject to several constraints:

- **Limited compute flexibility and instruction support:** Bitwise and simple integer operations are efficient natively; floating-point and complex arithmetic often incur heavy emulation penalties [2110.01709]. SRAM-PIM advances such as DB-PIM with value-/bit-level sparsity pruning raise macro utilization, but full generality is not achieved [2505.18954].
- **Inter-node/inter-core communication:** Most PIM substrates lack true low-latency on-die interconnect for aggregation or synchronization, with the notable impact on scaling of distributed ML algorithms [2404.07164].
- **Software toolchains and programming models:** The absence of unified, portable APIs and runtime abstractions (language-level, compiler support, debug tools) remains a major barrier. Emerging programming frameworks (SimplePIM, PrIM) and toolflows (A$^3$PIM, DAMOV) partially address this, but full-system support remains underdeveloped [2310.01893, 2205.14647].
- **Coherence and virtual memory integration:** Efficient, scalable cache coherence and zero-copy data sharing between PIM and host processors remain unresolved challenges. Hardware/software co-design of PIM-aware MMUs (PIM-MMU) and relaxed coherence (CoNDA, LazyPIM) are active areas [2409.06204, 1907.12947].
- **Security and isolation:** PIM raises distinct concerns in isolation, error protection, and side-channel attack surface (e.g., RowHammer, NVM endurance) [2012.03112, 2205.14647].
- **Scaling to new domains:** Future trends include: hybrid logic+analog PIM, integration with emerging NVMs, support for direct atomic operations and collective communication primitives, and explicit hardware/software co-design for important algorithmic domains (decentralized ML, transactional memory, fairness-aware data migration) [2505.18954, 2205.14647, 2510.07719].

PIM is transitioning from a purely research concept to practical deployments, with empirical evidence for substantial improvements in system throughput and efficiency for data-centric workloads. Recognition of its limitations—both in scope and generality—fuels targeted research in hardware, programming infrastructure, operating systems, and security to fully realize the benefits of memory-centric, data-driven computing [2012.03112, 2105.03814, 2205.14647].

Source: https://www.emergentmind.com/topics/processing-in-memory-pim