---
title: Processing-In-Memory (PIM) Overview
url: https://www.emergentmind.com/topics/processing-in-memory-pim-f50eb929-ab7b-4baa-8c2d-1fecc2dcbec0
type: topic
---

# Processing-In-Memory (PIM) Overview

Processing-In-Memory (PIM) is a computer architecture paradigm that integrates computational capabilities directly within or near memory arrays to minimize data movement between processing units and memory. PIM architectures target the fundamental inefficiency—termed the “memory wall”—of von Neumann systems, where the energy and latency of transporting data via narrow interfaces dominates many workloads, especially those that are memory-bound and data-intensive. By embedding computation at or within the locus of data storage, PIM fundamentally alters the scaling relationships of throughput, energy efficiency, and system performance, making it central to current research in systems architecture, circuit design, and applications across deep learning, database analytics, and scientific computing [2012.03112][1907.12947][2505.18954].

## 1. Motivations and Architectural Principles

Data movement costs dominate modern systems. Empirical analyses demonstrate that transferring a 64 B cache line from DRAM to a CPU may consume several hundreds of picojoules, while arithmetic on that data in the CPU requires orders of magnitude less energy [2205.14647][2012.03112]. In high-throughput and data-centric domains (e.g., machine learning, graph analytics, data warehousing), the overhead of shuttling data between physically separated compute and memory subsystems imposes prohibitive energy and latency penalties, constituting up to 80–90% of system energy [1907.12947].

PIM architectures challenge this separation by provisioning compute logic modules—
1. **Processing-Near-Memory (PNM):** Co-locates programmable logic (in-order cores or accelerators) with DRAM via 3D stacking or package-level integration. Example platforms include logic layers in 3D-stacked DRAM (HBM, HMC) and commercial systems such as UPMEM DIMMs [2012.03112][2205.14647][1903.03988].
2. **Processing-Using-Memory (PuM):** Performs computation by exploiting the analog/electrical properties of the memory array itself, executing logic at the cell or subarray level using specialized timing or sense amplifier tricks (e.g., RowClone, Ambit, SIMDRAM) [2012.03112][2205.14647][1905.04376].

The computational model of PIM replaces traditional movement of data-to-compute with the principle “compute-where-the-data-lives,” essentially transforming memory into an active substrate.

## 2. Canonical PIM Mechanisms and Substrates

A deep taxonomy of PIM designs includes:

| PIM Class                        | Mechanism                      | Example Substrates     |
|-----------------------------------|-------------------------------|-----------------------|
| Processing-Near-Memory (PNM)      | Logic layer in 3D DRAM         | HMC/HBM, UPMEM        |
| Processing-Using-Memory (PuM)     | DRAM cell/sense amp analog     | Ambit, RowClone       |
| Bit-serial SRAM/Nonvolatile PIM   | Compute-enabled SRAM/RRAM      | PIMSAB, DB-PIM        |

**PNM** achieves high effective bandwidth by leveraging wide internal interfaces (TSVs) and integrating programmable cores or fixed-function accelerators[2012.03112][2205.14647]. For instance, Tesseract provides graph-processing engines per vault, and UPMEM exposes thousands of DPUs (Data Processing Units) per system, with local memory and simple RISC-like cores[1903.03988][2602.09174].

**PuM** exploits physical effects. RowClone performs in-DRAM page copying using back-to-back activations; Ambit achieves AND/OR/NOT via triple-row activations and dual-contact sense amplifiers, implementing Bitwise Majority logic at the subarray level. Bulk-bitwise operations are conducted in parallel across thousands of bits, delivering ultrahigh throughput and marked energy efficiency improvements (e.g., 44× speedup and 35× energy reduction over CPU for Ambit)[1905.04376][2012.03112][2203.10486].

Bit-serial SRAM-PIM (e.g., PIMSAB, DB-PIM) leverages crossbar architectures and custom in-SRAM digital logic[2505.18954][2311.11384], exploiting algorithm-architecture co-design for further efficiency enhancements, notably through aggressive sparsity management.

## 3. Algorithm-Architecture Co-Design and Workload Mapping

Achieving PIM’s potential involves significant co-design at both hardware and software layers. The Dyadic Block PIM (DB-PIM) approach exemplifies this trend by enforcing algorithm-driven value-level and bit-level sparsity, tightly aligning data pruning, bit encoding (Canonical Signed Digit), and custom digital SRAM macros (e.g., IPU, DBMU, CSD adder tree). Hybrid pruning combines:
- Value-level mask $M^{val}$ achieving block-wise skip for coarse pruning.
- Bit-level mask $M^{bit}$ targeting sparsity in encoded weights, exploiting the structured nature of real applications where zeros cluster at the value and bit levels [2505.18954].

Macros like the Input Pre-processing Unit dynamically skip all-zero input-columns, while the hierarchy of Dyadic Block Multiply/Adder units maps precisely onto the compressed sparsity structure, yielding up to 8.01× speedup and 85.28% energy savings for dense neural network workloads.

For workloads with regular, high data-parallelism (linear algebra, convolution, reduction), mapping onto bank-level parallel PIM—typified by UPMEM, HMC/HBM, or even crossbar RRAM—enables near-optimal exploitation of on-die memory bandwidth[2602.09174][2311.11384].

Database operations leverage bulk-bitwise crossbar PIMs for in-place filter, GROUP-BY, and aggregation (e.g., tree-of-bitwise adders), achieving full OLAP workloads inside the memory module[2302.01675][2203.10486]. Data is often laid out in bit-sliced form to enable maximal bit-parallel exploitation and mapped physically to minimize transfer and maximize locality.

In ML and transactional workloads, UPMEM-like architectures enable the offloading of both compute-intensive and memory-bound phases (e.g., parallel SGD, transactional memory) by decomposing the workload into fine-grained partitions assigned to large numbers of lightweight near-memory cores[2404.07164][2401.09281].

## 4. System-Level Software, Programming Models, and Toolchains

Adopting PIM requires a rethinking of both programming interfaces and system software:
- **Programming Models:** Support for familiar abstractions (e.g., iterator-based APIs, transactional memory, offloading directives), as well as PIM-enabled instructions (PEIs) for fine-grained, semantics-preserving offload, is essential for deployment [1907.12947][2205.14647][2310.01893][2401.09281].
- **Compiler and Runtime:** Automated partitioning of code into PIM and host-resident segments leverages static analysis of arithmetic intensity, parallelism, and data-locality[2402.18592]. Tools like A³PIM statically analyze memory access patterns, port pressure, and communication costs, statically mapping code to PIM vs. CPU while mitigating cross-segment data movement and synchronization penalties. This yields mean speedups of 2.63× vs. CPU-only and 4.45× vs. PIM-only[2402.18592].
- **OS and Memory Management:** Support for PIM-aware virtual memory (IMPICA), data placement, and page migration is required to maximize PIM utility and adapt to mm-scale hardware budgets and mapping constraints [2205.14647][2409.06204].
- **Data Transfers:** Efficient management of host⇔PIM transfers is critical in commercial systems where address space partitioning and mapping (e.g., via PIM-MMU’s hardware copy engine and scheduling[2409.06204]) mitigate bottlenecks arising from conventional memory controller limitations.

Benchmarks and simulation frameworks at device, circuit, architectural, and full-system levels (e.g., NVSim, PIMSim, gem5, PIMeval+PIMbench) underpin experimental evaluation and design exploration[2512.00096].

## 5. Quantitative Impact and Representative Workloads

A broad spectrum of application domains demonstrates PIM’s quantitative impact:
- **Deep Learning Inference:** DB-PIM achieves speedups up to 8× and energy reductions >85%, exploiting value/bit-level sparsity for CNN layers while retaining full integer/floating-point precision[2505.18954].
- **Database Analytics:** Bitwise crossbar PIMs accelerate full OLAP workloads (filter, GROUP-BY, JOIN), outperforming MonetDB by 4–8× in speed and reducing energy by 4–18× using memristive or RRAM arrays [2302.01675][2203.10486].
- **Graph Analytics:** In real PIM systems (e.g., UPMEM), linear algebraic algorithms (SpMV, SpMSpV) yield 10–50× kernel-level speedup, up to 2–4× end-to-end speedup vs. best CPU baselines, and balanced energy usage, although utilization is defined by memory partitioning and communication constraints[2602.09174].
- **ML Training:** Distributed SGD and ADMM on PIM can achieve parity or surpass high-end CPUs/GPUs in memory-bound regimes, especially for quantized or fixed-point kernels, with 3× energy gains and ~8× strong scaling up to several thousand DPUs[2404.07164].
- **Transactional and Data-Structure Operations:** PIM-STM and PIM-tree support high-throughput transactional memory and index operations, scaling to batches of millions of queries with 10–70× throughput gains under skewed access[2401.09281][2211.10516].

## 6. Cross-Layer Challenges and Open Research Directions

PIM adoption confronts significant cross-layer challenges:
- **Reliability and Manufacturing:** Integration of logic with DRAM or emerging memories (PCM, RRAM, MRAM, QCA) increases variability sensitivity and process challenges[1602.02249]. Some specialized technologies (QCA, for example) demonstrate compelling theoretical properties (e.g., 0.33 meV/operation, nanosecond delays), but require cryogenic operation and high-fidelity fabrication.
- **Thermal and Power Management:** 3D-stacked PIM cores are subject to more severe thermal constraints due to confined die stacking; power budgets for logic are tightly limited[1905.04376].
- **Programming Models:** The split between composability and low abstraction (favoring adoption) vs. expressiveness and optimization (exposing PIM details) is unsolved; DSLs, autocompilation, and PIM libraries (e.g., SimplePIM) are under active development[2310.01893].
- **Coherence and Synchronization:** Scalable, low-overhead cache coherence for mixed CPU+PIM execution is required; mechanisms including region-based invalidation, signature-based conflict detection (CoNDA, LazyPIM), and decoupled consistency are explored[1907.12947][2205.14647].
- **Inter-PIM Communication:** UPMEM-style DPUs lack direct interconnect, limiting decentralized, collective computation (e.g., all-reduce); hardware networks among near-memory compute units would allow more scalable distributed algorithms[2602.09174][2404.07164].
- **Simulation, Benchmarking, and Standardization:** Instruments for reproducible, validated evaluation across abstraction levels (device ⇄ algorithm) and standardized APIs/ISAs are an active area[2512.00096].

Future research must tightly integrate algorithm–architecture co-design, develop PIM-aware compilers and runtimes, and address manufacturability and reliability for emerging memory technologies. The overall direction portends a paradigm shift away from processor-centric to fully memory-centric system design.

## 7. Broader Impact and Outlook

PIM architectures are positioned to deliver:
- Orders-of-magnitude reductions in data-movement energy (often 10–100×).
- 2–10× speedups for memory-bound and bandwidth-constrained workloads.
- Architectural and software co-optimization as a foundational design principle.

Adoption will depend on robust support for programming, simulation, and system integration, coupled with continued progress in memory logic integration and cross-layer reliability.

PIM’s evolution marks a shift to data-centric computing, where storage arrays become active computation substrates and system organization is reimagined for memory-side intelligence[2012.03112][2205.14647][1905.04376][2505.18954].

Source: https://www.emergentmind.com/topics/processing-in-memory-pim-f50eb929-ab7b-4baa-8c2d-1fecc2dcbec0