UPMEM Processing-in-Memory System

Updated 29 October 2025

UPMEM Processing-in-Memory is a general-purpose architecture that embeds DRAM Processing Units within standard memory modules for massively parallel, near-data computation.
The system features dedicated MRAM, WRAM, and IRAM per DPU, enabling efficient offloading of memory-bound tasks through explicit DMA and SPMD programming.
High-level frameworks like DaPPA and SimplePIM abstract memory management complexities, achieving significant speedups, energy efficiency, and scalability for diverse workloads.

The UPMEM Processing-in-Memory (PiM) system is a commercially available general-purpose architecture that integrates general-purpose processor cores—termed DRAM Processing Units (DPUs)—within DRAM modules, enabling programmable, massively parallel, near-data computation. Its primary aim is to mitigate the data movement bottleneck inherent to processor-centric systems by performing compute-intensive, memory-bound workloads directly where data resides, thus enhancing both throughput and energy efficiency for a diverse set of applications, including databases, analytics, graph algorithms, and neural network inference.

1. Architecture and System Design

UPMEM PiM modules are deployed in standard DDR4-2400 DIMMs, with each DIMM comprising multiple PIM-enabled DRAM chips, each chip integrating 8 DPUs. Each DPU is a 32-bit in-order RISC core, operating at up to 350–500 MHz, with 24 hardware threads (tasklets). Each DPU has dedicated memory regions: - MRAM: 64 MB DRAM bank per DPU (bulk data storage) - WRAM: 64 KB SRAM scratchpad per DPU (fast local workspace) - IRAM: 24 KB instruction RAM

At scale, UPMEM platforms can reach up to 2,560 DPUs per server, providing aggregate DRAM bandwidth >1 TB/s. The pipeline is 14-stage without interlocks; revolver scheduling and even/odd register file partitioning introduce performance-hazard constraints.

The programming model is SPMD: the host CPU orchestrates offload of data and code to DPUs, initiates computation, and collects results. Inter-DPU communication is not supported on hardware; all global coordination must involve the host CPU. DPUs interface with MRAM via explicit DMA operations managed by the programmer.

2. Programming Model and Software Ecosystem

Programming the UPMEM PiM system involves significant explicit management of data movement, memory allocation, and workload partitioning. The UPMEM SDK exposes APIs for allocating memory, launching DPU programs, orchestrating DMA transfers between host and DPUs, and managing synchronization primitives at the DPU level (barrier, handshake, semaphore).

Several high-level programming frameworks have been developed to abstract these complexities:

DaPPA: Provides high-level data-parallel primitives (map, filter, reduce, window, group), a pipeline-based dataflow programming interface, and template-based dynamic compilation, enabling users to express parallel computations declaratively. DaPPA significantly reduces programming effort (94.4% LoC reduction) and improves performance (2.1× average speedup over hand-tuned code) by automating data movement, memory management, and workload distribution (Oliveira et al., 2023).
SimplePIM: A C-based framework exposing parallel patterns (map, reduce, zip) as host-callable iterators, along with abstracted primitives for host-DPU and DPU-DPU communication. SimplePIM enables significant lines-of-code reductions (up to 5.93×) and often matches or outperforms hand-optimized kernels by automating best-practice optimizations and transfer management (Chen et al., 2023).

Programming at the device level still requires careful partitioning to fit application data within constrained per-DPU resources (WRAM, IRAM) and efficient alignment for MRAM transfers.

3. Performance Characterization and Application Suitability

The PrIM benchmark suite, tailored for UPMEM, comprises 16 diverse, memory-bound workloads spanning linear/sparse algebra, analytics, databases, graph algorithms, and neural networks. Performance studies consistently show:

Compute vs. Memory Bound: On UPMEM, most workloads are compute-bound, as simple integer add/sub are natively supported, while multiplication, division, and floating-point are emulated in software (10–100× slower). The in-core bandwidth rarely saturates for operational intensities typical of real workloads (Gómez-Luna et al., 2021, Gómez-Luna et al., 2021).
Suitable Workloads: Highest acceleration is observed for memory-bound, embarrassingly parallel tasks using simple arithmetic—vector add, reductions, scans, database predicates, histograms. Workloads requiring direct inter-core (global) synchronization or complex arithmetic are less suited.
Scalability: Linear weak scaling is achievable for workloads fitting the above criteria. Benchmarks with minimal inter-DPU synchronization scale well to >2,000 DPUs. For communication-intensive tasks, scaling is bottlenecked by host-mediated coordination.
Benchmark Results: On 2,556-DPU systems, speedups of up to 93× vs. CPUs and 2.5× vs. GPUs for optimal workloads have been demonstrated; energy efficiency improvements reach up to 5× for the most suitable tasks.

Workload Class	Observed Performance	Best Platform
Simple, streaming, add/sub	20–93× > CPU, 2–3× > GPU	UPMEM PiM
Communication-heavy, complex	Slower than CPU/GPU, bottlenecked	CPU/GPU
Low-precision, INT4/8 dot prod	Up to 10× > CPU (with optimizations)	UPMEM (optimized)

4. System Bottlenecks and Hardware/Software Bottleneck Mitigation

Identified bottlenecks in UPMEM's PiM include:

Compute throughput: Deep pipeline and lack of native support for multiply/div/fp ops limit DPU-side arithmetic rates.
Memory management: Manual scratchpad (WRAM) management and explicit DMA lead to complex code, and inefficient usage can severely limit throughput.
Host-DPU communication: Expensive and bandwidth-limited; scaling is limited by host CPU orchestration, NUMA effects, and inability to overlap communication and compute.
No hardware cache/virtual memory: All transfers must be managed at the application level; no transparent caching.

Optimization strategies:

Algorithmic restructuring to maximize local work, minimize global synchronization, and match DPU capabilities (bit-serial, low-precision ops; blocking for matrix-mul; vertex coloring for graph).
Software optimizations: Direct assembly modification (e.g., for INT8 native multiply), batched data movement, loop unrolling, and NUMA-aware DPU allocation can yield 1.6–5.9× speedups and improve host transfer throughput 2.9×, reducing jitter (Chmielewski et al., 3 Oct 2025).
Framework automation: High-level programmability via DaPPA/SimplePIM achieves competitive or superior performance while hiding hardware details.

5. Advanced Algorithms and Application Case Studies

The UPMEM system has been validated via end-to-end workloads:

ML training and inference: PiM matches or outperforms CPUs for dense, memory-bound mini-batch SGD, ADMM, and MLP inference (up to 259× for large batch MLPs) under suitable data types and batch sizes (Carrinho et al., 10 Aug 2025, Rhyner et al., 10 Apr 2024). However, no inter-DPU communication support remains a limitation for decentralized/distributed ML algorithms.
Triangle counting in graphs: Vertex coloring eliminates cross-core communication; reservoir sampling mitigates memory limits; achieves up to 80× speedup over CPUs for COO-formatted dynamic graphs with minimal error (Asquini et al., 7 May 2025).
Database analytics: Implementation of aggregation, selection, join, ordering (sort/hash join, radix), matches or surpasses CPUs and sometimes GPUs in full TPC-H queries. PIMDAL achieves 3.9× average speedup over CPUs (Frouzakis et al., 2 Apr 2025).
Transactional memory: PIM-STM exposes several STM algorithm variants; metadata placement in fast WRAM yields up to 5× speedup for transaction-heavy workloads (Lopes et al., 17 Jan 2024).
Indexing: PIM-tree, a distributed index structure, provides skew-resistance and balanced high throughput, up to 69.7× over previous techniques, exploiting both CPU and PIM roles (Kang et al., 2022).

6. Limitations, Simulation, and Recommendations for Future PiM

Current UPMEM hardware, despite its generality and programmability, is restricted by its "wimpy" core design, lack of MMU/virtual memory, explicit scratchpad-only model, absence of inter-DPU links, and minimal hardware acceleration for multiplication, floating-point, and communication.

Simulation studies using the open-source uPIMulator framework reveal that introduction of SIMT/vector or ILP microarchitectures, hardware-managed caches, and MMU support can yield 4.6–6.2× speedups and enable secure, multi-tenant deployment with negligible overhead (Hyun et al., 2023). Hardware/software codesign, cache-based memory hierarchies, and enhanced DPU interconnection are indicated as key for broadening applicability and achieving performance/programmability parity with contemporary CPUs/GPUs.

7. Summary Table: UPMEM PiM System Overview

Attribute	Specification / Observation
DPU count per server	Up to 2,560 DPUs (32b, 14-stage, 24 threads) per 20 DIMMs
MRAM per DPU	64 MB
WRAM per DPU	64 KB
Compute support	Native: int add/sub; Slow: mul/div/fp
Host-DPU communication	Explicit, bandwidth-limited, NUMA-sensitive
Programming model	SPMD, manual management, frameworks available (DaPPA, SimplePIM)
Bottlenecks	Compute throughput, transfer management, lack of hardware comm
Suitability	Memory-bound, simple arithmetic, minimal global sync workloads
Scalability	Linear (for suitable workloads), limited for comm/comp. workloads
Performance (best case)	Up to 93× > CPU; 2.5× > GPU (select workloads)

UPMEM demonstrates the practical viability and research value of real, general-purpose PiM solutions, highlighting hardware/software co-design and expressive programming frameworks as crucial for unleashing processing-in-memory’s full potential across broad application domains (Gómez-Luna et al., 2021, Oliveira et al., 2023, Chen et al., 2023, Frouzakis et al., 2 Apr 2025, Kang et al., 2022, Carrinho et al., 10 Aug 2025, Hyun et al., 2023).