---
title: 'LazyPIM: Speculative Coherence for PIM'
url: https://www.emergentmind.com/topics/lazypim
type: topic
---

# LazyPIM: Speculative Coherence for PIM

LazyPIM is a hardware cache coherence mechanism designed specifically for processing-in-memory (PIM) architectures, where computation is moved into the logic layer of 3D-stacked memory to exploit high internal bandwidth for data-intensive workloads [1706.03162]. Its central purpose is to keep the portions of a program that execute in memory coherent with the portions that continue to execute within the processor, without incurring the high off-chip traffic of traditional coherence protocols. LazyPIM combines speculative cache coherence with compressed coherence signatures, allowing a PIM kernel to execute speculatively and defer coherence checking until kernel or partial sub-kernel boundaries. The mechanism preserves the logical behavior of a fine-grained MESI protocol while reducing off-chip traffic and avoiding the serialization effects associated with coarse-grained locking and non-cacheable mappings [1706.03162].

## 1. Problem setting and motivation

Processing-in-memory systems are motivated by the processor–memory bandwidth bottleneck of modern data-intensive applications. Multithreaded pointer-intensive workloads, including graph algorithms, in-memory databases, and key-value stores, suffer from random accesses that stress off-chip bandwidth. PIM cores in the DRAM logic layer exploit TSV-backed internal bandwidth, but they must remain coherent with processor-resident threads to ensure correctness under conventional multithreading [1706.03162].

The coherence problem is particularly acute because traditional fine-grained directory protocols require a request for every cache miss. In the PIM setting, those coherence messages traverse the narrow off-chip link between the processor and the memory cube, and can therefore eliminate much of the bandwidth advantage that motivated PIM in the first place. The reported consequence is that fine-grained coherence can make PIM slower, more traffic-heavy, and more energy-hungry than CPU-only execution [1706.03162].

Alternative workarounds also degrade efficiency under heavy sharing. Fine-grained coherence (FG) saturates the off-chip link. Coarse-grained locks (CG), or flush-and-lock approaches, block or serialize processor threads that share data with PIM. Marking PIM data non-cacheable (NC) forces every processor access to go to DRAM, inflating memory traffic and energy. An analysis of graph kernels such as Ligra PageRank and Connected Components, as well as hybrid transactional/analytical in-memory databases, shows that sharing between PIM kernels and processor threads is often heavy. None of these naive coherence workarounds retain PIM’s benefits when sharing is high [1706.03162].

In this context, LazyPIM is intended to provide fine-grained synchronization semantics at low off-chip cost. This suggests that its contribution is not merely a bandwidth optimization, but a coherence design that makes general-purpose PIM compatible with conventional multithreaded execution models.

## 2. Architectural context and memory-system organization

LazyPIM targets a 3D-stacked memory organization such as HMC or HBM. The processor chip contains multi-core out-of-order cores with private L1 instruction and data caches and a shared L2 plus directory. The memory cube contains a logic layer with PIM cores, described as simple in-order cores with private L1 caches, and a small PIM coherence directory. The processor directory maintains global sharership of all cache lines and interfaces with both processor caches and PIM, while PIM cores keep a local MESI-like directory for intra-PIM coherence [1706.03162].

The architectural partition is organized around a narrow off-chip bus between the processor directory and the PIM directory. This narrow link is the resource that makes conventional coherence especially costly. Pages allocated for PIM data are annotated via a per-page flag in the page table so that both the processor side and the PIM side know which addresses belong to the PIM region [1706.03162].

LazyPIM therefore operates in a heterogeneous coherence domain. Processor-side execution remains conventional and non-speculative with respect to ordinary coherence, while PIM-side execution is allowed to proceed speculatively within bounded regions. A plausible implication is that the page-level identification of PIM data is essential to keeping the protocol scoped to addresses for which speculative deferred validation is semantically intended.

## 3. Speculative coherence protocol

LazyPIM treats each PIM kernel, or partial sub-kernel, as an atomic chunk of memory operations that takes effect only at commit time. During execution, the PIM core reads and writes data speculatively, marking each modified L1 line with a one-bit speculative flag. It appends each read address to the PIMReadSet Bloom filter and each write address to PIMWriteSet. Concurrently, the processor tracks each write to a PIM-addressed line, including writes that occur before kernel launch and those during kernel execution, in the CPUWriteSet Bloom filters, while continuing non-speculative execution on other threads [1706.03162].

Partial kernel commits are triggered when either the PIMReadSet or PIMWriteSet reaches a capacity threshold, to bound false positives, or when the instruction count of a chunk exceeds a ceiling, to bound rollback cost. At chunk completion, PIMReadSet and PIMWriteSet are sent across the off-chip link to the processor directory. The directory intersects $PIMReadSet \wedge CPUWriteSet$. If the result is nonempty, a read-after-write conflict is suspected. In that case, the directory selectively writes back only those dirty CPU cache lines whose addresses match PIMReadSet, awaits the writebacks, and then signals the PIM core to invalidate its speculative lines, roll back to the chunk’s start PC, clear signatures, and re-execute [1706.03162].

If no conflict is detected, the directory intersects $PIMWriteSet \wedge CPUWriteSet$, and any matching lines are merged or invalidated in the processor caches. The PIM’s speculative writes are then committed to DRAM atomically, with directory entries locked to enforce atomicity. All signatures are reset, and the next partial kernel begins [1706.03162].

The protocol is explicitly described as preserving sequential consistency. The speculative window and chunk-based atomicity guarantee that the global memory order remains sequentially consistent. Reads that would otherwise have seen a concurrent CPU write are caught at commit. Concurrent writes or processor reads of speculative PIM data do not cause conflicts by design; they follow WAR/WAW ordering under coarse-grained atomicity [1706.03162].

A useful clarification concerns the scope of conflict detection. LazyPIM does not attempt continuous miss-by-miss coherence enforcement during PIM execution. Instead, it defers validation and catches read-after-write conflicts at commit boundaries. This can be misread as weakening correctness; the design claim is instead that correctness is retained through speculative execution plus rollback, with atomic commit at chunk granularity.

## 4. Compressed coherence signatures

LazyPIM uses parallel Bloom filters to compress hundreds of cache-line addresses into fixed-length bitmaps. Each signature is an $N$-bit array split into $M$ equal segments, each with its own hash function. Insertion and membership test run in $O(M)$ time. The insertion rule is: for each segment $i = 1 \ldots M$, set bit $h_i(a)$. Membership testing checks $\bigwedge_{i=1..M} bit[h_i(a)]$ [1706.03162].

The false positive probability, with no false negatives, is given by

$$
P_{fp} = (1 - e^{-kn/m})^k
$$

for $k$ hash functions, $m$ total bits, and $n$ inserts. In practice, LazyPIM sets $N = 2048$, $M = 4$, $k = 4$ and caps $n \approx 250$ per chunk, yielding $P_{fp} \approx 30\%$. By bounding $n$ via partial commits, the mechanism keeps the false positive rate acceptable [1706.03162].

Because the Bloom filters are fixed-size, each PIMReadSet and PIMWriteSet is only 256 bytes, regardless of the number of addresses. The CPUWriteSet uses 16 such registers in round-robin to capture a larger working set of writes without increasing per-filter size. The compression ratio is described as

$$
R = \frac{\#addresses \times address\_size\_bits}{filter\_size\_bits}
$$

and the example given is 250 addresses times 48-bit tags, or 12,000 bits, compressed into a 2,048-bit filter, for approximately $5.9\times$ reduction in off-chip bits [1706.03162].

These signatures are central to LazyPIM’s traffic profile. Rather than transmitting per-access coherence messages, the system transmits two fixed-length signatures at the end of each kernel chunk. This suggests that the coherence protocol’s efficiency depends on a deliberate trade-off: some re-execution from false positives is accepted in exchange for sharply lower off-chip communication.

## 5. Evaluation methodology and quantitative results

The evaluation uses graph kernels—Ligra PageRank, Connected Components, and Radii—on real-world graphs, together with a hybrid in-memory database combining TPC-C-like transactions and TPC-H-like analytics. Simulation is conducted on gem5 plus DRAMSim2 with 4–16 cores; PIM cores are configured at 2 GHz, 1-issue, with 64 KB L1 caches [1706.03162].

The metrics are defined explicitly. Speedup is $S = T_{baseline} / T_{LazyPIM}$. Traffic reduction is

$$
R = \frac{Traffic_{baseline} - Traffic_{LazyPIM}}{Traffic_{baseline}} \times 100\%
$$

and energy savings are likewise measured relative to baselines [1706.03162].

For the 16-thread average, LazyPIM improves average performance across a range of data-intensive PIM applications by 19.6% over fine-grained coherence, and comes within 9.8% of an ideal no-coherence-cost PIM. Off-chip traffic is reduced by 30.9% versus the best prior PIM coherence scheme, identified as coarse-grained locks, while overall traffic is cut by 86.3% compared to CPU-only execution. Energy savings are 18.0% versus the best prior approach and 43.7% versus CPU-only; LazyPIM comes within 4.4% of ideal-PIM energy [1706.03162].

Scalability is illustrated with PageRank on the arXiv graph, where LazyPIM scales well from 4 to 16 threads, whereas CG and NC either degrade or flatten. A breakdown attributes two specific effects to the design choices: partial-commit chunking cuts conflict rates by 50–60% and bounds rollback costs, and a small 2 Kbit signature provides a sweet-spot trade-off between false positives, approximately 30%, and off-chip bits [1706.03162].

Taken together, these results position LazyPIM as an attempt to recover most of the benefit of ideal PIM while preserving correctness under coherence. The quantitative pattern also indicates that the principal gains are not limited to latency reduction; traffic and energy improvements are comparably central.

## 6. Comparison, limitations, and interpretive context

LazyPIM is compared directly against three prior or baseline approaches. Fine-grained MESI floods the bus with misses and loses PIM’s advantage. Coarse-grained locks purge tens of thousands of lines per kernel and serialize the CPU. Non-cacheable mappings bypass caches and force all CPU reads to DRAM [1706.03162]. Relative to GPU–CPU coherence optimizations such as HSC and FUSION, LazyPIM is described as addressing the harsher off-chip penalty of PIM cores in memory while retaining fine-grained atomicity with very low traffic [1706.03162].

Several limitations are identified. False positives in the Bloom filters still cause re-execution; reducing $P_{fp}$ through larger filters or more hashes could help, but would raise off-chip cost. Rollback of wide kernels can be costly; dynamic chunk sizing or hardware transactional memory support in PIM could reduce rollback overhead further. Synchronization primitives fence speculative state, but more sophisticated ordering, such as release consistency, may require extensions [1706.03162].

These limitations clarify a common misconception that LazyPIM eliminates coherence overhead altogether. The mechanism reduces overhead by speculating and batching coherence checks; it does not remove the possibility of rollback, nor does it make false positives irrelevant. Its core claim is instead that the overheads become low enough that PIM execution remains advantageous even when sharing is substantial.

More broadly, LazyPIM re-enables general-purpose PIM under a conventional multithreaded programming model by speculating in the PIM logic layer and reconciling at chunk boundaries with small compressed signatures [1706.03162]. This suggests a general design principle for PIM coherence: the decisive optimization is to replace high-frequency off-chip coherence traffic with compact summaries plus bounded re-execution, while preserving the logical behavior expected by existing multithreaded software.

Source: https://www.emergentmind.com/topics/lazypim