---
title: 'NVM-in-Cache: Architectural Insights'
url: https://www.emergentmind.com/topics/nvm-in-cache-architecture
type: topic
---

# NVM-in-Cache: Architectural Insights

Searching arXiv for the provided papers to ground the article in current records.
NVM-in-cache architecture denotes a family of architectural strategies in which non-volatile memory participates directly in cache organization, cache-state persistence, or cache-adjacent recovery mechanisms rather than serving only as a lower storage tier. In the literature represented here, the term covers several distinct but related designs: DRAM used as a cache for PCM or 3DXPoint-class main memory; processor caches built wholly or partly from NVM such as STT-RAM; cache-aware persistence protocols that exploit volatile caches while preserving correctness in NVM-backed systems; and middleware or I/O caches that use NVMM as a persistent page store or write log. This suggests that the topic is best understood as a design space organized around three recurring questions: what state resides in the cache hierarchy, what portion of that state must survive failures, and how the asymmetries of NVM latency, write cost, endurance, and persistence should shape placement, migration, and recovery policy [1804.11040] [2204.09504] [1902.00660] [2510.15904].

## 1. Conceptual scope and architectural boundaries

The literature does not treat NVM-in-cache architecture as a single canonical microarchitecture. One strand studies **memory-side caching**, in which a smaller DRAM tier fronts a larger NVM main memory and the memory controller decides what data should reside in DRAM. Another strand studies **processor-cache implementation**, replacing or augmenting SRAM cache arrays with NVM such as STT-RAM or hybrid SRAM/STT-RAM organizations. A third strand studies **cache-aware persistence**, where caches remain volatile but are deliberately incorporated into durability, recovery, or secure-metadata protocols. A fourth strand studies **software and middleware caches**, where NVMM or Optane is placed between applications and conventional storage as a persistent page cache or write-back log [2303.13026] [2203.02226] [1709.02610] [2105.10397].

A common misconception is that NVM-in-cache necessarily means a physically non-volatile CPU cache. Several influential works explicitly assume the opposite. “Fine-Grain Checkpointing with In-Cache-Line Logging” treats caches as transient and exploits same-cache-line persistence ordering under the Persistent Cache Store Order model [1902.00660]. “Efficient Logging in Non-Volatile Memory by Exploiting Coherency Protocols” likewise assumes ordinary coherent volatile caches and derives a one-round-trip logging protocol from line-local persistence ordering [1709.02610]. Conversely, works such as “L2C2: Last-Level Compressed-Cache NVM and a Procedure to Forecast Performance and Lifetime” and “A Method for Hiding the Increased Non-Volatile Cache Read Latency” treat NVM as the physical medium of the last-level cache itself [2204.09504] [2112.10632]. The phrase therefore marks a design territory, not a single implementation style.

## 2. DRAM fronting NVM in hybrid main memory

In hybrid main memory, DRAM frequently acts as a cache to a larger PCM or 3DXPoint-class NVM. “A Memory Controller with Row Buffer Locality Awareness for Hybrid Memory Systems” studies a **large PCM memory** with a **smaller DRAM cache** and argues that cache placement should be driven not simply by hotness but by the asymmetry between DRAM and PCM on **row buffer misses**. Its RBLA policy tracks **row buffer miss counts per row** in a **stats store** located in the memory controller, promotes a row when its count exceeds **MissThresh**, resets counters every **10 million cycles**, and in RBLA-Dyn adapts the threshold by a cost-benefit hill-climbing rule. Relative to the frequency-based baseline FREQ, RBLA-Dyn improves average system performance by **14%**, improves maximum slowdown by **6%**, and improves memory energy efficiency by **10%**; the modeled **9.25 KB** stats store reaches within **0.3%** of unlimited-size stats-store performance and within **2.5%** of unlimited-size memory lifetime [1804.11040].

“A Cycle-level Unified DRAM Cache Controller Model for 3DXPoint Memory Systems in gem5” moves from policy to controller realism. Its Unified DRAM Cache Controller models a **direct-mapped**, **insert-on-miss**, **write-back**, **64 B cache block size** DRAM cache over NVM main memory, with a unified **Outstanding Request Buffer**, **Conflict Request Buffer**, and **NVM write-back queue** [2303.13026]. The model makes explicit the access amplification of DRAM-cached NVM systems: a **read hit** incurs one DRAM access, a **write hit** incurs one DRAM read and one DRAM write, a **read miss** incurs a DRAM tag read, an NVM read, and a DRAM fill, and a **write miss** adds an NVM write-back. Its evaluation shows that FR-FCFS helps mainly when DRAM hits dominate: at **100% hit**, FR-FCFS improves bandwidth over FCFS by **2.56×** for read-only traffic and **2.85×** for write-only traffic, whereas at **0% hit** and **write-only** traffic it provides **1.0×** improvement because the system is fully limited by NVM. The same work also models wear leveling as an extra **60 µs** delay for every **14,000 writes to each 256 B region** of NVRAM, reducing peak bandwidth from **1.92 GB/s** to **1.77 GB/s** in an all-write, 100%-miss configuration [2303.13026].

Taken together, these works imply that in DRAM-over-NVM hierarchies the central problem is not merely cache hit rate. Row-buffer locality, migration overhead, request serialization, finite buffering, and NVM write-side stalls materially determine whether the DRAM tier behaves as a latency shield or becomes a source of access amplification.

## 3. NVM as the cache medium in on-chip caches

A more literal reading of NVM-in-cache places NVM inside the processor cache itself. “Accelerating Non-volatile/Hybrid Processor Cache Design Space Exploration for Application Specific Embedded Systems” studies **NVM-only caches** and **hybrid volatile/non-volatile caches** under line wear-out, assuming that **one faulty bit makes a cache line completely unusable** and that line failures independently reduce per-set associativity. Its Breakneck Cache Performance Evaluation Method exploits per-set decomposition and interpolation to avoid the combinatorial explosion of degraded configurations; on SPEC CPU 2000 it accelerates FIFO cache design-space exploration by up to **249×** with miss-prediction accuracy within **+6% and -5%** [1506.03193].

At L1, “Efficient Placement and Migration Policies for an STT-RAM based Hybrid L1 Cache for Intermittently Powered Systems” proposes a **4-way** cache with **2 SRAM ways + 2 STT-RAM ways**, plus per-line **RIC**, **WIC**, and **CONF** metadata and a **PR** predictor to direct placement. The policy places **write-intensive blocks** in SRAM and **read-intensive blocks** in STT-RAM, migrating on threshold crossings with the threshold set to **7**. Compared with the baseline hybrid architecture, it reduces STT-RAM writes from **63.35%** to **35.93%**, yields a **32.85%** performance gain, reduces energy consumption by **23.42%**, and reports backup-time reduction of **34.46%** in the abstract [2203.02226].

At the LLC, two different issues dominate. “A Method for Hiding the Increased Non-Volatile Cache Read Latency” addresses read latency in an STT-RAM LLC with **Page Buffers** and a page-aware layout. Cloak uses L1 TLB misses as a signal for page-level reuse, promotes LLC-resident page lines into small SRAM Page Buffers, and reports that on average it outperforms an SRAM LLC by **23.8%** and an NVM-only LLC by **8.9%**, while its \(ED^2\) is **39.9%** and **17.5%** lower, respectively [2112.10632]. “L2C2: Last-Level Compressed-Cache NVM and a Procedure to Forecast Performance and Lifetime” addresses endurance rather than latency. It uses compression not to increase associativity but to reduce writes and to keep partially degraded frames usable. In its 16 MB STT-RAM LLC study, L2C2 extends the time to **50% effective-capacity degradation** by roughly **6× to 37×** over a frame-disabling baseline and raises LLC hit latency from **30 cycles to 32 cycles** [2204.09504]. A closely related forecasting study reports that its compression-based organization increases by **more than 6 times** the time a NV-LLC takes to reach **50% effective capacity**, and that the time to lose **15% of IPC** is **3.1×** longer than in the frame-disabling baseline [2204.03512].

These results underscore a persistent theme: when NVM becomes the cache medium, density and leakage advantages are real, but architectural viability depends on explicit countermeasures for read latency, write wear, degraded-capacity operation, and placement policy.

## 4. Recoverability, secure metadata, and volatile cache state

Secure and persistent memory systems introduce a different NVM-in-cache problem: data survive crashes, but the security metadata needed to decrypt and authenticate them may reside in volatile metadata caches. “Phoenix: Towards Persistently Secure, Recoverable, and NVM Friendly Tree of Counters” studies SGX-style tree-of-counters protection and emphasizes that, unlike a conventional Merkle tree, “the whole tree cannot be recovered from leaves” [1911.01922]. Phoenix therefore reconstructs the **metadata cache** rather than strictly persisting every tree update. It persists intermediate ToC nodes eagerly, records dirty metadata addresses in a protected **Cache Mirror**, and reconstructs encryption-counter leaves using an external recovery mechanism such as Osiris. Quantitatively, relative to a secure write-back baseline without recoverability, Anubis adds **87% extra writes** on average, Phoenix cuts that to **12.9% extra writes**, and Phoenix\(^+\) reports **3.8% fewer writes than the baseline write-back scheme itself**. Recovery remains a function of metadata cache size rather than total memory size; for cache sizes up to **4 MB**, Phoenix\(^+\) recovery is reported around **0.12 s** in the worst case, compared with about **0.015 s** for Anubis [1911.01922].

A plausible implication is that persistent NVM does not eliminate the need to reason about cache state; it changes which cache-resident structures must be persisted exactly, which can be reconstructed, and which require small trusted summaries. Phoenix makes this explicit with its asymmetric persistence policy: unrecoverable internal tree state is persisted, recoverable leaf state is reconstructed. In that sense, volatile cache state becomes a first-class recovery object rather than merely a liability.

## 5. Cache-aware persistence protocols and intermittent-state preservation

A second major body of work keeps caches volatile but exploits their ordering or constrains their dirty state so that NVM-backed recovery is still efficient. “Fine-Grain Checkpointing with In-Cache-Line Logging” divides execution into **64 ms epochs**, flushes the entire cache with `wbinvd` at epoch boundaries, and embeds undo metadata in the same cache line as the protected data [1902.00660]. Its key invariants rely on the Persistent Cache Store Order rule that same-cache-line writes persist in program order. Applied to Masstree, the technique incurs **5.9–15.4% runtime overhead**, with the global flush costing **1.38–1.39 ms** per epoch, or roughly **2.2% overhead** [1902.00660]. “Efficient Logging in Non-Volatile Memory by Exploiting Coherency Protocols” generalizes the same idea into a logging subsystem that requires only **one round trip to non-volatile memory**, using validity metadata written last within the same cache line under \(\mathcal{P}\)CSO [1709.02610].

In intermittent systems, the issue is not ordering but bounded backup cost. “An Efficient NVM based Architecture for Intermittent Computing under Energy Constraints” keeps a volatile SRAM L1, places **STT-RAM at the LLC**, uses **PCM as main memory**, and constrains the number of dirty L1 blocks through a **Dirty Block Table (DBT)** and **Writeback Queue (WBQ)** so that backup always fits within capacitor energy [2212.08993]. The architecture reports **17.56%** lower energy consumption, **18.97%** fewer writes to STT-RAM LLC, and **10.66%** fewer writes to PCM main memory compared to the baseline architecture, while deriving the safe dirty-block budget from capacitor energy via \(K = M + N\) and related energy expressions [2212.08993].

These works jointly show that cache-aware persistence is not reducible to flush instructions. It encompasses same-line ordering, epochal recovery points, dirty-state budgeting, and deliberate use of volatile cache semantics as part of the durability protocol.

## 6. Middleware caches, optimization frameworks, and emerging directions

At the software and middleware layer, NVM-in-cache design becomes a problem of hierarchy composition and workload-specific optimization. “Stochastic Modeling of Hybrid Cache Systems” analyzes **flat** and **layered** hybrid page-cache architectures using DRAM and PCM, concluding that under the paper’s common PCM setting **flat architecture** is preferable, whereas **layered architecture** outperforms if PCM write performance can be significantly improved in the future [1607.00714]. “Memory Hierarchy Design for Caching Middleware in the Age of NVM” formulates media selection, capacity sizing, and replication versus tiering as a **Multiple Choice Knapsack Problem**, showing that selective replication is appropriate with certain failure rates and workload characteristics, while “with a slim failure rate and frequent data updates, tiering of data across the different storage media that constitute the cache is superior to replication” [2506.05071].

Two NVMM cache designs illustrate the implementation spectrum. “NVCache: A Plug-and-Play NVMM-based I/O Booster for Legacy Systems” uses NVMM as a **write-back cache in NVMM** implemented entirely in user space, with a persistent circular log, asynchronous cleanup thread, and durable linearizability; on write-oriented workloads it is at least **1.9×** faster than DM-WriteCache+SSD and plain SSD among large-storage-space systems, and in one ideal-case FIO benchmark reaches **493 MiB/s** versus **403 MiB/s** for NOVA [2105.10397]. “NVMM cache design: Logging vs. Paging” compares an NVMM page cache (**NVPages**) with an NVMM write log plus small DRAM cache (**NVLog**) and finds that logging performs significantly better in almost every workload; one reason is that NVPages serves hits from NVMM and therefore inherits NVMM bandwidth limitations, whereas NVLog preserves DRAM read bandwidth [2305.02244]. “Writes Hurt: Lessons in Cache Design for Optane NVRAM” reaches a related conclusion for a volatile Optane block cache: on Optane, writes suppress reads so strongly that hit rate becomes an inadequate objective, and the cache regulates churn with \(OBP = \frac{blocks\_inserted + blocks\_removed}{blocks\_looked\_up}\), targeting roughly **10%** [2205.14122].

An emerging endpoint of the design space no longer treats NVM-in-cache primarily as a latency or persistence mechanism but as a compute substrate. “NVM-in-Cache: Repurposing Commodity 6T SRAM Cache into NVM Analog Processing-in-Memory Engine using a Novel Compute-on-Powerline Scheme” embeds two RRAM devices into a conventional 6T SRAM cell, forming a **6T-2R** bit-cell that preserves SRAM cache behavior while enabling analog MAC on cache power lines [2510.15904]. In **GlobalFoundries 22nm FDSOI** simulations, the design reports **0.4 TOPS**, **491.78 TOPS/W**, and **91.27%** CIFAR-10 accuracy for a ResNet-18 mapping with **128 row-parallel operations** [2510.15904]. This suggests that the phrase NVM-in-cache now spans not only caching and persistence but also direct repurposing of cache arrays for processing-in-memory.

Across these layers, the unifying principle is not the mere presence of non-volatility. It is the deliberate redefinition of cache roles—fast tier, persistent checkpoint boundary, secure-recovery object, write log, or analog compute fabric—under the asymmetries introduced by NVM media.

Source: https://www.emergentmind.com/topics/nvm-in-cache-architecture