NVM-in-Cache: Architectural Insights

• NVM-in-Cache architecture refers to design strategies that integrate non-volatile memory into cache systems to enhance persistence, recovery, and performance.
• It encompasses diverse implementations such as DRAM fronting NVM main memory, NVM-based processor caches, and cache-aware persistence protocols addressing latency, endurance, and write amplification.
• Emerging approaches repurpose caches for analog processing and in-memory computation, demonstrating NVM's potential beyond traditional storage hierarchies.

Searching arXiv for the papers on arXiv to ground the article in current records. NVM-in-cache architecture denotes a family of architectural strategies in which non-volatile memory participates directly in cache organization, cache-state persistence, or cache-adjacent recovery mechanisms rather than serving only as a lower storage tier. In the literature represented here, the term covers several distinct but related designs: DRAM used as a cache for PCM or 3DXPoint-class main memory; processor caches built wholly or partly from NVM such as STT-RAM; cache-aware persistence protocols that exploit volatile caches while preserving correctness in NVM-backed systems; and middleware or I/O caches that use NVMM as a persistent page store or write log. This suggests that the topic is best understood as a design space organized around three recurring questions: what state resides in the cache hierarchy, what portion of that state must survive failures, and how the asymmetries of NVM latency, write cost, endurance, and persistence should shape placement, migration, and recovery policy (Yoon et al., 2018, Escuin et al., 2022, Cohen et al., 2019, Chakraborty et al., 15 Sep 2025).

1. Conceptual scope and architectural boundaries

The literature does not treat NVM-in-cache architecture as a single canonical microarchitecture. One strand studies memory-side caching, in which a smaller DRAM tier fronts a larger NVM main memory and the memory controller decides what data should reside in DRAM. Another strand studies processor-cache implementation, replacing or augmenting SRAM cache arrays with NVM such as STT-RAM or hybrid SRAM/STT-RAM organizations. A third strand studies cache-aware persistence, where caches remain volatile but are deliberately incorporated into durability, recovery, or secure-metadata protocols. A fourth strand studies software and middleware caches, where NVMM or Optane is placed between applications and conventional storage as a persistent page cache or write-back log (Babaie et al., 2023, Badri et al., 2022, Cohen et al., 2017, Dulong et al., 2021).

A common misconception is that NVM-in-cache necessarily means a physically non-volatile CPU cache. Several influential works explicitly assume the opposite. “Fine-Grain Checkpointing with In-Cache-Line Logging” treats caches as transient and exploits same-cache-line persistence ordering under the Persistent Cache Store Order model (Cohen et al., 2019). “Efficient Logging in Non-Volatile Memory by Exploiting Coherency Protocols” likewise assumes ordinary coherent volatile caches and derives a one-round-trip logging protocol from line-local persistence ordering (Cohen et al., 2017). Conversely, works such as “L2C2: Last-Level Compressed-Cache NVM and a Procedure to Forecast Performance and Lifetime” and “A Method for Hiding the Increased Non-Volatile Cache Read Latency” treat NVM as the physical medium of the last-level cache itself (Escuin et al., 2022, Kokolis et al., 2021). The phrase therefore marks a design territory, not a single implementation style.

2. DRAM fronting NVM in hybrid main memory

In hybrid main memory, DRAM frequently acts as a cache to a larger PCM or 3DXPoint-class NVM. “A Memory Controller with Row Buffer Locality Awareness for Hybrid Memory Systems” studies a large PCM memory with a smaller DRAM cache and argues that cache placement should be driven not simply by hotness but by the asymmetry between DRAM and PCM on row buffer misses. Its RBLA policy tracks row buffer miss counts per row in a stats store located in the memory controller, promotes a row when its count exceeds MissThresh, resets counters every 10 million cycles, and in RBLA-Dyn adapts the threshold by a cost-benefit hill-climbing rule. Relative to the frequency-based baseline FREQ, RBLA-Dyn improves average system performance by 14%, improves maximum slowdown by 6%, and improves memory energy efficiency by 10%; the modeled 9.25 KB stats store reaches within 0.3% of unlimited-size stats-store performance and within 2.5% of unlimited-size memory lifetime (Yoon et al., 2018).

“A Cycle-level Unified DRAM Cache Controller Model for 3DXPoint Memory Systems in gem5” moves from policy to controller realism. Its Unified DRAM Cache Controller models a direct-mapped, insert-on-miss, write-back, 64 B cache block size DRAM cache over NVM main memory, with a unified Outstanding Request Buffer, Conflict Request Buffer, and NVM write-back queue (Babaie et al., 2023). The model makes explicit the access amplification of DRAM-cached NVM systems: a read hit incurs one DRAM access, a write hit incurs one DRAM read and one DRAM write, a read miss incurs a DRAM tag read, an NVM read, and a DRAM fill, and a write miss adds an NVM write-back. Its evaluation shows that FR-FCFS helps mainly when DRAM hits dominate: at 100% hit, FR-FCFS improves bandwidth over FCFS by 2.56× for read-only traffic and 2.85× for write-only traffic, whereas at 0% hit and write-only traffic it provides 1.0× improvement because the system is fully limited by NVM. The same work also models wear leveling as an extra 60 µs delay for every 14,000 writes to each 256 B region of NVRAM, reducing peak bandwidth from 1.92 GB/s to 1.77 GB/s in an all-write, 100%-miss configuration (Babaie et al., 2023).

Taken together, these works imply that in DRAM-over-NVM hierarchies the central problem is not merely cache hit rate. Row-buffer locality, migration overhead, request serialization, finite buffering, and NVM write-side stalls materially determine whether the DRAM tier behaves as a latency shield or becomes a source of access amplification.

3. NVM as the cache medium in on-chip caches

A more literal reading of NVM-in-cache places NVM inside the processor cache itself. “Accelerating Non-volatile/Hybrid Processor Cache Design Space Exploration for Application Specific Embedded Systems” studies NVM-only caches and hybrid volatile/non-volatile caches under line wear-out, assuming that one faulty bit makes a cache line completely unusable and that line failures independently reduce per-set associativity. Its Breakneck Cache Performance Evaluation Method exploits per-set decomposition and interpolation to avoid the combinatorial explosion of degraded configurations; on SPEC CPU 2000 it accelerates FIFO cache design-space exploration by up to 249× with miss-prediction accuracy within +6% and -5% (Haque et al., 2015).

At L1, “Efficient Placement and Migration Policies for an STT-RAM based Hybrid L1 Cache for Intermittently Powered Systems” proposes a 4-way cache with 2 SRAM ways + 2 STT-RAM ways, plus per-line RIC, WIC, and CONF metadata and a PR predictor to direct placement. The policy places write-intensive blocks in SRAM and read-intensive blocks in STT-RAM, migrating on threshold crossings with the threshold set to 7. Compared with the baseline hybrid architecture, it reduces STT-RAM writes from 63.35% to 35.93%, yields a 32.85% performance gain, reduces energy consumption by 23.42%, and reports backup-time reduction of 34.46% in the abstract (Badri et al., 2022).

At the LLC, two different issues dominate. “A Method for Hiding the Increased Non-Volatile Cache Read Latency” addresses read latency in an STT-RAM LLC with Page Buffers and a page-aware layout. Cloak uses L1 TLB misses as a signal for page-level reuse, promotes LLC-resident page lines into small SRAM Page Buffers, and reports that on average it outperforms an SRAM LLC by 23.8% and an NVM-only LLC by 8.9%, while its $ED^2$ is 39.9% and 17.5% lower, respectively (Kokolis et al., 2021). “L2C2: Last-Level Compressed-Cache NVM and a Procedure to Forecast Performance and Lifetime” addresses endurance rather than latency. It uses compression not to increase associativity but to reduce writes and to keep partially degraded frames usable. In its 16 MB STT-RAM LLC study, L2C2 extends the time to 50% effective-capacity degradation by roughly 6× to 37× over a frame-disabling baseline and raises LLC hit latency from 30 cycles to 32 cycles (Escuin et al., 2022). A closely related forecasting study reports that its compression-based organization increases by more than 6 times the time a NV-LLC takes to reach 50% effective capacity, and that the time to lose 15% of IPC is 3.1× longer than in the frame-disabling baseline (Escuin et al., 2022).

These results underscore a persistent theme: when NVM becomes the cache medium, density and leakage advantages are real, but architectural viability depends on explicit countermeasures for read latency, write wear, degraded-capacity operation, and placement policy.

4. Recoverability, secure metadata, and volatile cache state

Secure and persistent memory systems introduce a different NVM-in-cache problem: data survive crashes, but the security metadata needed to decrypt and authenticate them may reside in volatile metadata caches. “Phoenix: Towards Persistently Secure, Recoverable, and NVM Friendly Tree of Counters” studies SGX-style tree-of-counters protection and emphasizes that, unlike a conventional Merkle tree, “the whole tree cannot be recovered from leaves” (Alwadi et al., 2019). Phoenix therefore reconstructs the metadata cache rather than strictly persisting every tree update. It persists intermediate ToC nodes eagerly, records dirty metadata addresses in a protected Cache Mirror, and reconstructs encryption-counter leaves using an external recovery mechanism such as Osiris. Quantitatively, relative to a secure write-back baseline without recoverability, Anubis adds 87% extra writes on average, Phoenix cuts that to 12.9% extra writes, and Phoenix$^+$ reports 3.8% fewer writes than the baseline write-back scheme itself. Recovery remains a function of metadata cache size rather than total memory size; for cache sizes up to 4 MB, Phoenix$^+$ recovery is reported around 0.12 s in the worst case, compared with about 0.015 s for Anubis (Alwadi et al., 2019).

A plausible implication is that persistent NVM does not eliminate the need to reason about cache state; it changes which cache-resident structures must be persisted exactly, which can be reconstructed, and which require small trusted summaries. Phoenix makes this explicit with its asymmetric persistence policy: unrecoverable internal tree state is persisted, recoverable leaf state is reconstructed. In that sense, volatile cache state becomes a first-class recovery object rather than merely a liability.

5. Cache-aware persistence protocols and intermittent-state preservation

A second major body of work keeps caches volatile but exploits their ordering or constrains their dirty state so that NVM-backed recovery is still efficient. “Fine-Grain Checkpointing with In-Cache-Line Logging” divides execution into 64 ms epochs, flushes the entire cache with wbinvd at epoch boundaries, and embeds undo metadata in the same cache line as the protected data (Cohen et al., 2019). Its key invariants rely on the Persistent Cache Store Order rule that same-cache-line writes persist in program order. Applied to Masstree, the technique incurs 5.9–15.4% runtime overhead, with the global flush costing 1.38–1.39 ms per epoch, or roughly 2.2% overhead (Cohen et al., 2019). “Efficient Logging in Non-Volatile Memory by Exploiting Coherency Protocols” generalizes the same idea into a logging subsystem that requires only one round trip to non-volatile memory, using validity metadata written last within the same cache line under $\mathcal{P}$CSO (Cohen et al., 2017).

In intermittent systems, the issue is not ordering but bounded backup cost. “An Efficient NVM based Architecture for Intermittent Computing under Energy Constraints” keeps a volatile SRAM L1, places STT-RAM at the LLC, uses PCM as main memory, and constrains the number of dirty L1 blocks through a Dirty Block Table (DBT) and Writeback Queue (WBQ) so that backup always fits within capacitor energy (Badri et al., 2022). The architecture reports 17.56% lower energy consumption, 18.97% fewer writes to STT-RAM LLC, and 10.66% fewer writes to PCM main memory compared to the baseline architecture, while deriving the safe dirty-block budget from capacitor energy via $K = M + N$ and related energy expressions (Badri et al., 2022).

These works jointly show that cache-aware persistence is not reducible to flush instructions. It encompasses same-line ordering, epochal recovery points, dirty-state budgeting, and deliberate use of volatile cache semantics as part of the durability protocol.

6. Middleware caches, optimization frameworks, and emerging directions

At the software and middleware layer, NVM-in-cache design becomes a problem of hierarchy composition and workload-specific optimization. “Stochastic Modeling of Hybrid Cache Systems” analyzes flat and layered hybrid page-cache architectures using DRAM and PCM, concluding that under the paper’s common PCM setting flat architecture is preferable, whereas layered architecture outperforms if PCM write performance can be significantly improved in the future (Ju et al., 2016). “Memory Hierarchy Design for Caching Middleware in the Age of NVM” formulates media selection, capacity sizing, and replication versus tiering as a Multiple Choice Knapsack Problem, showing that selective replication is appropriate with certain failure rates and workload characteristics, while “with a slim failure rate and frequent data updates, tiering of data across the different storage media that constitute the cache is superior to replication” (Ghandeharizadeh et al., 5 Jun 2025).

Two NVMM cache designs illustrate the implementation spectrum. “NVCache: A Plug-and-Play NVMM-based I/O Booster for Legacy Systems” uses NVMM as a write-back cache in NVMM implemented entirely in user space, with a persistent circular log, asynchronous cleanup thread, and durable linearizability; on write-oriented workloads it is at least 1.9× faster than DM-WriteCache+SSD and plain SSD among large-storage-space systems, and in one ideal-case FIO benchmark reaches 493 MiB/s versus 403 MiB/s for NOVA (Dulong et al., 2021). “NVMM cache design: Logging vs. Paging” compares an NVMM page cache (NVPages) with an NVMM write log plus small DRAM cache (NVLog) and finds that logging performs significantly better in almost every workload; one reason is that NVPages serves hits from NVMM and therefore inherits NVMM bandwidth limitations, whereas NVLog preserves DRAM read bandwidth (Dulong et al., 2023). “Writes Hurt: Lessons in Cache Design for Optane NVRAM” reaches a related conclusion for a volatile Optane block cache: on Optane, writes suppress reads so strongly that hit rate becomes an inadequate objective, and the cache regulates churn with $$OBP = \frac{blocks\_inserted + blocks\_removed}{blocks\_looked\_up}$$, targeting roughly 10% (Fedorova et al., 2022).

An emerging endpoint of the design space no longer treats NVM-in-cache primarily as a latency or persistence mechanism but as a compute substrate. “NVM-in-Cache: Repurposing Commodity 6T SRAM Cache into NVM Analog Processing-in-Memory Engine using a Novel Compute-on-Powerline Scheme” embeds two RRAM devices into a conventional 6T SRAM cell, forming a 6T-2R bit-cell that preserves SRAM cache behavior while enabling analog MAC on cache power lines (Chakraborty et al., 15 Sep 2025). In GlobalFoundries 22nm FDSOI simulations, the design reports 0.4 TOPS, 491.78 TOPS/W, and 91.27% CIFAR-10 accuracy for a ResNet-18 mapping with 128 row-parallel operations (Chakraborty et al., 15 Sep 2025). This suggests that the phrase NVM-in-cache now spans not only caching and persistence but also direct repurposing of cache arrays for processing-in-memory.

Across these layers, the unifying principle is not the mere presence of non-volatility. It is the deliberate redefinition of cache roles—fast tier, persistent checkpoint boundary, secure-recovery object, write log, or analog compute fabric—under the asymmetries introduced by NVM media.