DeepNVM++: NVM GPU Cache Framework

• DeepNVM++ is a cross-layer framework that characterizes non-volatile memories as GPU last-level caches optimized for deep learning workloads.
• It integrates device-level, NVSim-based cache design, and workload analysis to evaluate SRAM, STT-MRAM, and SOT-MRAM under both iso-capacity and iso-area scenarios.
• Results show that MRAM variants can significantly reduce leakage power, area, and energy-delay product, especially as GPU cache capacity scales.

Searching arXiv for DeepNVM++ and closely related NVM-DL work to ground the article in current literature. DeepNVM++ is a cross-layer framework for characterizing, modeling, and analyzing non-volatile memories as GPU last-level caches for deep learning workloads. Introduced as an extended and improved version of the earlier DeepNVM framework, it combines technology-specific bitcell and circuit characterization, NVSim-compatible cache modeling, GPU architecture assumptions, and workload-level memory behavior to compare SRAM, STT-MRAM, and SOT-MRAM under both iso-capacity and iso-area conditions. Its stated target is the GPU L2 or last-level cache rather than latency-critical structures such as L1 caches or register files, and its central architectural question is whether dense, low-leakage MRAM variants can replace SRAM as deep learning workloads drive continued growth in on-chip cache capacity (Inci et al., 2020, Inci et al., 2022).

1. Problem setting and architectural scope

DeepNVM++ is motivated by the interaction between deep learning memory behavior and GPU cache scaling. The framework is built around the observation that deep neural networks have large model footprints, large activation tensors, and substantial data movement, while modern GPUs increasingly rely on larger L2 caches to reduce expensive off-chip DRAM traffic. In this setting, SRAM becomes problematic because its bitcell area is large and its leakage power is high, which makes very large on-chip caches increasingly expensive in power, performance, and area (Inci et al., 2020).

The framework therefore studies NVM technologies at the GPU last-level cache, where capacity sensitivity matters more than the ultra-low latency required by upper hierarchy levels. DeepNVM++ does not claim that NVM is uniformly superior across the hierarchy. On the contrary, the framework explicitly treats the LLC or L2 as the natural insertion point for STT-MRAM and SOT-MRAM, while SRAM remains the better choice for small-capacity, latency-critical applications. This distinction is fundamental to the framework’s interpretation of results and to its broader design recommendations (Inci et al., 2022).

A common misconception is to treat DeepNVM++ as a generic analog in-memory-computing platform. The framework is not a compute-in-memory crossbar model. Its scope is last-level caches in GPU architectures for deep learning, with memory technology evaluated through workload transaction counts, cache organization, and system-level energy-delay product rather than through analog MAC-array nonidealities (Inci et al., 2020).

2. Cross-layer methodology and optimization flow

DeepNVM++ is organized as a pipeline from device characterization to system-level evaluation. The flow begins with bitcell and device characterization, feeds those results into cache design exploration, and then combines cache parameters with measured or simulated workload memory statistics to obtain energy, latency, area, and EDP. The framework’s inputs include a 16 nm technology node, bitcell models for SRAM, STT-MRAM, and SOT-MRAM, cache capacities and organizations, optimization targets, GPU architectural parameters, and deep learning workloads with their memory access statistics. Its outputs include cache read and write latency, read and write dynamic energy, leakage power, area, total energy, delay, EDP, and scaling trends over cache size (Inci et al., 2020).

At the cache-design stage, DeepNVM++ uses NVSim with technology files modified to reflect the 16 nm process, and it independently tunes SRAM, STT-MRAM, and SOT-MRAM caches rather than forcing all memories into one fixed organization. The design space spans memory type $mem \in \{SRAM, STT, SOT\}$, capacity $cap \in \{1,2,4,8,16,32\}$, optimization target $opt$, and access type $acc \in \{Normal, Fast, Sequential\}$. The cache-level objective is EDAP, used as $EDAP = Energy \times Delay \times Area$, so the framework does not optimize only latency or only energy in isolation (Inci et al., 2020).

At the workload-analysis stage, the framework uses a transaction-count model. Dynamic cache energy is computed as $E_{\text{dyn}} = N_r E_r + N_w E_w$, leakage energy as $E_{\text{leak}} = P_{\text{leak}} \cdot T$, and total energy as $$E_{\text{total}} = E_{\text{dyn}} + E_{\text{leak}}$$. Delay is formed from read and write counts and per-access latencies, and EDP is reported as $EDP = E \times D$. This is a deliberately simple architecture-level model: the framework does not present a closed-form miss-ratio equation, but instead relies on real or simulated workload statistics and technology-specific cache costs (Inci et al., 2022).

3. Device models, cache assumptions, and workload characterization

DeepNVM++ demonstrates the framework on STT-MRAM and SOT-MRAM. At the bitcell level, STT-MRAM is modeled with sense latency $650$ ps, sense energy $cap \in \{1,2,4,8,16,32\}$0 pJ, write latency $cap \in \{1,2,4,8,16,32\}$1 ps for set and $cap \in \{1,2,4,8,16,32\}$2 ps for reset, write energy $cap \in \{1,2,4,8,16,32\}$3 pJ for set and $cap \in \{1,2,4,8,16,32\}$4 pJ for reset, fin count $cap \in \{1,2,4,8,16,32\}$5, and area $cap \in \{1,2,4,8,16,32\}$6 normalized to the SRAM bitcell. SOT-MRAM is modeled with sense latency $cap \in \{1,2,4,8,16,32\}$7 ps, sense energy $cap \in \{1,2,4,8,16,32\}$8 pJ, write latency $cap \in \{1,2,4,8,16,32\}$9 ps for set and $opt$0 ps for reset, write energy $opt$1 pJ for both set and reset, fin counts $opt$2 for write plus $opt$3 for read, and area $opt$4 normalized to the SRAM bitcell. These parameters encode the framework’s central device-level trade-off: both MRAM variants are denser than SRAM, while SOT-MRAM is much better than STT-MRAM for writes (Inci et al., 2020).

For cache-level studies, the reference system is a GTX 1080 Ti-like GPU at 16 nm. In the iso-capacity analysis, all memories are compared at a 3 MB L2. The corresponding cache parameters are: SRAM with read latency $opt$5 ns, write latency $opt$6 ns, read energy $opt$7 nJ, write energy $opt$8 nJ, leakage power $opt$9 mW, and area $acc \in \{Normal, Fast, Sequential\}$0 mm$acc \in \{Normal, Fast, Sequential\}$1; STT-MRAM at 3 MB with read latency $acc \in \{Normal, Fast, Sequential\}$2 ns, write latency $acc \in \{Normal, Fast, Sequential\}$3 ns, read energy $acc \in \{Normal, Fast, Sequential\}$4 nJ, write energy $acc \in \{Normal, Fast, Sequential\}$5 nJ, leakage power $acc \in \{Normal, Fast, Sequential\}$6 mW, and area $acc \in \{Normal, Fast, Sequential\}$7 mm$acc \in \{Normal, Fast, Sequential\}$8; and SOT-MRAM at 3 MB with read latency $acc \in \{Normal, Fast, Sequential\}$9 ns, write latency $EDAP = Energy \times Delay \times Area$0 ns, read energy $EDAP = Energy \times Delay \times Area$1 nJ, write energy $EDAP = Energy \times Delay \times Area$2 nJ, leakage power $EDAP = Energy \times Delay \times Area$3 mW, and area $EDAP = Energy \times Delay \times Area$4 mm$EDAP = Energy \times Delay \times Area$5 (Inci et al., 2020).

The workload suite includes AlexNet, GoogLeNet, VGG-16, ResNet-18, and SqueezeNet, with both training and inference considered. For the iso-capacity path, workloads are run on a real NVIDIA 1080 Ti using Caffe and profiled with nvprof to obtain device memory transactions and L2 read and write transactions. For the iso-area path, DeepNVM++ uses GPGPU-Sim extended to model a GTX 1080 Ti-like platform and to support larger L2 alternatives through extended DarkNet integration. A key empirical result of this profiling is that DL workloads are read-dominant at the LLC: on average, $EDAP = Energy \times Delay \times Area$6 of SRAM dynamic energy comes from reads and only $EDAP = Energy \times Delay \times Area$7 from writes. This asymmetry is one of the main reasons STT-MRAM remains viable despite its severe write-latency penalty (Inci et al., 2022).

4. Iso-capacity, iso-area, and scalability results

Under iso-capacity assumptions, DeepNVM++ asks what happens when a 3 MB SRAM LLC is replaced by a 3 MB MRAM LLC. In this regime, STT-MRAM and SOT-MRAM provide up to $EDAP = Energy \times Delay \times Area$8 and $EDAP = Energy \times Delay \times Area$9 EDP reduction and $E_{\text{dyn}} = N_r E_r + N_w E_w$0 and $E_{\text{dyn}} = N_r E_r + N_w E_w$1 area reduction compared to SRAM, respectively. The 2020 paper also reports average leakage-energy reductions of $E_{\text{dyn}} = N_r E_r + N_w E_w$2 for STT-MRAM and $E_{\text{dyn}} = N_r E_r + N_w E_w$3 for SOT-MRAM, with total-energy reduction dominated by leakage rather than by dynamic-access energy (Inci et al., 2020).

Under iso-area assumptions, the framework fixes the area budget instead of the cache capacity. Using the area of the 3 MB SRAM cache, DeepNVM++ fits a 7 MB STT-MRAM cache or a 10 MB SOT-MRAM cache, corresponding to $E_{\text{dyn}} = N_r E_r + N_w E_w$4 and $E_{\text{dyn}} = N_r E_r + N_w E_w$5 larger capacity than the SRAM baseline. In the AlexNet GPGPU-Sim study, these larger caches reduce total DRAM accesses by $E_{\text{dyn}} = N_r E_r + N_w E_w$6 for STT-MRAM and $E_{\text{dyn}} = N_r E_r + N_w E_w$7 for SOT-MRAM. The 2020 paper reports up to $E_{\text{dyn}} = N_r E_r + N_w E_w$8 and $E_{\text{dyn}} = N_r E_r + N_w E_w$9 EDP reduction with DRAM included, while the 2022 overview reports headline maxima of up to $E_{\text{leak}} = P_{\text{leak}} \cdot T$0 and $E_{\text{leak}} = P_{\text{leak}} \cdot T$1 EDP reduction for STT-MRAM and SOT-MRAM, respectively (Inci et al., 2020, Inci et al., 2022).

The scalability analysis extends the cache capacity sweep from 1 MB to 32 MB and is the framework’s strongest forward-looking result. As capacity increases, same-capacity area reduction averages $E_{\text{leak}} = P_{\text{leak}} \cdot T$2 for STT-MRAM and $E_{\text{leak}} = P_{\text{leak}} \cdot T$3 for SOT-MRAM. SRAM has lower read latency below about 3 MB, but beyond about 4 MB both MRAM variants have lower read latency than SRAM. At larger capacities, DeepNVM++ reports up to $E_{\text{leak}} = P_{\text{leak}} \cdot T$4 energy reduction for STT-MRAM and $E_{\text{leak}} = P_{\text{leak}} \cdot T$5 for SOT-MRAM, together with up to $E_{\text{leak}} = P_{\text{leak}} \cdot T$6 and $E_{\text{leak}} = P_{\text{leak}} \cdot T$7 EDP reduction, respectively. This is the basis for the paper’s claim that NVM LLCs become increasingly attractive as GPU cache capacity grows (Inci et al., 2022).

Batch-size sensitivity further refines the picture. For AlexNet training, STT-MRAM yields $E_{\text{leak}} = P_{\text{leak}} \cdot T$8 to $E_{\text{leak}} = P_{\text{leak}} \cdot T$9 EDP reduction as batch size increases, while SOT-MRAM yields $$E_{\text{total}} = E_{\text{dyn}} + E_{\text{leak}}$$0 to $$E_{\text{total}} = E_{\text{dyn}} + E_{\text{leak}}$$1. For inference, STT-MRAM yields $$E_{\text{total}} = E_{\text{dyn}} + E_{\text{leak}}$$2 to $$E_{\text{total}} = E_{\text{dyn}} + E_{\text{leak}}$$3 and SOT-MRAM $$E_{\text{total}} = E_{\text{dyn}} + E_{\text{leak}}$$4 to $$E_{\text{total}} = E_{\text{dyn}} + E_{\text{leak}}$$5. These results reinforce the conclusion that SOT-MRAM is robust across access mixes, whereas STT-MRAM benefits more when workloads become increasingly read-dominant (Inci et al., 2020).

5. Design insights, trade-offs, and interpretive significance

DeepNVM++ does not reduce the SRAM-versus-NVM question to a single scalar ranking. Its principal insight is that leakage and density are the decisive advantages of MRAM at the LLC. Even when STT-MRAM or SOT-MRAM incur higher dynamic energy for particular accesses, especially reads, the large reduction in leakage can dominate total cache energy. In iso-area mode, density creates a second-order benefit by allowing larger caches that cut DRAM traffic, and the framework repeatedly emphasizes that DRAM accesses are much more expensive than on-chip accesses (Inci et al., 2022).

A second central insight is that read/write asymmetry matters. Because DL workloads are read-dominated at the L2, STT-MRAM’s poor write behavior is less catastrophic than it would be in write-heavy applications. This is not a universal endorsement of STT-MRAM. Rather, it is a workload-contingent architectural claim. SOT-MRAM is generally the most attractive option in DeepNVM++ because it combines density and low leakage with much better write latency and write energy than STT-MRAM, and it usually provides the strongest EDP improvement (Inci et al., 2020).

A third insight is methodological. Iso-capacity and iso-area comparisons answer different questions and should not be conflated. Iso-capacity isolates the direct effect of swapping memory technology at fixed storage size, thereby foregrounding area and leakage reduction. Iso-area instead evaluates whether the density of NVM can be converted into larger cache capacity and lower DRAM traffic under the same silicon budget. DeepNVM++ treats both comparisons as necessary because many earlier studies focused on only one of them (Inci et al., 2022).

This also clarifies a recurrent misunderstanding: DeepNVM++ does not argue that NVM caches are always lower-latency than SRAM. For small capacities, SRAM remains superior in read latency. The framework’s stronger claim is a scalability claim: as LLC capacity grows, SRAM scales poorly in area, leakage, and eventually latency, whereas MRAM variants scale more favorably. This suggests that the architectural case for NVM strengthens with the continued growth of GPU last-level caches.

6. Relationship to adjacent NVM-DL research and framework limitations

Within the broader NVM-for-deep-learning literature, DeepNVM++ occupies a cache-centric position. It is complementary to analog compute-in-memory training frameworks such as the crossbar-oriented methodology of “Training DNN IoT Applications for Deployment On Analog NVM Crossbars,” which addresses global range uniformity, unipolar weight mappings, and analog peripheral reuse rather than GPU LLC substitution (García-Redondo et al., 2019). It is likewise orthogonal to training-side robustness methods such as OVF, which target stochastic NVCIM programming variation and reduced reliance on write-verify during deployment, but do not analyze GPU caches or iso-capacity versus iso-area cache trade-offs (Qin et al., 17 Aug 2025).

The framework is also distinct from cache-embedded or device-novel IMC proposals. “NVM-in-Cache” studies a 6T-2R cache cell with compute-on-powerline and retention-aware in-cache analog MAC, which is a cache-compatible processing-in-memory architecture rather than a last-level-cache technology-replacement study (Chakraborty et al., 15 Sep 2025). The 2D piezoelectric-transistor work on negative-voltage-enabled PeFET memories instead explores dense nonvolatile storage and low-precision IMC primitives such as addition, subtraction, and MAC through device- and line-bias co-design, again addressing a different layer of the design stack (Victor et al., 5 Apr 2026). This suggests that DeepNVM++ is best understood as a cross-layer evaluation and design-space exploration methodology for NVM caches, not as a full-stack CIM deployment framework.

Its limitations are correspondingly specific. The framework is demonstrated on STT-MRAM and SOT-MRAM only, even though it claims generality to other NVMs. It studies only GPU last-level caches, not L1 caches, register files, or a fully redesigned hierarchy. The architecture-level model is intentionally simple, using transaction counts multiplied by per-access latency and energy. The iso-area study uses AlexNet in GPGPU-Sim rather than the full benchmark suite. The framework does not model endurance, write wear, retention failure, or broader hybrid-cache policy design in depth (Inci et al., 2020). A plausible implication is that DeepNVM++ is most valuable as a comparative infrastructure for last-level-cache technology selection and scaling analysis, while broader deployment studies would need to combine it with training-aware robustness, mapping-aware CIM models, or device-specific reliability frameworks.