Short-term RAM (StRAM) Overview

• Short-term RAM (StRAM) is a memory-class abstraction optimized for fleeting data lifetimes (microseconds to milliseconds) with improved density and energy efficiency.
• It leverages retention-relaxed devices like gain-cell eDRAM and short-retention STT-RAM to offer near-SRAM latencies while reducing leakage and refresh energy.
• Integrating StRAM as an intermediate memory tier enhances performance in applications such as deep learning activations, key-value stores, and transient system workloads.

Short-term RAM (StRAM) designates a memory-class abstraction optimized for ephemeral, highly dynamic data whose lifetimes are measured in microseconds to a few hundred milliseconds. Unlike persistent storage or primary memory layers that maximize data retention and non-volatility, StRAM is architected for frequent read and write operations on data structures that are discarded rapidly. This specialization enables trade-offs in device retention, energy efficiency, area, and system integration. StRAM implementation is realized via retention-relaxed volatile devices—such as short-retention STT-RAM, gain-cell eDRAM, or volatile RRAM—typically deployed between on-chip SRAM and off-chip DRAM in emerging non-hierarchical memory ecosystems. By tuning device and system parameters to short data lifetimes, StRAM closes the gap between high-throughput cache and bulk DRAM, substantially reducing leakage and refresh energy, while maintaining read/write symmetry and near-SRAM latency for transient workloads (Li et al., 5 Aug 2025, Kuan et al., 2019, Kuan et al., 2020, Khoshavi et al., 2016).

1. Core Concept and Motivation

StRAM emerged in response to the scaling bottlenecks of traditional SRAM and DRAM, which have reached limits in density, efficiency, and economic viability. Standard memory hierarchies, optimized for near-unbounded data retention, result in substantial overprovisioning for dynamic, short-lived data prevalent in contemporary workloads—including neural network activations, ephemeral message-passing buffers, and transient pointer structures (Li et al., 5 Aug 2025).

StRAM is formally defined as a memory tier for highly dynamic, frequently accessed, and transient data. Its essential features include:

• Symmetrical read/write performance at latencies within an order of magnitude of SRAM, yet with significantly higher physical density (2–4× that of six-transistor SRAM).
• Unbounded endurance, enabling continuous high-frequency updates without cell wear constraints.
• Relaxed retention, with data reliably stored for lifetimes sufficient for common transient use (from several microseconds to low hundreds of milliseconds).
• Low static energy consumption, i.e., much lower leakage than SRAM.

By trading long retention for increased density and energy efficiency, StRAM efficiently supports activation buffers in AI models, in-memory pointer structures, and temporary computational working sets (Li et al., 5 Aug 2025, Kuan et al., 2019).

2. Underlying Device Technologies

Practical StRAM instantiations leverage volatile memory devices whose retention characteristics can be aggressively managed. Three representative device candidates are prominent in the literature:

• Gain-cell eDRAM: Promotes short retention via capacitance scaling, reaching ∼6–12 ns access latency, refresh intervals down to 200 μs, and over 10¹⁴ write cycles endurance. On-die variants deliver energy-per-access below 5 pJ/read and 10 pJ/write, with density 2–4× higher than SRAM (Li et al., 5 Aug 2025).
• Retention-relaxed STT-RAM (also called "short-retention STTRAM" or LRSC): By reducing the MTJ free-layer area (thus lowering the thermal stability factor Δ), designers obtain tunable retention (1 μs–100 ms) with dramatically reduced write energy and pulse width. At τ = 10 ms, write energy is ≈5 pJ and write latency ≈25 ns; read latency is largely unaffected (≈2 ns) (Kuan et al., 2019, Khoshavi et al., 2016).
• Volatile resistive and ferroelectric memories: RRAM and FeRAM, when engineered for millisecond-class retention, yield comparable symmetry and endurance, with further trade-offs in process complexity and integration (Li et al., 5 Aug 2025).

Device-level energy and retention relationships are typically characterized as:

$τ = τ_0 \exp(Δ), \quad τ_0 ≈ 1\ \mathrm{ns}$

where lowering Δ via physical scaling reduces τ, write current ($I_w$), and write energy ($E_w = I_w^2 R t_{wp}$). These relationships underpin all STTRAM-based StRAM tuning (Kuan et al., 2019).

3. System Integration and Memory Hierarchy Placement

StRAM is integrated as an intermediate or parallel tier alongside SRAM (caches) and DRAM (main memory), breaking from the strictly layered latency-based hierarchy. This non-hierarchical placement is intended for data with well-profiled, short lived access patterns (Li et al., 5 Aug 2025). Key system integration modalities include:

• On-chip banks: Deployed as private or shared L1/L2 cache banks, or as scratchpad arrays, typically with explicit device-level retention control.
• 3D-stacked or near-die banks: Placed physically close to accelerators or processors for high-bandwidth, low-latency access exceeding 10 GB/s per pin; aggregate throughput can reach hundreds of GB/s.
• Dynamic and programmable allocation: Operating systems and runtimes can allocate transient buffers directly to StRAM by page-table flags, API calls, or programmer/runtime annotations (e.g., "Short-Term" zones in high-level languages).

Profiling of application access patterns directs optimal placement and retention parameterization. Lightweight sampling (e.g., every 10 M instructions) or machine-learned models can dynamically select retention for each core or workload (Kuan et al., 2019).

4. Performance, Energy, and Endurance Characteristics

StRAM achieves distinct trade-offs compared to both SRAM and DRAM, as summarized in benchmark studies:

Memory Type	Latency (ns)	Read Energy (pJ)	Write Energy (pJ)	Density (vs SRAM)	Retention
6T SRAM	1–2	20–30	20–30	1×	Indefinite (while powered)
DRAM	15–30	40–60	40–60	~4×	32 ms (refresh)
StRAM (STT-RAM, τ = 10 ms)	2–25	0.1–0.5	5	2–4×	10 ms
eDRAM-based StRAM	6–12	<5	<10	2–4×	0.2–1 ms (refresh)

Performance evaluations demonstrate:

• StRAM with τ = 10 ms delivers ≈60–75% energy savings over SRAM in mobile processors, with negligible (<5%) execution time overhead. For quad-core systems, energy savings reach ≈72% with up to 14% speedup, driven by reduced leakage (Kuan et al., 2019).
• In hybrid L2 designs using short-retention STTRAM, mean L2 read-miss ratio can be reduced by 51.4% and overall IPC increased by 11.7% over SRAM L2, with 80–90% lower L2 energy expenditures (Khoshavi et al., 2016).
• FPGA prototypes of gain-cell eDRAM StRAM exhibit 3 pJ/read, 6 pJ/write, and 35% lower total system power under realistic kernel-scratch workloads (Li et al., 5 Aug 2025).

Endurance in StRAM arrays routinely exceeds competitive requirements for ephemeral data (e.g., >10¹⁴ write cycles for eDRAM, ~10¹² for STTRAM at 10 ms refresh), enabling effectively unconstrained operation throughout system lifetimes.

5. Runtime Adaptation: Prefetching and Retention Tuning

StRAM's retention exposes a critical trade-off: reducing τ improves write energy and latency, but overly short τ increases "expiration misses" as data is lost before potential reuse. Cache and prefetching algorithms on StRAM must be retention-aware.

A key runtime metric, expired_unused_prefetches (EUP), measures the proportion of prefetched lines that expire without serving a useful access (Kuan et al., 2020). The metric guides allocation of both retention time (τ) and prefetch distance (d) by the following workflow:

• Sweep over τ at minimal d, monitoring EUP(τ,1).
• Select τ* as the minimal safe retention avoiding EUP blowup.
• Determine d* via a lookup table on measured EUP(τ*,1); lower EUP admits larger prefetch distances.

Empirical results show that this retention-prefetch co-tuning (PART+RPC) reduces average cache energy and latency by 22.2% and 24.6% over multi-τ STTRAM with static prefetch, outperforming best-in-class dynamic prefetch throttling by 3.5%/3.6%, with just a handful of counters and minimal hardware (Kuan et al., 2020).

6. System-Level Use Cases and Evaluation

StRAM enables several performance-critical memory use cases:

• Deep learning inference/training: Transient activation and feature maps, updated every pass, achieve 1.8× lower per-layer energy than DRAM and 2.3× higher density than L2 SRAM (Li et al., 5 Aug 2025).
• Key-value stores: Memcached hot sets with microsecond-lived pointers and buckets yield 25 ns round-trips in StRAM (vs. 60–70 ns in DRAM), improving 99th-percentile latency by 30%.
• Log and telemetry buffers: Placing event buffers with millisecond flush cycles in StRAM reduces system static power by 40% relative to SRAM.

Simulator studies show that interposing a 4 GB StRAM tier between cache and DRAM reduces subsystem energy by 22% and trims system slowdown from 1.15× to 1.03× for mixed workloads (Li et al., 5 Aug 2025). Quad-core mobile SoCs realize ≈72% energy savings and ≈14% speedup in full-app runs when adopting 10 ms retention StRAM over traditional SRAM cache (Kuan et al., 2019).

7. Research Challenges and Future Directions

Advancing StRAM adoption at scale introduces several unresolved technical frontiers:

• Profiling and prediction: Efficiently determining allocation lifetimes and access patterns at sub-page granularity, potentially guided by compiler or runtime annotations.
• Data placement and space partitioning policies: Designing adaptive, robust migration triggers and capacity-sharing policies for dynamic and multi-tenant workloads.
• Consistency and coherence: Managing StRAM’s risk of unexpected data expiration within cache coherence protocols, especially as retention tuning or per-core τ heterogeneity increases.
• Thermal and packaging integration: Addressing hotspot formation and power-delivery constraints in dense, rapidly refreshed StRAM arrays embedded within advanced 3D IC stacks.
• System software and programming model: Standardizing OS interfaces, allocator APIs, and language annotations to expose StRAM as a first-class, explicitly-allocated resource.

A plausible implication is that, as memory requirements diversify and traditional DRAM/SRAM scaling falters, the division and explicit management of short-term (StRAM) and long-term (LtRAM) memory classes will become standard in performance-sensitive system designs (Li et al., 5 Aug 2025). StRAM provides the foundation for post-hierarchical, application-specialized memory topologies, supporting the demands of modern AI, server, and graphics workloads with efficient, flexible, and robust transient storage.