MEMTIER: Memory Tiering & AI Memory Architecture

• MEMTIER is a memory tiering approach that dynamically allocates data across multiple performance tiers, including DRAM, persistent memory, and compressed pools.
• It uses metrics like fast-memory miss ratios and heat-gradient binning to dynamically place hot versus cold data, thereby optimizing QoS and reducing overhead.
• MEMTIER also extends into autonomous AI agent systems with a tripartite memory design—episodic, semantic, and procedural stores—to enhance retrieval and coherence.

MEMTIER commonly denotes memory tiering: the organization of main memory into multiple performance tiers, with dynamic placement of hot and cold data to balance performance, cost, capacity, and QoS. In current systems research, the tiers may be DRAM and Intel Optane persistent memory, multiple software-defined compressed pools, remote memory plus disk, or near/far segments created inside DRAM and PCM. In a distinct 2026 line of work, “MEMTIER” also names a tripartite memory architecture for long-running autonomous AI agents, combining episodic, semantic, and procedural stores with attribution-aware retrieval and policy adaptation (Raybuck et al., 2023, Kumar et al., 2024, Rayhan et al., 3 Mar 2026, Song et al., 2020, Sidik et al., 5 May 2026).

1. Scope and conceptual foundations

In server and systems literature, memory tiering places an application’s hottest data in fast memory and colder data in slow memory, then continuously adapts placement as access patterns evolve. On DRAM + Intel Optane persistent memory platforms, the fast tier is local DRAM and the slow tier is a larger, higher-latency PMem region in App-Direct mode. In database settings, the same idea generalizes to an $n$-tier hierarchy in which Tier-0 is local DRAM, Tier-1 is remote memory, and the last tier is disk; the first $n-1$ tiers are byte-addressable memory and the final tier is disk (Raybuck et al., 2023, Rayhan et al., 3 Mar 2026).

The notion of a “tier” is not restricted to physical media. TierScape defines multiple software-defined compressed memory tiers through combinations of compression algorithms, compressed-object allocators, and backing media, so that each tier occupies a distinct point on the access-latency, compressibility, and unit-cost spectrum. The evaluated representative tiers span configurations such as zbud + lz4 on DRAM, zbud + lz4 on Optane, zsmalloc + lz4 on Optane, zsmalloc + lzo on DRAM, and zsmalloc + deflate on Optane (Kumar et al., 2024).

A further generalization appears in MNEME, which introduces intra-memory asymmetry by splitting long bitlines with an isolation transistor. This creates near and far segments inside both DRAM and PCM, producing a four-tier substrate from the processor’s perspective: DRAM-near, DRAM-far, NVM-near, and NVM-far (Song et al., 2020).

In the autonomous-agent setting, MEMTIER refers to a tripartite architecture rather than to hardware-media placement. The episodic tier stores write-once daily session logs in structured JSONL; the semantic tier stores asynchronously consolidated facts; and the procedural tier encodes stable workflows or rules. The architectural motivation is a long-running memory coherence problem in OpenClaw, where tool-execution success degrades by 14 percentage points over 72 hours under a flat-file design (Sidik et al., 5 May 2026).

2. Telemetry, heat estimation, and retrieval signals

A central problem in memtier systems is observability: the system must estimate which pages or regions are hot without excessive overhead. MaxMem/TopTier uses Intel PEBS via perf_event_open to sample roughly 1% of loads that miss in L3 and are served from DRAM versus PMem. Samples include virtual address and PID, enabling per-process accounting. Samples are quantized to huge pages, accumulated into counters, and grouped into 6 per-process hotness bins with exponential thresholds such as 1, 2, 4, 8, 16, and 32. A cooling step halves counts when the hottest bin reaches its maximum threshold, thereby maintaining a usable dynamic range (Raybuck et al., 2023).

TopTier’s principal control signal is the fast-memory miss ratio (FMMR), defined as the fraction of a process’s memory accesses served from the slow tier. Per epoch,

$$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$

and the system maintains an exponentially weighted moving average

$$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$

with $\lambda = 0.5$, evaluated once per second. The heat gradient is then the ordered list of bins from hottest to coldest, and placement fills DRAM from hottest bins while evicting from coldest bins (Raybuck et al., 2023).

TierScape also relies on PEBS, but it profiles MEM_INST_RETIRED.ALL_LOADS/ALL_STORES, aggregates hotness at 2 MB region granularity, and uses 120 s profile windows with a 100 s warm-up window. The analytical model takes the average hotness per region over the last four windows and solves an ILP with OR-Tools; the waterfall model instead demotes regions progressively from T1 to T2 to T3 and so on, while any region accessed during the window is faulted or decompressed back to DRAM (Kumar et al., 2024).

ARMS replaces explicit hotness thresholds with relative ranking and dual-horizon moving averages. For page $p$ at interval $t$, with access count $x_t$, it computes

$$\mathrm{STMA}_t = \alpha_s x_t + (1-\alpha_s)\mathrm{STMA}_{t-1}, \quad \alpha_s=0.7,$$

$$\mathrm{LTMA}_t = \alpha_l x_t + (1-\alpha_l)\mathrm{LTMA}_{t-1}, \quad \alpha_l=0.1,$$

and a score

$n-1$0

Pages are ranked by score, and the top $n-1$1 pages, where $n-1$2 equals DRAM capacity in pages, are classified as hot. Page–Hinkley detection on far-memory bandwidth triggers a recency mode that doubles PEBS sampling rate and increases policy cadence from 500 ms to 100 ms until the bandwidth stabilizes (Yadalam et al., 6 Aug 2025).

In the agent-memory MEMTIER, observability is retrieval-centric rather than page-centric. The scoring function combines semantic relevance, BM25, time decay, cognitive weight, and a tier bonus:

$n-1$3

Default weights are $n-1$4, BM25 uses $n-1$5 and $n-1$6, time decay is $n-1$7 with $n-1$8, and cognitive weight is mapped as $n-1$9 with $$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$0 (Sidik et al., 5 May 2026).

3. Representative architectures

System	Substrate	Defining mechanism
TopTier / libMaxMem (Raybuck et al., 2023)	DRAM + Intel Optane PMem	User-space manager, FMMR targets, 6 heat bins
TierScape (Kumar et al., 2024)	DRAM + multiple compressed tiers	Multiple active zswap pools with (algorithm, allocator, backing-media) triples
ARMS (Yadalam et al., 6 Aug 2025)	DRAM + NVM or CXL-like remote NUMA	Threshold-free ranking, cost/benefit migration, bandwidth-aware batches
vmcache$$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$1 (Rayhan et al., 3 Mar 2026)	DRAM + RMem + disk	Stable VA, per-tier resident sets, move_pages2
MNEME (Song et al., 2020)	Segmented DRAM + segmented PCM	Near/far tiers inside each memory, FTI-guided placement
MEMTIER (Sidik et al., 5 May 2026)	Agent runtime memory	Episodic JSONL, semantic facts, procedural boost

TopTier implements memtier entirely in user space. A central manager coordinates tiering across participating processes, while libMaxMem intercepts mmap for large allocations, registers regions with a userfaultfd-based handler, and cooperates in page remapping because of limited kernel support for remapping another process’s address space. Migration uses I/OAT DMA when available and otherwise falls back to parallel copy threads; pages are write-protected during migration, with writes deferred until migration completes (Raybuck et al., 2023).

TierScape extends Linux zswap to support multiple concurrent pools rather than a single active pool. Each pool is configured by a compression algorithm, a compressed-object allocator, and a backing medium, and a modified madvise() sets a per-page tier_id. Page migration between compressed tiers is currently implemented by decompressing and recompressing, although the paper notes that future optimization can avoid decompression when algorithms match (Kumar et al., 2024).

ARMS is a user-level library injected through LD_PRELOAD. It intercepts mmap/free, manages tiers via DAX files mapped at startup, tracks pages at 2 MB huge-page granularity, collects PEBS samples for LLC load misses and retired stores, and migrates pages concurrently in prioritized batches sized by available bandwidth headroom (Yadalam et al., 6 Aug 2025).

vmcache$$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$2 is a virtual-memory-assisted buffer pool in which each database page keeps a stable virtual address for its entire lifetime. Pages are never replicated across tiers; exactly one resident copy exists at any time. The design adds per-tier resident sets, clock replacement, probabilistic promotion and demotion, and batched page migration through a new syscall, move_pages2, which exposes migration mode, explicit batch-size control, and optimistic per-page error handling (Rayhan et al., 3 Mar 2026).

MNEME combines hardware changes and OS policy. It inserts an isolation transistor per bitline in both DRAM and PCM, uses a segment-select bit in the row address to choose per-tier timing and voltage from a LUT, and predicts first-touch behavior via two Bloom filters, FTI_R and FTI_W, plus a small Access Intensity Record. The initial allocation policy places write-intensive pages in DRAM near, read-intensive pages in PCM near, conservative cases in DRAM near, and unrecognized first touches in PCM far if space is available, otherwise DRAM far (Song et al., 2020).

The agent-memory MEMTIER stores episodic entries in ~/.openclaw/workspace/memory/episodic/YYYY-MM-DD.jsonl with fields id, timestamp, session_id, project, content, tokens, promoted, and cognitive_weight. A consolidation daemon extracts facts via heuristic + LLM, deduplicates them by Jaccard, tags origin agent and project, writes to the semantic tier, and rebuilds a BM25 index used in Stage 1 session scoping (Sidik et al., 5 May 2026).

4. Control laws, optimization objectives, and migration economics

TopTier operationalizes QoS through per-process target miss ratios $$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$3, where lower values encode higher priority, $$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$4 disables tiering, and $$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$5 means best effort. At each epoch it computes $$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$6 and allocates reallocation bandwidth proportionally:

$$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$7

for processes above target, and

$$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$8

for processes below target, subject to not taking more DRAM than they own. The practical occupancy update is

$$a_{\text{miss,inst}} = \frac{a_{\text{slow}}}{a_{\text{slow}} + a_{\text{fast}}},$$9

If aggregate DRAM is insufficient, the system satisfies as many targets as possible on a first-come-first-served basis, flags the remainder, and minimizes reallocation churn until capacity frees up (Raybuck et al., 2023).

TierScape expresses placement as a constrained optimization over performance overhead and TCO. The maximum TCO savings opportunity is

$$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$0

and the model solves

$$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$1

Its estimator is

$$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$2

with $$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$3 in the experiments, while the tiered cost model is

$$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$4

The scalar $$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$5 is the user-visible knob along the performance–TCO frontier (Kumar et al., 2024).

ARMS makes migration decisions through a direct benefit-versus-cost test. For a candidate hot page $$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$6 in slow memory and the coldest DRAM page $$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$7 to demote,

$$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$8

$$a_{\text{miss}} \leftarrow \lambda \cdot a_{\text{miss,prev}} + (1-\lambda)\cdot a_{\text{miss,inst}},$$9

and promotion occurs iff $\lambda = 0.5$0. The scheduler then sizes concurrent migrations as

$\lambda = 0.5$1

so that batch size shrinks when the application is bandwidth-heavy and grows when spare bandwidth exists (Yadalam et al., 6 Aug 2025).

vmcache$\lambda = 0.5$2 makes the migration bottleneck explicit. The paper decomposes batch migration cost as

$\lambda = 0.5$3

and identifies TLB shootdowns as the dominant in-kernel cost during migration. move_pages2 therefore adds nr_max_batched_migration to amortize shootdowns, plus MIGRATE_ASYNC, MIGRATE_SYNC, and MIGRATE_SYNC_LIGHT modes and optimistic error handling that records per-page failures without aborting the entire migration (Rayhan et al., 3 Mar 2026).

MNEME broadens the optimization space from latency alone to energy and reliability. It uses the simplified relations

$\lambda = 0.5$4

and models NBTI aging as

$\lambda = 0.5$5

This makes near segments attractive not only because they are faster, but also because they require lower operating voltages and reduce aging stress in PCM peripherals (Song et al., 2020).

5. Empirical behavior in servers and databases

On a tiered DRAM + Intel Optane platform, MaxMem’s evaluation shows that when the working set fits, TopTier incurs $\lambda = 0.5$6 overhead versus HeMem in a single-process microbenchmark. When hot + warm sets exceed DRAM, heat-gradient binning delivers approximately 30% higher throughput than HeMem’s threshold-based placement. In static colocation, TopTier matches HeMem’s best static partition on tail latency and throughput while using 32% less DRAM on average. In dynamic colocation, as additional best-effort tasks arrive and FlexKVS’s hot set grows from 42 GB to 74 GB, TopTier delivers 11% higher throughput than HeMem and 38% higher than AutoNUMA, with up to 80% lower 99th percentile latency than HeMem and up to an order of magnitude lower 99th percentile latency than AutoNUMA (Raybuck et al., 2023).

TierScape evaluates DRAM plus five compressed tiers on Memcached, Redis, BFS, PageRank, and XSBench. Across real benchmarks, it reports 22–40 percentage point higher memory TCO savings while maintaining performance parity compared to 2-tier baselines, or 2–10 percentage point performance improvements while maintaining memory TCO parity. The detailed examples include Redis, where 2T-A saves 34.84% TCO with 18.33% slowdown, 6T-WF-A saves 56.11% with 19.48% slowdown, and 6T-AM-0.1 saves 64.10% with 14.37% slowdown; and PageRank, where 6T-WF-C simultaneously improves performance and TCO relative to 2T-C (Kumar et al., 2024).

ARMS evaluates DRAM + NVM and DRAM + CXL-like remote NUMA settings across GapBS, Silo, Btree, XSBench, and a dynamic-hot-set GUPS microbenchmark. It is within approximately 3% of the best tuned HeMem across workloads, improves out-of-the-box performance by a geometric mean of 1.26× over default HeMem, 1.34× over Memtis, and 2.3× over TPP, and still leads on NUMA/CXL-like hardware by approximately 10% over HeMem. The paper attributes these gains to fewer wasteful migrations, faster demotions and promotions, and ranking that automatically adapts to DRAM size (Yadalam et al., 6 Aug 2025).

For database buffer management, vmcache$\lambda = 0.5$7 demonstrates the value of adding remote memory as an intermediate tier while preserving stable virtual addresses. With remote memory at 2× and 4× DRAM capacity, TPC-C throughput improves by up to 1.67× and 3.82× over two-tier vmcache, while random read improves by 1.36× at 4× DRAM. Against move_pages, move_pages2 improves random-read query throughput by 1.42× and page-migration throughput by 1.32×, although migration remains the primary bottleneck when the working set fits in memory tiers and access is highly random (Rayhan et al., 3 Mar 2026).

MNEME reports that, in a DRAM + PCM hybrid shared-address-space system, execution time decreases by 21% versus Baseline and 16% versus Nimble on average. In a PCM-main-memory system with DRAM as a write cache, MNEME improves by 15% versus Baseline and 14% versus segmentation-only. It directs approximately 64% of accesses to near segments, versus approximately 13% for Nimble, reduces migration-related channel accesses by 71.2%, cuts total memory energy by 19% versus Baseline and 18% versus Nimble, improves endurance-related lifetime by 20% versus Nimble, and reduces NBTI-related aging by 33% versus Nimble (Song et al., 2020).

6. MEMTIER for long-running autonomous AI agents

In OpenClaw, MEMTIER is a software memory architecture motivated by a flat-file failure mode rather than by heterogeneous hardware. The reported production symptom is a 14 percentage point drop in tool-execution success over 72-hour operation windows. The cited causes are context collapse from a 20 KB MEMORY.md cap, compaction discontinuity in which 62% of context-compaction events produce measurable behavioral breaks, structural blindness caused by unstructured flat text, and the absence of an attribution loop linking tool outcomes back to the memory entries that influenced them (Sidik et al., 5 May 2026).

The architecture is tripartite. The episodic tier is an agent-private, append-only JSONL log of session-bound events, tool calls, and content, with per-entry provenance and cognitive_weight. The semantic tier is project-shared and stores distilled, deduplicated facts produced asynchronously by a consolidation daemon using heuristic + LLM extraction and Jaccard-based deduplication. The procedural tier is optional and encodes stable workflows or rules. Retrieval is two-stage: Stage 1 runs BM25 on the semantic tier and returns the top-5 session IDs; Stage 2 scores episodic entries scoped to those sessions using BM25, time decay, cognitive weight, and tier boost (Sidik et al., 5 May 2026).

The weighting scheme is deliberately explicit. Tier multipliers are $\lambda = 0.5$8 for episodic, semantic, and procedural, respectively. BM25 is normalized to $\lambda = 0.5$9, but the raw BM25 score retains a bypass role: if raw BM25 exceeds 2.0, time decay is suppressed. Cognitive weight is updated through an EMA-like rule,

$p$0

where $p$1 is an attribution score. The preferred attribution source is logprob-based attention attribution via SGLang, but the evaluated system uses lexical Jaccard overlap as fallback because hardware constraints limited activation of the logprob path (Sidik et al., 5 May 2026).

The paper also defines a PPO-based framework for learning the retrieval weight vector. The clipped surrogate objective is

$p$2

with task-success reward

$p$3

However, the paper explicitly reports that infrastructure was validated while performance gains from the camera-ready PPO configuration were still pending, and that PPO-learned weights matched the default weights in the initial run because BM25’s unbounded signal dominated bounded auxiliary signals (Sidik et al., 5 May 2026).

Evaluation uses the full 500-question LongMemEval-S benchmark across 53 sessions. With Qwen2.5-7B on a consumer laptop with a 6 GB GPU, MEMTIER reaches $p$4 and $p$5, improving the full-context baseline from 0.050 to 0.382, i.e., from 5% to 38%, a gain of 33 percentage points. With DeepSeek-V4-Flash semantic fact pre-population, single-session recall reaches 0.686–0.714, exceeding the paper’s RAG BM25 GPT-4o baseline of 0.560 on those categories. Reported temporal reasoning is 0.323 in the abstract and 0.316 in the table; multi-session synthesis is 0.173 in the abstract and 0.180 in the table, with the paper attributing the differences to minor run variance and rounding (Sidik et al., 5 May 2026).

The ablation results identify the semantic tier as the dominant contributor. Removing semantic pre-population drops accuracy by 0.128; removing two-stage scoping drops accuracy by 0.038; removing time decay, cognitive weight, or tier boost changes accuracy by about 0.014–0.016. The token-efficiency analysis reports that LLM fact extraction reduces semantic facts per question from approximately 509 to approximately 3.1, a roughly 164× reduction, coinciding with large accuracy and F1 gains (Sidik et al., 5 May 2026).

7. Limitations, misconceptions, and open problems

A recurring misconception is that memtier is equivalent to a binary DRAM-versus-slow-memory cache with fixed thresholds. The surveyed systems instead span per-process QoS control via FMMR, multi-tier compressed memory, threshold-free top-$p$6 ranking, VA-stable buffer pools across DRAM, remote memory, and disk, and hardware-created near/far tiers inside both DRAM and PCM. This suggests that “memtier” now functions as a broad design space rather than a single policy family (Raybuck et al., 2023, Kumar et al., 2024, Yadalam et al., 6 Aug 2025, Rayhan et al., 3 Mar 2026, Song et al., 2020).

The limitations are correspondingly diverse. MaxMem intentionally ignores NVM-specific read/write asymmetry and access granularity to generalize to future slow tiers, so write-heavy workloads on PMem may deviate from its read-centric assumptions; its experiments also pin to a single NUMA node, and the paper notes that a misbehaving process could inflate observed heat by touching unused ranges. TierScape is sensitive to data compressibility, access-pattern stability, migration overhead from decompress→recompress, and deployment-specific media pricing and compression statistics. ARMS handles many threshold pathologies, but streaming or scan-like access patterns can make frequency-based promotion counterproductive, motivating application hints such as madvise to bypass promotion (Raybuck et al., 2023, Kumar et al., 2024, Yadalam et al., 6 Aug 2025).

vmcache$p$7 exposes a separate boundary condition: the approach requires tiers that the OS exposes as normal system RAM in System-DRAM mode, while Device DAX/App-Direct mappings are incompatible with its stable-VA remapping model. Even with batching, migration remains a bottleneck because TLB shootdowns are expensive. MNEME reduces inter-memory migrations and channel occupancy, but its gains are limited on low-MPKI or compute-bound workloads, and rapid changes in first-touch behavior can still induce suboptimal initial placements until the next relearning phase (Rayhan et al., 3 Mar 2026, Song et al., 2020).

In agent memory, a different misconception appears: scaling the generator or adding policy learning does not by itself remove a retrieval ceiling. The MEMTIER paper reports “generator invariance” and “weight invariance,” with DeepSeek-V4-Flash and Qwen2.5-7B achieving similar accuracy under identical pipelines and PPO-learned weights matching default weights because BM25 dominates bounded auxiliary signals. The remaining open problems are explicit: dense or hybrid retrieval, stricter normalization of BM25, higher-fidelity logprob attribution, richer relation extraction, and better temporal normalization for multi-session synthesis and temporal reasoning (Sidik et al., 5 May 2026).