FlashMem: Memory Efficiency in Diverse Systems

• FlashMem denotes diverse architectures optimizing memory use by reusing computation in LLMs, mobile inference, and flash-based storage, addressing different systems constraints.
• In large language models, FlashMem distills latent memory from existing activations to reduce recomputation, achieving comparable performance at up to 5× inference speedup.
• Mobile-GPU and flash-memory applications of FlashMem demonstrate practical gains with metrics like 3.5× memory reduction and significant energy savings through static planning and efficient coding.

FlashMem is an overloaded research term applied to several technically unrelated systems concerned with memory formation, memory movement, or flash-based state retention. In recent arXiv usage, it names both a latent-memory framework for LLM agents and a mobile-GPU inference framework for large-scale DNNs; in broader flash-memory and device literature, the same label is also used for flash-based in-memory learning and non-volatile device concepts. Taken together, these usages suggest a recurrent emphasis on reducing recomputation, peak memory, or erase burden under stringent systems constraints (Hou et al., 9 Jan 2026, Shu et al., 17 Feb 2026, Ghazal et al., 2024).

1. Scope and disambiguation

A common source of confusion is the assumption that FlashMem denotes a single architecture. In the literature represented here, it instead denotes multiple systems with distinct objectives, substrates, and evaluation regimes.

Usage	Domain	Defining mechanism
FlashMem (Hou et al., 9 Jan 2026)	LLM agents	Distills latent memory from the last hidden state via a Shared-KV Consolidator and a Cognitive Monitor
FlashMem (Shu et al., 17 Feb 2026)	Mobile DNN inference	Replaces full weight preloading with static overlap planning and dynamic streaming into 2.5D texture memory
“FlashMem” architectures (Ghazal et al., 2024, Frydendahl et al., 2020, Liu et al., 2020)	Flash devices and in-memory learning	Use flash-compatible or flash-derived devices for non-volatile storage, analog state encoding, or neuromorphic operation

The two 2026 works share the name but not the problem setting. One addresses persistent cognition in stateless transformers; the other addresses weight placement and transfer in resource-constrained mobile GPUs. Earlier flash-memory works provide a broader systems and device context in which endurance, wear, erasures, and non-volatility are central concerns (III et al., 2012, Ma, 2012).

2. FlashMem as intrinsic latent memory in LLMs

In "FlashMem: Distilling Intrinsic Latent Memory via Computation Reuse" (Hou et al., 9 Jan 2026), the starting point is the statelessness of standard LLMs. Each turn follows

$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t),$

so long-horizon interaction requires re-encoding the entire history $\tau_{<t}$ at every step, incurring $O(t)$ compute. The framework proposes an explicit memory variable $\mathcal M$ so that

$$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$

without rereading the full trajectory.

The key theoretical claim is that latent memory can be distilled directly from transient reasoning states already produced by the backbone. FlashMem appeals to the injectivity of transformer representations and identifies the last hidden state

$h_t \in \mathbb R^d$

as a sufficient statistic for the interaction history. This directly contrasts with token-level memory, which bloats the context window, and parametric memory, which is slow to adapt. The framework therefore treats memory synthesis as a decoding problem over existing internal representations rather than as a separate encoding pathway (Hou et al., 9 Jan 2026).

This formulation is important because it rejects architectural segregation between reasoning and memory. Instead of training an auxiliary memory encoder that duplicates computation, FlashMem reuses activations and cached keys and values that the frozen backbone has already produced. A plausible implication is that the method targets persistent cognition primarily through representational reuse rather than through larger context windows or online parameter updates.

3. Shared-KV consolidation and entropy-gated activation

The architectural core is the Shared-KV Consolidator, a small decoder $\mathcal C_\psi$ that attends directly to the backbone’s frozen KV cache. At time $t$, the backbone processes the current input $c_t$ and returns the last hidden state $h_t$ and accumulated cache $\tau_{<t}$0. If consolidation is triggered, $\tau_{<t}$1 is projected to an initial memory token $\tau_{<t}$2, and the consolidator auto-regressively generates $\tau_{<t}$3 latent memory tokens by cross-attending to the frozen cache. Only the query projection is learned; keys and values are reused:

$\tau_{<t}$4

After $\tau_{<t}$5 layers, the system outputs $\tau_{<t}$6 (Hou et al., 9 Jan 2026).

FlashMem couples this with a parameter-free Cognitive Monitor. For each head in the last backbone layer, it masks attention sinks, renormalizes attention, computes Shannon entropy, and averages across heads to obtain $\tau_{<t}$7. Consolidation runs only when

$\tau_{<t}$8

The threshold $\tau_{<t}$9 is calibrated to the 85th percentile of $O(t)$0 on a held-out set, so only high-uncertainty outliers trigger memory formation. This makes consolidation event-driven rather than periodic.

Implementation is intentionally lightweight. The backbone $O(t)$1 is fully frozen; only consolidator parameters $O(t)$2 are trained, using supervised fine-tuning on expert CoT trajectories. Generated memory tokens are soft-injected by passing them through the backbone once to produce KV pairs, then appending those pairs to the active cache. The reported configuration uses memory token count $O(t)$3, consolidator depth $O(t)$4, hidden dimension matched to the backbone dimension $O(t)$5, AdamW with learning rate $O(t)$6, weight decay $O(t)$7, gradient clip $O(t)$8, batch size $O(t)$9, $\mathcal M$0 epochs, cosine decay with $\mathcal M$1 warmup, greedy decode for memory, and an entropy threshold set per model via validation (Hou et al., 9 Jan 2026).

4. Empirical profile of the LLM FlashMem framework

The experimental suite covers mathematical reasoning, code, and long-context summarization: GSM8K, MATH, GPQA, KodCode, BookSum, and GovReport. Backbones include Qwen-2.5B, Qwen-4B, Llama-8B, and Llama-3B. Baselines are Vanilla, CoT-SC, SnapKV, and MemGen (Hou et al., 9 Jan 2026).

The central empirical claim is that FlashMem matches or slightly exceeds MemGen while substantially reducing inference cost. On Qwen-2.5B, reported results include GSM8K $\mathcal M$2 versus $\mathcal M$3, MATH $\mathcal M$4 versus $\mathcal M$5, and BookSum R-1 $\mathcal M$6 versus $\mathcal M$7. On Llama-3B, reported results include MATH $\mathcal M$8 versus $\mathcal M$9, KodCode $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$0 versus $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$1, and GovReport R-1 $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$2 versus $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$3 (Hou et al., 9 Jan 2026).

At $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$4K context, FlashMem reports a $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$5 inference-time speedup over MemGen: latency per token is $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$6 ms versus $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$7 ms, throughput is $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$8 tokens/s versus $$a_t \sim \pi_\theta(a \mid \tau_{<t}, o_t, \mathcal M),$$9 tokens/s, and VRAM peak is $h_t \in \mathbb R^d$0 GB versus $h_t \in \mathbb R^d$1 GB, with the Vanilla baseline at $h_t \in \mathbb R^d$2 GB. The framework attributes this profile to computation reuse and to the fact that high task fidelity is maintained with only $h_t \in \mathbb R^d$3 latent tokens and $h_t \in \mathbb R^d$4 consolidator layer. The reported trade-off is therefore a modest increase in memory-management machinery in exchange for much lower recurrent inference overhead (Hou et al., 9 Jan 2026).

5. FlashMem as a mobile-GPU weight-streaming framework

A different system, "FlashMem: Supporting Modern DNN Workloads on Mobile with GPU Memory Hierarchy Optimizations" (Shu et al., 17 Feb 2026), addresses a deployment bottleneck rather than a cognition bottleneck. Its target is on-device inference for large models and multi-DNN pipelines on mobile GPUs. Existing frameworks are characterized as using a preload strategy in which all model parameters are loaded into memory before execution. FlashMem argues that this is inadequate for modern workloads involving very large models or several distinct models executed in succession.

The framework replaces full preloading with a combination of offline scheduling and runtime streaming. Peak memory under preloading is

$h_t \in \mathbb R^d$5

whereas streaming reduces the footprint to $h_t \in \mathbb R^d$6, with $h_t \in \mathbb R^d$7 acting as a tunable active-window budget. Static Overlap Plan Generation models the DNN as an execution DAG with a linearized layer order. Weights are split into fixed-size chunks, decision variables specify when chunks first enter unified memory and how many chunks are transformed into texture memory by each layer, and the offline optimization minimizes preload count versus early-load distance under completeness, load-distance, per-layer peak, and layer-capacity constraints. The problem is reduced to CP-SAT and solved offline with Google OR-Tools under a $h_t \in \mathbb R^d$8 s limit; if infeasible, an Adaptive Fusion stage splits or unfuses heavy kernels to increase capacity (Shu et al., 17 Feb 2026).

At runtime, FlashMem tiles weights offline into small 2D micro-tiles aligned to the GPU’s 2.5D image layout. This removes costly 1D-to-2.5D transformations during execution. Each operator kernel is rewritten into a fine-grained, branch-free pipeline in which iteration $h_t \in \mathbb R^d$9 computes on tile $\mathcal C_\psi$0 while issuing the texture-load for tile $\mathcal C_\psi$1, keeping GPU SIMD lanes coherent and hiding load latency behind prior computation. The paper states that the amortized streaming cost is often below $\mathcal C_\psi$2 of a kernel’s native runtime (Shu et al., 17 Feb 2026).

Evaluation spans $\mathcal C_\psi$3 models on a OnePlus 12 with Adreno 750 and $\mathcal C_\psi$4 GB RAM, including GPTNeo-Small, GPTNeo-1.3B, GPTNeo-2.7B, ResNet50, ViT, DeepViT, SAM-2, SD-UNet, Whisper-Medium, and DepthAnything variants. Against SmartMem, FlashMem reports average memory reduction of $\mathcal C_\psi$5 with range $\mathcal C_\psi$6-$\mathcal C_\psi$7, and geo-mean latency speedup of $\mathcal C_\psi$8 with range $\mathcal C_\psi$9-$t$0. Relative to all frameworks, end-to-end speedup ranges from $t$1 over LiteRT to $t$2 over ExecuTorch. The framework is also reported as the only one able to load GPTNeo-2.7B on the tested mobile platform, and it yields energy savings up to $t$3 over ExecuTorch and $t$4 over SmartMem on DeepViT and SD-UNet (Shu et al., 17 Feb 2026).

6. FlashMem in flash-memory storage, coding, and device lineages

The broader flash-memory literature gives the term a materially different meaning. At the systems level, "A Durable Flash Memory Search Tree" introduces the FM Tree, a B-tree variant optimized for multi-level flash memory, where increment operations are fast and non-destructive but erase operations are slow, destructive, and bounded by block endurance. The FM Tree stores keys in unsorted order within a block, tracks a flash-incrementable liveCount and barrenFlag, postpones erasures through lazy erasures, and uses B$t$5-style lazy rebalancing. Search, insert, and delete retain amortized $t$6 time, while experiments on a Python-based emulator show $t$7 to $t$8 fewer block erasures than a standard B-tree on synthetic workloads of $t$9 initial inserts plus $c_t$0 random inserts and deletes (III et al., 2012). In database buffering, FaCE extends the DRAM buffer with flash and uses Multi-Version FIFO and Group Second Chance to stage dirty pages, optimize sequential flash writes, and accelerate recovery. On TPC-C, FaCE+GSC with an $c_t$1 GB cache reports about $c_t$2 tpmC versus about $c_t$3 for HDD-only and about $c_t$4 for SSD-only, while restart time drops from about $c_t$5 s to about $c_t$6 s (Kang et al., 2012).

At the coding-theoretic level, "On Coding Efficiency for Flash Memories" defines payload

$c_t$7

and coding efficiency

$c_t$8

and proves that there is a fundamental trade-off between payload and coding efficiency: very low payload can drive $c_t$9, whereas higher payload forces larger expected voltage-level decrements and smaller coding efficiency (Ma, 2012). Related rewriting-code work studies the same asymmetry between easy level increases and expensive block erasure from an endurance-maximization viewpoint. "Multidimensional Flash Codes" gives a recursive construction with deficiency

$h_t$0

for storing $h_t$1 bits (0901.0702). "Rewriting Codes for Flash Memories" improves flash-code constructions to write deficiency $h_t$2 if $h_t$3 and at most $h_t$4 otherwise, and also improves buffer-code bounds (Yaakobi et al., 2012). "Writing on dirty flash memory" converts inter-cell interference into a memory-with-defective-cells model via one pre-read and uses partitioned linear block codes for additive encoding with side information (Kim et al., 2014). "Storage Coding for Wear Leveling in Flash Memories" shows that coding-based data movement requires only $h_t$5 erasures, versus at least $h_t$6 for sorting-based non-coding schemes, while using one auxiliary block and balancing erasures within one across blocks (0911.3992).

At the device and in-memory-computing level, FlashMem denotes flash-derived substrates rather than controller or data-structure policies. "In-Memory Learning Automata Architecture using Y-Flash Cell" maps each Tsetlin Automaton to a single floating-gate Y-Flash cell fabricated in a standard $h_t$7 nm CMOS process. The device supports up to $h_t$8 discrete states with $h_t$9s pulses, expandable to more than $\tau_{<t}$00 states with $\tau_{<t}$01s pulses; the implementation uses blind-write updates via a divergence counter, reports $\tau_{<t}$02 ns inference latency, and projects array densities above $\tau_{<t}$03 M cells per mm$\tau_{<t}$04 (Ghazal et al., 2024). In "Photo memtransistor based on CMOS flash memory technology on Graphene with neuromorphic applications," a MONOS-like graphene device uses short gate pulses for programmable doping, gives an On/Off ratio of about $\tau_{<t}$05 for $\tau_{<t}$06 V/$\tau_{<t}$07 V, $\tau_{<t}$08 s pulses, and estimates about $\tau_{<t}$09 fJ write energy per $\tau_{<t}$10 V spike; UV illumination enhances detrapping and improves neuromorphic plasticity (Frydendahl et al., 2020). "Ultrafast non-volatile flash memory based on van der Waals heterostructures" reports a MoS$\tau_{<t}$11/h-BN/multilayer-graphene structure with write/erase time around $\tau_{<t}$12 ns, maximum memory window $\tau_{<t}$13 V, projected retention above $\tau_{<t}$14 s with more than $\tau_{<t}$15 threshold shift remaining, endurance of at least $\tau_{<t}$16 write/erase cycles, and estimated energy of about $\tau_{<t}$17 fJ/bit for a feature-size cell (Liu et al., 2020).

These lineages are not interchangeable. The LLM and mobile-GPU FlashMem systems are software frameworks, whereas the flash-memory works concern non-volatile media, erasure asymmetry, coding, database buffering, or device physics. The shared label therefore reflects a convergence in naming around memory efficiency rather than a single coherent family of methods.