Graph-Optimized Memory (GOM)

• GOM is a memory optimization paradigm that employs graph structures to expose dependencies, improve locality, and enhance retrieval efficiency across systems.
• It spans techniques from hierarchical layout (HBA) to trainable memory for LLM agents and compiler-assisted scheduling, yielding notable speedups and reduced memory usage.
• The approach transforms memory management into an optimization problem over dependencies, lifetimes, and tensor placements, enabling both hardware and algorithmic improvements.

Searching arXiv for the cited papers to ground the article. Graph-Optimized Memory (GOM) denotes a family of graph-centered memory formulations in which memory organization, retrieval, placement, or execution is optimized through explicit graph structure rather than treated as a flat store or a purely reactive runtime mechanism. In recent literature, the term has been used in several distinct senses: as the opinion-aware memory subsystem for LLM-driven core agents in GASim (Zhou et al., 8 May 2026); as compiler-driven hierarchical memory management over a computation graph in HyperOffload (Liu et al., 31 Jan 2026); as feature-decomposed execution for memory-efficient GNN inference (Zhou et al., 2021); as graph-level operator ordering and tensor layout optimization in ROAM (Shu et al., 2023); as a trainable graph memory for strategic LLM-agent guidance (Xia et al., 11 Nov 2025); and, in an earlier systems sense, as hierarchy-sensitive graph layout for improving locality across cache lines, TLBs, and DRAM pages (Roy, 2012). This suggests that GOM is best understood as a recurring design pattern rather than a single standardized algorithm.

1. Scope, lineage, and major senses of the term

An early systems formulation of graph-optimized memory is the Hierarchical Blocking Algorithm (HBA), which lays out graph nodes in main memory according to a parameterized hierarchy of locality units $s_1 < s_2 < \dots < s_n$ and copies nodes using a hierarchy of breadth first searches. In the reported evaluation, $s[1] = 64$, $s[2] = 1024$, $s[3] = 4096$, and $s[4] = 2097152$; under the stated hierarchy separation assumptions, the traversal bound $B_i^i \le 4$ holds for all levels. On graph workloads, HBA(all) improved BFS by 1.40 on Watts-Strogatz, 1.11 on Barabasi-Albert, 3.80 on 2d mesh, and 21.31 on 4-ary tree, while a JikesRVM integration achieved about 29% speedup on a Java BST (Roy, 2012).

Later work broadened the label from physical graph layout to graph-structured retrieval, graph-level tensor scheduling, and trainable strategic memory. The term therefore spans both semantic memory and systems memory. A plausible implication is that the shared idea is not the storage medium itself, but the use of graph structure to expose dependencies, lifetimes, or relevance relations that flat memory abstractions obscure.

Usage	Graph object	Optimization target
GASim (Zhou et al., 8 May 2026)	Per-agent message memory graph	Retrieval without LLM calls
HyperOffload (Liu et al., 31 Jan 2026)	Computation graph with cache operators	Offload/prefetch scheduling
GNN inference (Zhou et al., 2021)	Feature-partitioned message passing	Peak memory and cache locality
ROAM (Shu et al., 2023)	Training DAG	Operator order and tensor layout
Trainable graph memory (Xia et al., 11 Nov 2025)	Query-path-meta graph	Strategy selection via RL
HBA (Roy, 2012)	In-memory graph layout	Locality across the memory hierarchy

2. Opinion-aware retrieval memory in GASim

Within GASim, GOM is the memory subsystem for LLM-driven core agents. GASim partitions the population into core agents, selected by Entropy-Driven Grouping (EDG), and ordinary agents, updated numerically by Graph Message Passing (GMP). GOM sits between the environment and the LLM for core agents: incoming neighbor messages are converted into a query vector $\mathbf{q}$, the memory graph $$G_{\text{mem}} = (\mathcal{V}, \mathcal{E}, \mathbf{W})$$ is updated, a retrieval score vector $\mathbf{f}$ is computed, Top-$R$ memories are selected, and only those textual memories are passed to the LLM for Observe–Recall–Act reasoning. Each node stores raw content $s[1] = 64$0, content embedding $s[1] = 64$1, keyword embedding $s[1] = 64$2, and scalar opinion $s[1] = 64$3; edges are constructed by k-nearest neighbors in embedding space with signed weights

$s[1] = 64$4

so the graph is sparse, weighted, and opinion-aware (Zhou et al., 8 May 2026).

Retrieval is formulated as optimization over this graph. Initial relevance is

$s[1] = 64$5

and the original objective combines adherence to $s[1] = 64$6, graph smoothness through the normalized Laplacian, and $s[1] = 64$7 regularization: $s[1] = 64$8 Because signed edges can make $s[1] = 64$9 non-PSD, GASim introduces diagonal correction

$s[2] = 1024$0

yielding a corrected Laplacian $s[2] = 1024$1 and a convex objective. The closed-form optimum solves a linear system, but practical retrieval uses the iterative propagation rule

$s[2] = 1024$2

after which the highest-scoring memories are selected. With sparse $s[2] = 1024$3, each iteration is dominated by sparse matrix–vector multiplication.

The design goal is to decouple retrieval from LLM calls. GOM makes no LLM calls to build or update the memory graph, compute edges or weights, or score memory relevance. In the reported 10k-agent, 30-step simulation, the core-agent stage fell from 316.33 minutes in HiSim to 19.30 minutes in GASim, a 16.39× speedup; total simulation time fell from 401.84 minutes to 40.43 minutes, a 9.94× speedup; and total tokens at 10k agents fell from 316,944 in the HiSim hybrid baseline to 61,771 in GASim, with the paper explicitly attributing part of that efficiency gain to GOM. On the LoCoMo benchmark, GOM achieved 71.56% overall accuracy, outperforming A-Mem, LangMem, Zep, Mem0, and Mem0g overall, and leading on Single Hop, Multi-Hop, and Temporal categories. In GASim ablation on the Politics dataset, replacing GOM with HiSim’s LLM-based memory worsened $s[2] = 1024$4Bias, $s[2] = 1024$5Div, Corr., and $s[2] = 1024$6, with correlation falling from 0.4261 to 0.2942.

3. Graph-driven hierarchical memory management in HyperOffload

In HyperOffload, Graph-Optimized Memory denotes compiler-assisted hierarchical memory management for SuperNode architectures. The central mechanism is to represent remote memory operations as explicit operators in the computation graph rather than as hidden runtime actions. Cache operators such as Prefetch, Store, and Detach are inserted into MindSpore’s static IR, lower to primitives such as H2R, R2H, R2D, D2R, and D2D, and are scheduled together with ordinary compute operations. This gives the compiler global visibility into tensor lifetimes, execution dependencies, and candidate transfer windows, enabling compile-time planning of where tensors reside across HBM, the shared memory pool, and possibly host memory (Liu et al., 31 Jan 2026).

HyperOffload’s Graph-Driven Execution-Order Optimization refines a legal topological order of the graph. For each independent cache operator $s[2] = 1024$7, the compiler finds the first consumer, enumerates feasible insertion positions that preserve dependencies, estimates transfer completion time $s[2] = 1024$8, estimates overlappable compute $s[2] = 1024$9, and selects the position minimizing a cost $s[3] = 4096$0 that trades off communication overlap against HBM residency. The stated purpose is to move prefetches into a just-in-time region: not so late that transfer latency is exposed, and not so early that prefetched data inflates peak device memory.

The framework applies this policy to activations, optimizer states, and KV caches. Activations with long forward-to-backward lifetimes are offloaded near the end of forward and prefetched before backward consumption. Optimizer states are treated as long-lived, infrequently accessed tensors and are prefetched while backward compute is still running. KV movement is represented explicitly in the decode graph, with relevant KV blocks prefetched ahead of attention use. The general heuristic is to offload large, infrequently used, or long-lived tensors and keep small or hot tensors in HBM.

Reported results emphasize both capacity and latency hiding. On representative LLM workloads, HyperOffload reduced peak device memory usage by up to 26% for inference while maintaining end-to-end performance. For DeepSeek-V3 + NSA, peak device memory dropped from 61.2 GB to 45.0 GB and maximum sequence length increased from 71k to 123k, about 1.73× longer. In a long-sequence run, memory defragmentation events dropped from 57 to 0, prefill latency fell from 129.33 s to 99.41 s, and end-to-end latency fell from 187.21 s to 161.41 s. Under low memory pressure, prefill latency changed by less than 1%, decode latency increased from 0.117 s to 0.146 s because of CPU-side sparse-block work, and overall end-to-end latency remained essentially unchanged at 177.373 s versus 177.109 s.

4. Graph-level execution and layout optimization in neural networks

A GOM-style technique for GNN inference appears as feature decomposition on edge devices. Instead of aggregating full $s[3] = 4096$1-dimensional node features in one pass, each $s[3] = 4096$2 is split into $s[3] = 4096$3 partitions, message passing is executed sequentially on each partition, partial aggregates are concatenated, and the update is applied once to the reconstructed result. The method is exact rather than a low-rank approximation: it reorganizes execution without changing model semantics. Its effect is to reduce the active feature width per aggregation pass from $s[3] = 4096$4 to $s[3] = 4096$5, improving cache locality and reducing intermediate working sets. Empirically, it yielded up to 3× speedup and up to 5× improvement in memory efficiency; on Reddit with GCN and hidden dimension 32, peak memory dropped from about 30 GB to about 6 GB, while cache behavior improved sharply, for example from 23.15% to 6.94% in L1 miss rate, from 43.49% to 5.09% in LLC miss rate, from 4.43% to 0.05% in TLB miss rate, and from 0.66 to 1.82 in IPC (Zhou et al., 2021).

ROAM addresses a different layer of the stack: graph-level training memory optimization on the computation DAG. It models operators as vertices and tensors as edges, optimizes valid execution order to minimize theoretical peak memory $s[3] = 4096$6, and optimizes memory layout to minimize actual peak buffer use $s[3] = 4096$7. The method exploits Independent Segments, models activation and temporary-buffer lifetimes across forward, backward, and optimizer phases, and uses a tree-based decomposition so that leaf subgraphs can be solved by ILP while the full graph is composed efficiently. Layout composition follows an activation-first discipline intended to suppress fragmentation, and the reported fragmentation is consistently below 1%, often exactly 0. Across evaluated models, ROAM reduced memory by 35.7% on average versus PyTorch, by 13.3% versus heuristics, and by 27.2% versus MODeL-Multi-Streaming, while achieving at least 53.6× speedup in optimization time and scaling to GPT2-XL graphs with more than 10,000 operators (Shu et al., 2023).

These two lines of work share a graph-centric optimization stance but operate on distinct objects. Feature decomposition treats the graph algorithm itself as fixed and reorganizes the feature dimension to better fit cache and memory limits. ROAM treats the computation graph as the optimization substrate and jointly changes operator order and tensor placement. One targets inference on resource-limited CPUs and Raspberry Pi systems; the other targets large-scale GPU training graphs. Their overlap lies in replacing memory-oblivious execution with graph-informed scheduling.

5. Trainable graph memory for LLM agents

A further use of GOM appears in trainable graph memory for LLM agents. The memory graph is a three-layer heterogeneous DAG

$s[3] = 4096$8

where query nodes $s[3] = 4096$9 store task instances and trajectories, transition-path nodes $s[4] = 2097152$0 store canonical decision paths in a finite state machine $s[4] = 2097152$1, and meta-cognition nodes $s[4] = 2097152$2 encode high-level strategic principles. Edges $s[4] = 2097152$3 and $s[4] = 2097152$4 carry trainable weights $s[4] = 2097152$5 and $s[4] = 2097152$6. Raw trajectories are grounded into FSM paths such as StrategyPlanning, ToolExecution, InformationAnalysis, DecisionMaking, DiagnosisHub, and related states, making stored experience structurally comparable across tasks (Xia et al., 11 Nov 2025).

The optimization target is not only relevance but empirical utility. For a new query $s[4] = 2097152$7, retrieval activates a subgraph of similar historical queries, and each meta-cognition receives score

$s[4] = 2097152$8

A softmax over these scores defines a strategy-selection policy $s[4] = 2097152$9. The system then runs guided and unguided trajectories, computes the reward gap

$B_i^i \le 4$0

and updates graph weights with the REINFORCE objective

$B_i^i \le 4$1

Selected meta-cognitions are verbalized and prepended to the prompt, so the graph acts as an explicit strategic prior during both inference and RL training.

The reported setting uses tool-augmented QA over seven datasets, with memory constructed only from HotpotQA. In inference-only evaluation, average Exact Match for Qwen3-8B rose from 0.334 with ITR to 0.365, and for Qwen3-4B from 0.279 to 0.351. In RL training, average Exact Match for Qwen3-8B rose from 0.395 in Search-R1 to 0.408, and for Qwen3-4B from 0.375 to 0.426; the latter exceeded the baseline Qwen3-8B score of 0.395. Ablation showed that disabling weight optimization caused a noticeable drop, especially on 2WikiMultiHopQA, and varying the number of retrieved meta-cognitions showed improvement from top$B_i^i \le 4$2 to top$B_i^i \le 4$3 before diminishing returns appeared.

6. Common principles, misconceptions, and open issues

Across these usages, GOM repeatedly denotes explicit graph exposure of relations that flat memory systems hide. In GASim, the graph links past messages by semantics and stance. In HyperOffload and ROAM, the graph exposes tensor lifetimes, dependencies, and legal reorderings. In feature decomposition for GNN inference, the graph computation is reorganized so that high-dimensional node states are processed in memory-friendly partitions. In trainable agent memory, the graph links episodic experience to procedural paths and strategic meta-cognition. In HBA, the graph is physically laid out to match multiple units of spatial locality. A plausible common abstraction is that graph structure is used to turn memory from a reactive storage problem into an optimization problem over dependencies, lifetimes, locality, or strategic relevance.

Several common misconceptions follow from the multiplicity of these definitions. GOM is not a single standardized architecture; it can be nonparametric, as in GASim; compiler-directed, as in HyperOffload; ILP-assisted, as in ROAM; REINFORCE-trained, as in trainable graph memory; or purely layout-oriented, as in HBA. It is not inherently tied to LLM prompting, nor inherently tied to hardware offloading. It is also not uniformly superior on every workload: GOM in GASim is slightly worse than Zep on LoCoMo open-domain questions; HyperOffload incurs about 25.5% slower decode latency under low memory pressure in one short-sequence setting because of CPU-side sparse-block work; feature decomposition revisits the edge set $B_i^i \le 4$4 times and requires manual partition tuning; ROAM addresses NP-hard subproblems by decomposition rather than exact global optimization; the trainable graph memory depends on a manually designed FSM and extra guided-versus-unguided rollouts; and HBA is fundamentally a batch layout transformation and does not fully address incremental dynamic updates (Zhou et al., 8 May 2026, Liu et al., 31 Jan 2026, Zhou et al., 2021, Shu et al., 2023, Xia et al., 11 Nov 2025, Roy, 2012).

The current literature therefore supports a broad but technically coherent view of GOM. In every instance, the graph is not merely an index; it is the locus of the optimization itself. What changes across domains is the optimized object: memories of messages, tensor transfers, operator schedules, feature partitions, strategic abstractions, or physical addresses. This suggests that future work on GOM is likely to continue along two axes already visible in the cited papers: richer graph models that capture more of the relevant dependency structure, and tighter optimization loops that connect graph state to measurable task utility, whether that utility is latency hiding, memory footprint, retrieval fidelity, or downstream reward.