Tree-Structured Memory in Neural Architectures

• Tree-structured memory is a hierarchical mechanism that organizes data in tree formats, where each parent node aggregates and filters multiple child signals.
• It leverages specialized gating in recurrent and convolutional models to combine structured inputs, yielding significant performance gains on language and image tasks.
• The approach extends to external memory for online learning and agent memory, enabling efficient retrieval, localized updates, and improved long-horizon context integration.

Tree-structured memory denotes a family of memory mechanisms in which state, storage, or access is organized over a tree rather than a linear chain or a flat collection. In the literature, the term spans several distinct constructions: recurrent neural architectures that place gated memory cells at tree nodes; convolutional memories whose recurrence follows branching image structures; external memories for agents and conversational systems that store episodes, summaries, or task state in hierarchical trees; and systems-level mechanisms in which persistent storage or transformer KV state is managed according to tree traversal or tree search. The common structural property is that a parent node receives, summarizes, filters, or routes information from multiple children rather than from a single predecessor (Tai et al., 2015).

1. Foundational recurrent formulations

In the recurrent-neural lineage, tree-structured memory emerged as a direct generalization of chain LSTM memory to hierarchical inputs. In S-LSTM, each node of an ordered binary tree stores a hidden state $h_t$ and a memory cell $c_t$, and the parent combines left and right child memories with separate forget gates: $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$ followed by

$h_t = o_t \otimes \tanh(c_t).$

The model was introduced for semantic composition over parse trees, where memory must follow a partial order rather than a total order (Zhu et al., 2015).

Tree-LSTM generalized this idea further into two standard forms. In the Child-Sum Tree-LSTM, suitable for variable-arity trees, a node computes

$$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$

In the $N$-ary Tree-LSTM, suitable for ordered trees such as binarized constituency parses, child positions receive distinct parameters and the forget gate for one child may depend on the hidden states of the others through off-diagonal terms $U^{(f)}_{k\ell}$ (Tai et al., 2015).

These formulations established the canonical meaning of tree-structured memory in deep learning: a memory cell at each node summarizes the subtree beneath it, while child-specific gates determine how much of each descendant trace survives upward. Empirically, the gains were substantial on syntax-sensitive language tasks. S-LSTM reached 48.0 roots / 81.9 phrases on 5-class sentiment classification and 43.5 with root labels only, compared with 29.1 for RNTN under the same sparse-supervision setting (Zhu et al., 2015). Tree-LSTM reported Pearson $r = 0.8676$, Spearman $\rho = 0.8083$, and MSE $= 0.2532$ on SICK, and up to 51.0 fine-grained and 88.0 binary accuracy on SST (Tai et al., 2015). These results fixed the basic intuition that when the data-generating structure is hierarchical, the memory topology itself can be hierarchical.

2. Spatial tree memory and hierarchical image analysis

A second line of work replaced vector-valued tree memory with spatial feature maps. Tree-structured ConvLSTM retains the LSTM cell-and-gate mechanism, but every hidden state and cell state is a convolutional tensor rather than a vector. For a tree node $c_t$0 with children $c_t$1, the model first aggregates child hidden states,

$c_t$2

then computes input, output, and candidate memory convolutionally, while assigning a separate forget gate to each child,

$c_t$3

and updates the parent cell as

$c_t$4

The hidden state remains

$c_t$5

This formulation was motivated by branching image data such as human body-part kinematic trees, vessel trees, airway trees, and coronary artery trees in 3D CTA (Kong et al., 2019).

The decisive change is that memory is simultaneously branch-aware and spatially structured. In standard ConvLSTM there is one predecessor cell; in tree-structured ConvLSTM the parent receives multiple child memories and modulates each with its own forget gate. Because gates and memories are feature maps, the retained information preserves within-patch topology rather than collapsing it into a single semantic vector. The architecture was inserted between encoder and decoder in an attention FCN segmentation pipeline, so encoder features first entered the tree memory module and only then the decoder reconstructed masks (Kong et al., 2019).

The empirical evidence tied the performance gain directly to the memory topology. On Tree-Moving-MNIST, TreeCLSTM achieved 13.5% overall classification error, compared with 17.4% for CNN, 19.6% for sequential CLSTM, and 17.6% for TreeLSTM. On coronary artery segmentation, pooled Dice was 0.8678 for TreeCLSTM versus 0.8591 for sequential ConvLSTM, and the bifurcation-focused Dice was 0.8438, compared with 0.8120 for sequential CLSTM and 0.7806 for DenseVoxNet (Kong et al., 2019). The bifurcation result is particularly significant because branch interaction is exactly where a tree-structured memory should matter most.

3. Reading, supervising, and interpreting subtree memory

Tree-structured memory is not only a write mechanism; it also requires a read mechanism and an adequate supervision signal for internal nodes. This became explicit in work on Japanese sentiment classification, which used a Binary Tree-LSTM over constituency parse trees together with an attention mechanism over subtree representations. At each node $c_t$6, the classifier computes

$c_t$7

with

$c_t$8

so the final prediction depends on both the usual root representation $c_t$9 and an attention-weighted readout over subtree memories (Miyazaki et al., 2017).

The same paper paired this read mechanism with distant supervision from polar dictionaries. Phrase or word spans matching lexicon entries were assigned hard sentiment labels, so the loss was no longer restricted to sentence roots. This addressed a concrete weakness of tree memory under sentence-only supervision: internal phrase states may exist architecturally but remain poorly trained unless the model has a way either to read them directly or to supervise them locally (Miyazaki et al., 2017).

The ablations showed that tree structure alone was insufficient in that setting. On Japanese sentiment classification, plain Tree-LSTM achieved 0.709, Tree-LSTM with attention 0.810, Tree-LSTM with dictionary supervision 0.829, and Tree-LSTM with both attention and dictionary supervision 0.844, compared with 0.826 for Tree-CRF. On English SST with sentence-only labels, Tree-LSTM rose from 43.52 to 44.97 with attention, while dictionary supervision was not consistently helpful (Miyazaki et al., 2017). This suggests that tree-structured memory has two separable problems: how memory is composed up the tree, and how useful internal memories are exposed and trained.

4. External hierarchical memory for online learning and LLMs

Outside recurrent architectures, tree-structured memory has also been implemented as an explicit external memory store. Eigen Memory Trees store exact experiences $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$0 at leaf nodes and use internal routers defined by an approximate top principal component together with a median threshold. Querying follows a single root-to-leaf path using

$$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$1

and then scores memories inside the reached leaf with a global self-consistent scorer. The resulting lookup cost is effectively $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$2 for fixed leaf capacity. Empirically, EMT beat CMT on 177 datasets while CMT beat EMT on 11, and the hybrid PEMT-CB beat the parametric baseline on 110 datasets while underperforming on only 8 (Rucker et al., 2022).

In LLM memory systems, the 2024 MemTree work defined each node as

$$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$3

with textual content $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$4, embedding $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$5, parent $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$6, children $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$7, and depth $$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$8. New information is inserted online by descending through children whose cosine similarity exceeds a depth-adaptive threshold

$$c_t = f_t^L \otimes c_{t-1}^L + f_t^R \otimes c_{t-1}^R + i_t \otimes \tanh(x_t),$$9

with reported settings $h_t = o_t \otimes \tanh(c_t).$0 and $h_t = o_t \otimes \tanh(c_t).$1. Parent nodes are rewritten by LLM-based aggregation and then re-embedded. Retrieval in the main method is collapsed tree retrieval: the query embedding is compared against all node embeddings, allowing the model to match either abstract internal nodes or detailed leaves (Rezazadeh et al., 2024).

That architecture was explicitly designed to avoid flat memory tables of isolated entries. The hierarchy is not only an index; it is a memory formation mechanism that creates broad semantic nodes near the root and specific details at depth. On MSC-E, MemTree reached 82.5 overall accuracy versus 80.7 for MemoryStream; on QuALITY it achieved 59.8 versus 43.8 for MemoryStream; and on MultiHop RAG it reached 80.5 overall, with 68.4 on temporal questions (Rezazadeh et al., 2024). This suggests that external tree memory becomes most useful when the workload requires both online updates and access at multiple abstraction levels.

5. Temporal and structured agent memory

A major contemporary theme is that the most important tree structure in long-horizon agents may be temporal rather than purely semantic. Semantic XPath assumes conversational memory is a rooted tree

$h_t = o_t \otimes \tanh(c_t).$2

and executes queries over weighted node sets $h_t = o_t \otimes \tanh(c_t).$3. Structural navigation uses XPath-like axes such as / and //, while semantic relevance updates weights multiplicatively: $h_t = o_t \otimes \tanh(c_t).$4 The language also supports aggregation over descendant evidence, enabling queries such as retrieving the day most associated with conference sessions. In the reported evaluation, Semantic XPath improved performance over flat-RAG baselines by 176.7% while using only 9.1% of the tokens required by in-context memory (Liu et al., 1 Mar 2026).

SegTreeMem made temporal order itself the organizing principle. It represents conversation history as a rooted segment tree whose nodes correspond to contiguous utterance intervals

$h_t = o_t \otimes \tanh(c_t).$5

and updates the tree online through a rightmost-frontier rule. Retrieval does not simply flatten the tree; it propagates relevance scores over tree edges: $h_t = o_t \otimes \tanh(c_t).$6 On LoCoMo, LongMemEval-MAB, and RealMem, the bottom-up variant scored 0.639, 0.647, and 0.621 LLM-judge accuracy, and temporal-order permutation caused markedly larger drops than in a non-temporal similarity-clustering tree, including $h_t = o_t \otimes \tanh(c_t).$7 on LoCoMo and $h_t = o_t \otimes \tanh(c_t).$8 on RealMem (Liu et al., 3 Jun 2026). The direct implication is that chronological organization is not incidental to performance.

Hybrid systems combined trees with graphs rather than choosing one structure exclusively. H-Mem builds a temporal and semantic tree together with a knowledge graph; its tree uses four levels—day, week, month, and year—with default consolidation thresholds $h_t = o_t \otimes \tanh(c_t).$9, $$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$0, and $$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$1. On LoCoMo, the full model reached F1 $$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$2, Acc. $$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$3, while w/o tree fell to 49.76 / 82.73, the largest ablation drop reported (Yu et al., 15 May 2026). MemForest similarly treats memory as a write-efficient temporal data-management problem and uses a scope-local MemTree whose balanced $$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$4-ary height is

$$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$5

Its post-extraction maintenance touches only affected tree paths, yielding $$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$6 dependency depth after extraction; on LongMemEval-S it achieved 79.8% pass@1 while sustaining memory construction throughput approximately 6x higher than EverMemOS (Chen et al., 16 May 2026).

Taken together, these systems shift the meaning of tree-structured memory away from “tree as a semantic hierarchy” alone. The tree is increasingly used to preserve temporal locality, maintain multiple versions or episodes, and localize updates so that new information rewrites only the relevant branch rather than a global profile. This suggests that in agent memory, tree structure functions as both a representation of state evolution and a control surface for efficient retrieval.

6. Systems interpretations, applications, and conceptual boundaries

The phrase also appears in lower-level systems work where “memory” means persistent storage or runtime state. An embedded B-tree implementation stored nodes as fixed-size pages on external storage and used only two page buffers in SRAM, with a total RAM footprint of about 1.5 KB for $$\tilde{h}_j = \sum_{k \in C(j)} h_k, \qquad c_j = i_j \odot u_j + \sum_{k \in C(j)} f_{jk} \odot c_k, \qquad h_j = o_j \odot \tanh(c_j).$$7 buffers and operation on devices with as little as 4 KB RAM (Ould-Khessal et al., 2023). A separate hardware-layout study treated tree memory as a physical node-ordering problem and reported performance improvements of up to 95% as offline optimization and 75% as online optimization by reordering tree nodes to match memory characteristics and traversal paths (Biebert et al., 29 Jan 2025). In transformer inference, ArborKV managed tree-search KV cache as a recoverable tree-organized working set, using a tree-aware allocation policy and lazy rehydration; it reported up to ~4x peak KV-memory reduction while preserving near-full-retention accuracy (Chen et al., 21 May 2026). In long-term tracking, RTracker used a Positive-Negative tree-structured memory with a root, positive branch, and negative branch, each branch capped at 10 nodes; on LaSOT it recovered about 80% of lost targets, compared with 63% for MixViT (Huang et al., 2024).

At the same time, several papers are relevant precisely because they are not strict instances of tree-structured memory. D-SMART couples a dynamic OWL-compliant knowledge graph with a reasoning tree; its own description is graph-structured memory plus tree-structured reasoning, not tree-structured memory in the strict sense (Lei et al., 15 Oct 2025). TreeMem uses a training-time rollout tree to assign agent-specific credit in a builder–summarizer–retrieval memory pipeline; the central novelty is tree-based credit assignment rather than a tree-shaped storage substrate (Mao et al., 6 May 2026). TALM is a tree-structured multi-agent reasoning framework with a shared vector-based long-term memory; the tree organizes task decomposition and localized re-reasoning, while memory itself is global and vector-indexed (Shen et al., 27 Oct 2025).

These boundary cases clarify a recurring misconception. A system is not tree-structured memory merely because it reasons over a tree, trains with a tree, or traverses a tree-shaped workflow. The stricter usage reserves the term for cases where the memory state, the persistent storage layout, or the retrieval/update substrate is itself organized as a tree. The broader literature nevertheless suggests three persistent tensions: tree versus graph expressivity, localized update efficiency versus cross-branch interaction, and explicit structural bias versus semantically learned organization.