---
title: 'StructMem: Hierarchical Memory for Conversational Agents'
url: https://www.emergentmind.com/topics/structmem
type: topic
---

# StructMem: Hierarchical Memory for Conversational Agents

StructMem is a structure-enriched hierarchical memory framework for long-horizon conversational agents. It is designed for settings in which memory must capture relationships between events rather than isolated facts, with the central memory unit being a temporally grounded relational event rather than a flat fact or a graph triplet. The framework combines event-level bindings, temporal anchoring, dual-perspective extraction, and periodic semantic consolidation to preserve local event structure while inducing cross-event connections without explicit graph construction. On \texttt{LoCoMo}, StructMem is reported to improve temporal reasoning and overall long-context question answering while using substantially fewer build tokens, API calls, and runtime than several prior memory systems [2604.21748].

## 1. Problem formulation and conceptual position

StructMem addresses persistent memory for long-term conversational agents that interact over many sessions and must answer questions requiring temporal dependencies, causal chains, interpersonal relations, and multi-hop connections across turns and sessions. The motivating benchmark context is \texttt{LoCoMo}, where conversations average about **588 turns** and **16,618 tokens** and span up to **32 sessions**. In this regime, the framework argues that long-horizon dialogue memory should not be modeled as a bag of disconnected facts, because facts are often meaningful only within an event context and many questions require reconstructing cross-event structure [2604.21748].

The framework is explicitly positioned between two prior paradigms. Flat memory is described as simple and efficient, but as storing facts or summaries as independent units, which weakens temporal reasoning and multi-hop question answering. Graph memory is described as preserving entities and relations explicitly, but as incurring expensive and fragile graph construction, cascading extraction pipelines, entity-resolution overhead, and rigid schemas. StructMem is proposed as a middle ground: it preserves event-level bindings and induces cross-event structure, but represents that structure in natural language plus temporal anchoring rather than as an explicit symbolic graph [2604.21748].

This design choice aligns with a broader line of work that treats memory organization itself as a first-class problem. For example, StructMemEval argues that long-term memory competence should be evaluated by an agent’s ability to organize knowledge into task-appropriate structures rather than merely retrieve facts, and MemIR argues that flat untyped storage causes provenance-role collapse by conflating evidence, retrieval cues, and claims [2602.11243] [2605.25869]. This suggests that StructMem belongs to a broader shift from flat archival memory toward explicitly organized memory substrates.

## 2. Memory hierarchy and representation

StructMem has a two-level memory organization. At the first level, it stores event-level memory entries extracted from each utterance. For each utterance \(m_i\), the system produces factual entries \(\Phi_i\) and relational entries \(\Psi_i\), and stores each extracted entry \(x\) together with its embedding \(\mathbf{e}_x\) and timestamp \(\tau_i\):

$$
\mathcal{M} \leftarrow
\bigcup_{i=1}^{N} \left\{ \langle x, \mathbf{e}_x, \tau_i \rangle \mid x \in \Phi_i \cup \Psi_i \right\}.
$$

This event-level store is the core episodic memory. It is lightweight and text-centric: the explicitly described data structures are entry text \(x\), embedding \(\mathbf{e}_x\), timestamp \(\tau\), the factual set \(\Phi_i\), the relational set \(\Psi_i\), the unconsolidated buffer \(\mathcal{M}_{buffer}\), retrieved semantic seeds \(\mathcal{S}_k\), reconstructed events \(E_\tau(x^*)\), and consolidated memory \(\mathcal{C}_{cons}\) [2604.21748].

At the second level, StructMem maintains cross-event synthesis memory. This is produced periodically rather than per event, and it functions as a higher-level abstraction over semantically related and temporally reconstructed event clusters. During inference, the framework retrieves from both levels. The appendix implementation details state that question answering retrieves **60 entries** and **5 synthesis memories**, so the effective inference-time memory is dual-source: atomic event entries plus synthesized consolidations [2604.21748].

A key representational property is that event-level bindings are preserved implicitly by shared timestamps rather than explicit edges. If two entries share \(\tau_i\), they belong to the same event-level unit. Cross-event relations are likewise not stored as graph edges; they are induced through semantic retrieval, timestamp-based event reconstruction, and natural-language synthesis. This gives StructMem a hierarchical structure without requiring explicit graph maintenance [2604.21748].

## 3. Dual perspectives and temporal anchoring

The phrase “temporally anchoring dual perspectives” refers to the combination of two operations. First, each utterance is processed from two perspectives: a factual perspective and a relational perspective. Second, all extracted entries are attached to the same originating timestamp. The dual-perspective extraction is defined as:

$$
\Phi_i \cup \Psi_i = \mathcal{L}( P_{fact} \| m_i ) \cup \mathcal{L}( P_{rel} \| m_i ),
$$

where \(P_{fact}\) is the factual extraction prompt, \(P_{rel}\) is the relational extraction prompt, and \(\mathcal{L}\) is the language model [2604.21748].

The factual extractor produces entries denoted

$$
\Phi_i = \{c_{i,1}, \ldots, c_{i,j}\},
$$

while the relational extractor produces

$$
\Psi_i = \{r_{i,1}, \ldots, r_{i,k}\}.
$$

The factual channel captures event content—what happened, who did what, what was stated. The relational channel captures interactional dynamics, causal influence, interpersonal relation, and temporal dependency. The framework’s claim is that factual entries alone do not preserve enough connective tissue for questions involving co-participation, influence, cause, or relative time expressions such as “last year” [2604.21748].

Temporal anchoring is then realized by assigning every entry extracted from utterance \(m_i\) the same timestamp \(\tau_i\), using the storage equation above. The timestamp is operational rather than merely metadata. It supports event reconstruction, chronological ordering, and grounded synthesis. The full event associated with a retrieved seed entry \(x^*\) is reconstructed by:

$$
{E}_{\tau}(x^*) = \{ x' \in \mathcal{M} \mid \tau(x') = \tau(x^*) \}.
$$

The appendix case study illustrates the effect with the question “When did Caroline and Melanie go to a pride festival together?” StructMem answers correctly because it stores not only factual mentions but also relational context and synthesis memory that make implicit co-participation explicit [2604.21748].

## 4. Semantic consolidation and retrieval

StructMem performs consolidation periodically rather than after every utterance. The reported implementation uses a **1 hour** threshold. Recent unconsolidated entries are placed in a buffer \(\mathcal{M}_{buffer}\), then sorted by timestamp:

$$
\mathcal{C}_{buf} = \text{Sort}_{\tau} \{ x \in \mathcal{M}_{buffer} \}.
$$

The buffered entries are concatenated into an aggregated query, embedded, and used to retrieve semantically similar historical seed entries. The implementation detail given in the appendix is **top-15** retrieval seeds for cross-event consolidation. For each retrieved seed \(x^*\), the system reconstructs the entire historical event by timestamp using \(E_\tau(x^*)\), then forms a cross-event context:

$$
\mathcal{C}_{cross} = \mathcal{C}_{buf} \cup \bigcup_{x^* \in \mathcal{S}_k} {E}_{\tau}(x^*).
$$

Finally, the language model synthesizes a consolidated memory:

$$
\mathcal{M} \leftarrow \mathcal{C}_{cons} = \mathcal{L}( P_{cons} \| \mathcal{C}_{cross} ).
$$

The paper describes this output as a complementary abstraction layer rather than a replacement for raw episodic memory [2604.21748].

This procedure is the mechanism by which StructMem induces cross-event links without explicit graphs. Semantic retrieval links buffered events to related historical seeds; timestamp reconstruction restores whole events around those seeds; synthesis then writes a natural-language summary that can express repeated preferences, causal progression, temporal continuity, or shared participation. The paper emphasizes that this is not generic window summarization. It is synthesis over semantically reconstructed event clusters [2604.21748].

At inference time, StructMem uses a dual-circuit retrieval procedure that combines retrieved atomic event entries and retrieved synthesis memories. The paper does not provide a formal inference-time ranking equation beyond semantic ranking by cosine similarity, and it does not specify a learned consolidation loss or merge objective. Consolidation remains a prompt-based LLM synthesis process rather than a trained optimization module [2604.21748].

## 5. Experimental results, fidelity, and efficiency

StructMem is evaluated on the \texttt{LoCoMo} question answering task, using **10 long-term conversations** with average **588 turns**, average **16,618 tokens**, and up to **32 sessions**. The evaluation categories are **Single-hop** (**841 questions**), **Multi-hop** (**282 questions**), **Temporal** (**321 questions**), and **Open-domain** (**96 questions**). The experimental setup uses `gpt-4o-mini` as the backbone LLM and `text-embedding-3-small` as the embedding model. Evaluation is performed with LLM-as-a-judge, and robustness is checked across `gpt-4o-mini`, `Qwen2.5-32B-Instruct`, and `DeepSeek-V3.2`, with reported inter-judge agreement of Fleiss’ \(\kappa = 0.8341\), pairwise Pearson \(r > 0.81\), and \(p < 10^{-300}\) [2604.21748].

On the main benchmark table, StructMem reports **76.82** overall, with **68.77** on multi-hop, **46.88** on open-domain, **81.09** on single-hop, and **81.62** on temporal reasoning. These results place it above the reported overall scores of OpenAI (**71.82**), FullContext (**73.83**), Mem0 (**66.88**), Zep (**75.14**), and Memobase (**75.78**). It is especially strong on temporal reasoning, where only Memobase is higher at **85.05**. In the paradigm comparison, StructMem exceeds the reported “Flat Memory,” “Graph Memory,” and “w/o Cross-Event” variants on both single-session and temporal reasoning, supporting the claim that cross-event consolidation adds value beyond event-level structure alone [2604.21748].

The efficiency profile is one of the framework’s strongest reported properties. StructMem uses **1.501M** build input tokens and **0.436M** output tokens, for a total of **1.937M** build tokens, with **1056** API calls and **22854 s** runtime. The paper highlights the contrast with graph-heavy alternatives such as Mem0\(^g\), which reports **35.825M** total build tokens, **53,514** API calls, and **115670 s** runtime. The argument is that graph memory requires multiple cascading LLM operations per event—entity extraction, entity deduplication, relation extraction, and relation deduplication—whereas StructMem shifts cross-event organization into periodic batched consolidation [2604.21748].

The paper also reports fidelity audits. Event-level extraction has an average hallucination rate of **2.36%** across conversations. For cross-event consolidation, the default constrained setting reports spurious link rates of **0.61%** under the GPT judge, **3.41%** under the Qwen judge, and **3.63%** under the DeepSeek judge. In an unconstrained variant, the corresponding hallucinated cross-event link rates rise to **7.45%**, **20.00%**, and **15.14%**, respectively. The authors interpret this as evidence that temporal and grounding constraints in the synthesis prompt materially improve faithfulness [2604.21748].

## 6. Broader significance, limitations, and relation to later structured-memory work

StructMem’s broader significance lies in its reformulation of conversational memory around the temporally grounded relational event. Rather than choosing between flat retrieval and explicit graph construction, it preserves factual and relational content together at event level and then induces higher-order connections through periodic synthesis. A plausible implication is that its main contribution is not merely improved recall, but a specific memory unit and update schedule: event-local binding first, cross-event abstraction second [2604.21748].

The paper also notes limitations. The quality of dual-perspective extraction depends heavily on prompt quality, and the framework currently lacks an explicit mechanism for revising outdated facts, resolving contradictions, or updating user preferences over time. Open-domain performance also lags behind some competing systems, notably Zep. These limitations place StructMem within an ongoing research agenda rather than as a complete solution to long-term agent memory [2604.21748].

Subsequent structured-memory work makes the surrounding design space more explicit. StructMemEval argues that many benchmarks reward retrieval more than structure and that memory agents often fail because they do not recognize the appropriate memory schema unless prompted [2602.11243]. Memora proposes a harmonic memory representation with primary abstractions, concrete memory values, and cue anchors, and formally shows that standard RAG and KG-based memory can emerge as special cases [2602.03315]. MemIR argues for typed memory atoms that separate page atoms, span atoms, handle atoms, time atoms, pivot atoms, and claim atoms, with only claims treated as truth-bearing and all retrieval routed through claim-centered bundles [2605.25869]. Together, these works suggest that StructMem occupies an influential position in a broader transition from flat conversational memory toward structured, hierarchical, and provenance-sensitive memory architectures.

In embodied and interactive domains, related ideas reappear under different forms. eMEM proposes a hybrid graph-based spatio-temporal memory with observation, episode, gist, and entity nodes, plus SQL, HNSW, and R-tree indexing for embodied agents [2606.03374]. AndroTMem introduces Anchored State Memory, where long-horizon GUI interaction trajectories are converted into sparse typed anchors with causal links [2603.18429]. MemCompiler reframes memory usage as state-conditioned memory compilation over a structured Brief State rather than static memory injection [2605.07594]. These systems target different settings, but they reinforce a shared lesson that StructMem helped crystallize for conversational agents: long-horizon performance depends not only on storing more context, but on preserving and exploiting structure within memory.

Source: https://www.emergentmind.com/topics/structmem