HeLa-Mem: Bio-Inspired Memory for LLMs

• HeLa-Mem is a bio-inspired memory architecture that models memory as a dynamic graph with episodic and semantic components, capturing associative human-like recall.
• It employs Hebbian learning, reflective consolidation, and dual-path retrieval to integrate association, consolidation, and spreading activation in LLMs.
• Empirical tests on benchmarks like LoCoMo and LongMemEval-S show enhanced performance and token efficiency, despite challenges such as cold start limitations.

HeLa-Mem is a bio-inspired memory architecture for LLM agents that models memory as a dynamic graph with Hebbian learning dynamics and a dual-level organization composed of an episodic memory graph and a semantic memory store populated via Hebbian Distillation. It is motivated by the claim that fixed context windows cannot preserve coherence across extended interactions and that existing memory systems, which store conversation history as unstructured embedding vectors and retrieve by semantic similarity, fail to capture the associative structure of human memory. Within this formulation, HeLa-Mem explicitly integrates association, consolidation, and spreading activation, and on LoCoMo it reports superior performance across four question categories while using approximately 1,010 tokens on average (Zhu et al., 18 Apr 2026).

1. Motivation and conceptual basis

HeLa-Mem is introduced against a specific diagnosis of long-term memory failure in LLM agents: “Long-term memory is a critical challenge for LLM agents, as fixed context windows cannot preserve coherence across extended interactions.” The critique is not only about token limits. The paper argues that the dominant paradigm treats storage and retrieval as static and separate, representing interaction history as unstructured embedding vectors and recovering items through semantic similarity alone. According to the paper, this “fails to capture the associative structure of human memory, wherein related experiences progressively strengthen interconnections through repeated co-activation” (Zhu et al., 18 Apr 2026).

The conceptual basis is explicitly drawn from cognitive neuroscience. Three mechanisms are identified as central to biological memory: association, consolidation, and spreading activation. The paper states that these mechanisms remain “largely absent in current research.” In that framing, human memory is not modeled as an isolated database of facts, but as “a dynamic system where connections are continuously reorganized by experience.” The stated consequence for LLM agents is that purely similarity-based memory systems may remember facts yet lack “the ‘continuity’ of a developing relationship” (Zhu et al., 18 Apr 2026).

A common misconception is that HeLa-Mem is simply another retrieval layer over embeddings. The paper’s formulation is narrower and more specific. Semantic similarity remains part of the retrieval pipeline, but it is supplemented by learned associations encoded as weighted graph edges and by a semantic store distilled from graph structure. This suggests that the proposal is not an abandonment of vector retrieval, but a reorganization of memory around an episodic-semantic distinction.

2. Dual-level memory architecture

The framework consists of three modules: Hebbian Association for dynamic graph construction, Reflective Consolidation for semantic knowledge distillation, and Retrieval and Response using a Dual-Path strategy. These are arranged as a pipeline in which conversation turns enter an episodic graph, highly connected substructures are periodically distilled into semantic memory, and inference retrieves from both stores (Zhu et al., 18 Apr 2026).

The episodic memory graph stores conversation turns as nodes in a weighted graph. Each node contains the original text, a dense embedding, the timestamp, extracted keywords, and the speaker role. Edges represent associative connections, and their weights encode the learned strength of association. Initially, consecutive turns are connected with small weights; these weights then evolve through Hebbian learning. The node fields are specified as text, embedding $e_i$, time $t_i$, keywords, and role, while each edge $(i,j)$ has weight $w_{ij}$.

The semantic memory store is populated through Hebbian Distillation. The paper states that hub-centered clusters in the episodic graph are distilled into semantic records with traceable evidence links to their source turns. The resulting store includes three categories: User Model, Factual Memory, and Agent Knowledge. User Model contains stable characteristics with confidence and evidence; Factual Memory contains extracted facts with absolute timestamps; Agent Knowledge contains persona, preferences, and behavior patterns.

Component	Representation	Contents
Episodic Memory Graph	Weighted graph of conversation turns	text, dense embedding, timestamp, keywords, role
Semantic Memory Store	Distilled semantic records	User Model, Factual Memory, Agent Knowledge
Reflective Agent	Consolidation mechanism	hub identification, distillation, evidence linking

The architectural significance lies in the explicit mirroring of the episodic-semantic distinction in human cognition. The episodic graph stores detailed, time-stamped experiences; the semantic store contains stable, distilled, reusable knowledge. A plausible implication is that the design seeks to preserve both high-fidelity local traces and higher-level abstractions without forcing both burdens onto a single vector store.

3. Association, consolidation, and retrieval dynamics

The online phase begins when a new turn arrives. It is encoded as a node and connected to temporally adjacent turns with small initial weights. During later retrieval or co-activation events, the edges between co-activated nodes are updated by a Hebbian rule with decay. The stated objective is to strengthen connections among memories that are activated together, thereby capturing latent context that may not be recoverable by semantic similarity alone (Zhu et al., 18 Apr 2026).

Reflective Consolidation is the second major dynamic. The Reflective Agent “monitors the graph’s structural evolution,” identifies hub nodes whose accumulated high total edge weight exceeds a threshold, retrieves those hubs together with strongly connected neighbors, and applies Hebbian Distillation to consolidate them into stable semantic entries. The paper also describes this as conceptually akin to sleep-based consolidation. At the same stage, adaptive forgetting prunes low-weight, dormant, and unused nodes.

Retrieval uses a Dual-Path strategy. For a query, base activation scores are computed from embedding similarity, keyword match, and temporal decay. Spreading activation then propagates through Hebbian edges, surfacing associatively linked nodes that may be semantically distant from the query. Final ranking merges a top-$k$ base list with up to $m$ additional “flipped” nodes promoted through spreading activation. Semantic memory is retrieved in parallel and merged with episodic context before response generation. The response-generation prompt instructs the model to answer concisely using episodic and semantic contexts, with date formatting such as “15 July 2023” and with output restricted to answer content.

The paper provides a qualitative case study in which a person entity is linked to a meeting location through a learned Hebbian edge of 0.52. Pure semantic similarity misses the relevant turn, but dual-path retrieval recovers it. This case is presented as evidence that associatively relevant items can be semantically weak yet still operationally necessary for multi-hop memory questions.

4. Formalization and reported hyperparameters

The conceptual inspiration is grounded in a standard Hebbian update from neuroscience:

$\Delta w_{ij} = \eta \cdot x_i \cdot x_j,$

where $x_i$ and $x_j$ are activation states and $\eta$ is the learning rate (Zhu et al., 18 Apr 2026).

For HeLa-Mem itself, the paper states that edge weights strengthen when memories are co-activated during retrieval and gives a malformed LaTeX equation for the update. The accompanying symbol definitions are clear: $t_i$0 and $t_i$1 are episodic nodes, $t_i$2 is the edge weight at time $t_i$3, $t_i$4 is the decay rate, $t_i$5 is the learning rate, $t_i$6 is the co-activation indicator, and $t_i$7 is the set of currently retrieved or activated nodes. The paper explicitly notes that there is no explicit normalization or inhibition term beyond multiplicative decay.

Hub identification is defined by total adjacent weight:

$t_i$8

Here, $t_i$9 is the neighbor set of node $(i,j)$0, $(i,j)$1 is total adjacent weight, and $(i,j)$2 is a positive threshold. No centrality or clustering metrics are formally defined beyond this criterion.

Base activation is defined as

$(i,j)$3

where $(i,j)$4 denotes cosine similarity between the query embedding $(i,j)$5 and node embedding $(i,j)$6, $(i,j)$7 is the temporal decay factor, and $(i,j)$8 controls the keyword-match bonus.

Spreading activation is defined as

$(i,j)$9

where $w_{ij}$0 controls propagation strength. The paper also reports a “spreading threshold” $w_{ij}$1, but does not provide a formal thresholding equation.

The main experimental hyperparameters are reported as: time decay constant $w_{ij}$2 days; episodic retrieval $w_{ij}$3; semantic retrieval $w_{ij}$4; Hebbian learning rate $w_{ij}$5; edge decay rate $w_{ij}$6; spreading activation strength $w_{ij}$7; and spreading threshold $w_{ij}$8. In the appendix setting for LongMemEval-S, the best reported configuration is $w_{ij}$9, $k$0, $k$1, $k$2, keyword weight $k$3, max flipped items $k$4, episodic top-15, and semantic top-5.

5. Empirical results and ablations

The principal evaluation is conducted on LoCoMo, which is described as designed for assessing long-term conversational memory capabilities, with ultra-long dialogues averaging 300 turns and about 9K tokens per conversation. The four evaluation categories are Multi Hop, Temporal, Open Domain, and Single Hop, and the reported metrics are F1 and BLEU-1 (Zhu et al., 18 Apr 2026).

The baselines listed are LoCoMo (Native), ReadAgent, MemoryBank, MemGPT, A-Mem, Mem0, LightMem, and MemoryOS. Against these systems, HeLa-Mem is reported to achieve strong performance with fewer tokens. For GPT-4o-mini, the paper reports Multi-hop 40.14/31.26, Temporal 47.29/41.28, Open 29.70/23.45, Single-hop 51.89/46.25, with token length approximately 1,010. For GPT-4o, it reports Multi-hop 39.12/29.82, Temporal 50.79/44.54, Open 24.38/19.24, Single-hop 49.69/44.08, with token length approximately 1,036. For Qwen2.5-14b, the corresponding values are 36.59/27.02, 36.08/29.91, 24.22/20.23, and 49.95/45.15, with token length approximately 944. For Qwen2.5-3b, the values are 20.12/14.59, 24.79/21.35, 12.24/10.24, and 29.51/25.91, with token length approximately 1,072.

The paper summarizes token efficiency by stating that HeLa-Mem achieves these results using only approximately 1,010 tokens on average, and Figure 1 places it in the “upper-left ideal region.” Across backbones, it reports the best Average Rank of 1.25, compared with 2.25 for MemoryOS.

Ablation results on GPT-4o-mini isolate the contribution of the major components. The full system reports Avg F1 34.74. Removing the Reflective Agent reduces Avg F1 to 29.87 and produces the largest drop, including Multi-hop 30.17/22.38. Removing spreading activation yields Avg F1 32.19, including Multi-hop 33.88/25.57. Removing forgetting has a smaller effect, with Avg F1 34.28. The paper interprets this as evidence that Reflective Consolidation and spreading activation are central to the method, while forgetting has only minor impact at LoCoMo scale.

The appendix reports an additional benchmark on LongMemEval-S. There, HeLa-Mem achieves overall accuracy of 65.40%, exceeding A-MEM at 62.60, NaiveRAG at 61.00, FullText at 56.80, Mem0 at 53.61, and MemoryOS at 44.80. Category-wise, it reports Temporal 50.38, Multi-Session 57.14, Knowledge-Update 78.21, and Single 78.85. The paper states that this is the best overall accuracy and the best performance on the three reasoning-intensive categories.

The evaluation has notable limits. The paper does not report confidence intervals or hypothesis tests. This implies that the reported ranking is descriptive rather than accompanied by formal statistical significance analysis.

6. Practical integration, limitations, and open issues

The implementation guidance centers on maintaining an episodic graph in which each new turn yields a node with text, embedding, timestamp, keywords, and role; initializing small edges to adjacent turns; and storing edge weights as floating-point values. The semantic store may be maintained as structured records with type, content, confidence, and evidence links. The reported supported backbones are GPT-4o-mini, GPT-4o, Qwen2.5-14b, and Qwen2.5-3b. Code is available at the GitHub repository linked by the paper, but setup instructions are not included in the paper itself (Zhu et al., 18 Apr 2026).

The paper also states several absences explicitly. It does not provide formal equations for consolidation schedules, inhibition terms, normalization, or spreading thresholds. It does not provide pseudocode for per-turn updates, consolidation, or retrieval. It does not specify the embedding model, graph storage library, vector database, or computational complexity and storage overhead. These omissions are consequential for reproduction because the method’s high-level flow is clear, but the engineering realization is underdetermined.

The principal limitation identified by the authors is cold start: Hebbian weights require sufficient interaction history to accumulate, so the benefits of associative retrieval are less pronounced in early conversation stages. A proposed remedy is initializing Hebbian edges using semantic similarity as a prior. Another limitation is dependence on LLM quality. The quality of both Semantic Memory and Hub Detection relies on the underlying model, and hallucinations or reasoning errors during distillation could propagate into long-term storage.

Ethical considerations are presented in restricted form. The paper notes bias risks in distilled memories, states that no private data were collected, and uses LoCoMo for evaluation. A broader controversy is therefore not about privacy claims in the benchmark setting, but about error persistence: once a distillation step writes semantic entries with evidence links, inaccurate abstractions could become stable retrieval targets.

In aggregate, HeLa-Mem defines long-term agent memory as an interaction between a dynamically rewired episodic graph and a distilled semantic store. Its distinctive claim is not merely that more memory helps, but that memory quality depends on explicitly modeling association, consolidation, and spreading activation. The reported gains on LoCoMo and LongMemEval-S support that claim at the benchmark level, while the stated limitations indicate that early-stage behavior, distillation reliability, and underspecified implementation details remain open technical issues (Zhu et al., 18 Apr 2026).