---
title: Neural Reasoning Memory
url: https://www.emergentmind.com/topics/reasoning-memory
type: topic
---

# Neural Reasoning Memory

Reasoning memory is the class of mechanisms by which a neural model or agent stores, updates, retrieves, and composes remembered state in support of multi-step inference. In the foundational formulation, reasoning is not merely computation but computation over remembered state: answering questions from stories, conducting dialogs, and executing multi-step algorithms all require a process that transforms representations and a memory that holds intermediate state and long-term context [1702.06186]. Across the literature, the term has expanded from differentiable external memory in Memory Networks and Neural Turing Machines to retrieval-augmented commonsense inference, executive and episodic memory for long-horizon agents, event-centric and neuro-symbolic memory graphs, and inference-time memory layers that reduce token and latency costs while preserving reasoning quality [1801.04622, 2601.08079, 2602.13530, 2511.12987].

## 1. Foundational formulation

The earliest neural treatment of reasoning memory in this corpus is architectural rather than symbolic. A simple recurrent model updates a hidden state by
$$
h_t = f(W_x x_t + W_h h_{t-1} + b),
$$
thereby compressing the entire past into a single state vector. The associated backpropagation signal multiplies Jacobians across time, so gradients tend to vanish or explode; this makes long-term credit assignment fragile, especially when the task requires remembering facts from many time steps earlier. LSTMs mitigate this by introducing gated cell dynamics,
$$
i_t = \sigma(W_i x_t + U_i h_{t-1} + b_i),\quad
f_t = \sigma(W_f x_t + U_f h_{t-1} + b_f),\quad
o_t = \sigma(W_o x_t + U_o h_{t-1} + b_o),
$$
$$
g_t = \tanh(W_g x_t + U_g h_{t-1} + b_g),\quad
c_t = f_t \odot c_{t-1} + i_t \odot g_t,\quad
h_t = o_t \odot \tanh(c_t),
$$
so that when $f_t \approx 1$ and $i_t \approx 0$, the cell behaves like a “perfect integrator.” Even so, the model still compresses all needed facts into a fixed-size state, which becomes inadequate when the task demands precise, compartmentalized memory over many steps [1702.06186].

Attention reframes memory access as differentiable retrieval rather than compression alone. Given a query $q$, keys $k_i$, and values $v_i$, dot-product attention computes
$$
\alpha_i = \operatorname{softmax}\!\left(\frac{q^\top k_i}{\sqrt{d_k}}\right),\quad
r = \sum_i \alpha_i\, v_i.
$$
This turns retrieval into a smooth lookup, allowing gradients to flow into both the query and memory contents. The core conceptual shift is the separation between “where information is kept” and “how it is processed,” which made later memory architectures possible [1702.06186].

A later line of work makes the same point from a different angle: reasoning-in-a-haystack tasks fail when models must search long, distractor-heavy contexts with only implicit internal state. MemReasoner therefore places explicit episodic memory outside the decoder, learns temporal order over fact latents with a bidirectional GRU, and performs iterative hops that update the query representation, yielding strong generalization under hard distractors, soft distractors, answer remapping, and task transfer with none-to-weak supporting fact supervision [2503.07903].

## 2. Differentiable external memory and retrieval-augmented reasoning

Memory Networks and Neural Turing Machines established the canonical external-memory paradigms. Memory Networks were defined with four components—I (input mapping), G (generalization), O (output or inference), and R (response)—and later End-to-End Memory Networks replaced hard supporting-fact selection with differentiable attention. In the multi-hop variant, discrete inputs $x_i$ and a query $q$ are embedded, attention is computed by
$$
p_i^{(1)} = \operatorname{softmax}(u^\top m_i),\quad
o^{(1)} = \sum_i p_i^{(1)} c_i,\quad
u^{(2)} = u + o^{(1)},
$$
and after $H$ hops the answer is produced as
$$
a = \operatorname{softmax}(W u^{(H)}).
$$
Each hop re-queries memory, allowing chaining of facts, temporal reasoning, and relational joins; in the survey, “more hops give improved performance,” and strongly supervised MemNNs markedly outperform LSTMs on synthetic multi-task QA [1702.06186].

Neural Turing Machines generalized the same idea to a controller plus a large addressable memory matrix $M_t \in \mathbb{R}^{N \times D}$. Content-based addressing uses a key $k_t$ and strength $\beta_t$,
$$
w_t^{\text{content}(i)} = \frac{\exp\big(\beta_t\, K(k_t, M_t(i))\big)}{\sum_j \exp\big(\beta_t\, K(k_t, M_t(j))\big)},
$$
reading is
$$
r_t = M_t^\top w_t,
$$
and writing uses erase/add operations,
$$
M_t' = M_t \odot \big(\mathbf{1} - w_t e_t^\top\big) + w_t a_t^\top.
$$
Because the operations are differentiable, the controller and addressing heads can be trained end-to-end, and the survey reports that NTMs learn copying and priority sorting more robustly than stand-alone LSTMs, including generalization from training sequences of length 20 to tests of length 100 [1702.06186].

A distinct extension replaces a fixed story with retrieval from an external corpus. In Top-$k$ Memory Candidates, a query $q$ is mapped to the $k$ highest-scoring documents from an indexed corpus,
$$
I_k = \operatorname{topk}\big(\{ f(q, z) \mid z \in W \}\big),
$$
and only the selected memories are passed to a two-hop End-to-End Memory Network. Attention within the selected set is standard,
$$
s_i = u^\top m_i,\quad
p_i = \mathrm{softmax}(s_i),\quad
o = \sum_{i \in I_k} p_i\, c_i,
$$
but the top-$k$ retrieval itself is a hard, non-differentiable pre-step. On a subset of 62 Winograd Schema Challenge problems mapped to cause–effect queries, the model answered 25/62 correctly, or approximately 40.3%, while the paper also noted the absence of baselines, ablations on $k$, and statistical significance tests [1801.04622].

## 3. Executive, distributed, and narrative memory for long-horizon agents

As LLM agents began to operate over tens of steps, memory stopped being just an addressable matrix and became an explicit control substrate for trajectory management. Three representative designs illustrate this shift.

| Family | Representative system | Memory organization |
|---|---|---|
| Executive memory | MemoBrain | Dependency-aware DAG of task, subtask, evidence, and summary nodes |
| Distributed active memory | ActiveMem | Planner plus distributed shards of distilled semantic gists |
| Narrative working memory | Amory | Episodic narratives with subplots plus semantic triples |

MemoBrain treats memory as executive control rather than passive storage. Each reasoning episode is represented as $x_t = (\tau_t, \omega_t)$, where $\tau_t$ are transient execution traces and $\omega_t$ is the resolved semantic outcome. The episode is abstracted into a compact thought $v_t = \phi(x_t, \mathcal{G}_{t-1})$, added to a directed memory graph
$$
\mathcal{G}_t = (\mathcal{V}_t, \mathcal{E}_t),\quad
\mathcal{V}_t = \mathcal{V}_{t-1} \cup \{v_t\},\quad
\mathcal{E}_t = \mathcal{E}_{t-1} \cup \{(v_i, v_t) \mid v_i \in Dep(v_t)\}.
$$
Under a fixed context budget, MemoBrain applies executive operations $\mathcal{O}_t \subseteq \{\mathrm{Fold}(\cdot), \mathrm{Flush}(\cdot)\}$, collapsing completed sub-trajectories into summaries or replacing invalid and superseded nodes with structural placeholders. Empirically, MemoBrain-8B raised GAIA average Pass@1 from 63.1 to 71.8 when paired with GLM-4.6, and from 68.9 to 74.5 with DeepResearch-30B-A3B, while also improving BrowseComp-Plus and WebWalkerQA [2601.08079].

ActiveMem separates the core reasoner from memory management more radically. A high-level Planner reasons over compact context, while Memorizers distill query-conditioned gists $g_c = \omega(c, q)$ into distributed Memory Shards, and an Operator handles routing, reuse, and asynchronous consolidation. The Planner state is
$$
s_t = (x, h_t, m_{t-1}),
$$
it emits retrieval tasks $\mathcal{Q}_t = \pi(s_t)$, and the interaction history is explicitly trimmed,
$$
h_t = \mathrm{Trim}(h_{t-1} \cup \{a_t, o_t\}).
$$
This decoupling is motivated by the claim that centralized memory forces a trade-off between context overflow and irreversible pruning-induced information loss. On BrowseComp-Plus, ActiveMem achieved LasJ 0.79 with PFLOPs 2,145 and ACT 0.785, outperforming Context-Folding at LasJ 0.72 and PFLOPs 3,920; on GAIA it reached LasJ 0.62 at PFLOPs 187, again the best reported result in its comparison set [2606.10532].

Amory takes a narrative approach. It organizes conversation history into episodic narratives with headlines, subplots, characters, and timestamped leaf fragments, while peripheral non-plot facts are semanticized into triples stored in Neo4j. Retrieval is coherence-driven rather than embedding-only: an LLM selects top-$k$ narratives by plot, actor continuity, causal connection, and temporal consistency. Its momentum-aware “inactive consolidation” is triggered only when a narrative has not received new bindings in the previous iteration, and the paper reports that this timing improves temporal reasoning relative to both no consolidation and rapid fixed-step consolidation. On LOCOMO, the combined episodic-plus-semantic configuration reached Overall J = 87.7 versus Full Context at 86.1, with p90 latency 3.21 s versus 6.08 s [2601.06282].

## 4. Episodic, event-centric, neuro-symbolic, and multimodal memory

A second branch of the literature insists that memory for reasoning must be explicitly time-aware and event-grounded. REMem formalizes episodic memory as a typed multigraph
$$
\mathcal{M} = (V, E),\quad
V = V_{\text{gist}} \cup V_{\text{phrase}},\quad
E = E_{\text{rel}} \cup E_{\text{ctx}} \cup E_{\text{syn}},
$$
where gist nodes store concise episode summaries with normalized time scopes, phrase nodes store concept-level elements, relation edges encode time-scoped triples, context edges bind gists to extracted phrases, and synonymy edges connect semantically similar episodes. Online inference uses a ReAct-style agent with curated tools such as `semantic_retrieve`, `lexical_retrieve`, `find_gist_contexts`, and `find_entity_contexts`, allowing iterative retrieval under explicit temporal constraints. Across four episodic memory benchmarks, REMem reports absolute gains of 3.4% on episodic recollection and 13.4% on episodic reasoning over strong baselines, and it also shows the best refusal F1 of 64.0% on LoCoMo adversarial queries [2602.13530].

CompassMem proposes an event graph as a “logic map.” Each event node is
$$
e = \langle o, \tau, s, \pi \rangle,
$$
with observations $o$, temporal information $\tau$, semantic summary $s$, and participants $\pi$. Relations are typed and explicit, including causal and temporal edges. Query-time navigation is not flat retrieval but subgoal-aware search: the Planner decomposes the query into $\mathcal{H}_q = \Psi_{\text{plan}}(q)$, tracks which subgoals remain unsatisfied, and prioritizes candidate nodes by
$$
p(u) = \max_{j : s_j = 0} \operatorname{sim}(v(s_u), v(h_j)).
$$
On LoCoMo with GPT-4o-mini, CompassMem reached average F1 52.18 versus 47.92 for HippoRAG, and on NarrativeQA it exceeded CAM by more than 5% F1 with GPT-4o-mini and more than 8% F1 with Qwen2.5-14B [2601.04726].

NS-Mem extends structured memory into multimodal agents by combining an episodic layer, a semantic layer, and a logic rule layer,
$$
M = (L_{\text{epi}}, L_{\text{sem}}, L_{\text{logic}}, E).
$$
Its logic nodes pair neural indices with procedural DAGs,
$$
N = (id, c, I, G, F, episodic\_links),
$$
where $G = (V, E, A)$ is a procedural graph and $F$ contains deterministic symbolic query functions such as `queryStepSequence(goal, C)`. SK-Gen builds these structures by extracting multimodal events, consolidating entity-centric semantics, mining sequential patterns with PrefixSpan, verifying them with an LLM, and incrementally updating both vector indices and symbolic transitions. On M3-Bench, NS-Mem reports an average 4.35% improvement in overall reasoning accuracy over pure neural memory systems, including gains of 12.5% on constrained queries [2603.15280].

EgoMemReason turns the multimodal case into an explicit benchmark. It decomposes week-long egocentric memory into entity memory, event memory, and behavior memory, over 500 questions spanning six challenges, with an average of 5.1 evidence video segments and 25.9 hours of memory backtracking per question. The best overall accuracy reported is 39.6%, and performance declines as temporal certification increases, showing that long-horizon multimodal reasoning remains far from solved [2605.09874].

## 5. Lifelong learning, inference-time efficiency, and scalable reasoning reuse

A major contemporary theme is that reasoning memory should not only improve correctness but also prevent recomputation. ENGRAM-R defines an inference-time memory layer external to model weights. Dialogue memory is typed into episodic, semantic, and procedural stores, retrieved with a fixed evidence budget $K$, rendered into compact fact cards
$$
\phi(m) = [\text{id}(m), \text{claim}(m), \text{anchor}(m)],
$$
and injected into a prompt that requires explicit citation of only valid card IDs. On LoCoMo, ENGRAM-R reduced input tokens from 28,371,703 to 3,293,478 and reasoning tokens from 1,335,988 to 378,424 while maintaining judge accuracy near full-context inference; on LongMemEval\_S it improved judge accuracy from 38.0% to 59.8% while also sharply reducing input and reasoning tokens [2511.12987].

Lifelong memory systems push beyond per-instance reuse. ReasoningBank stores distilled strategy-level memories
$$
M_i = (t_i, d_i, c_i, q_i, \tau_i, y_i),
$$
where $y_i \in \{\text{Success}, \text{Failure}\}$ comes from self-judging. Retrieval is query-to-query cosine similarity using stored source queries, and the crucial claim is that failures are first-class memory sources rather than mere discarded rollouts. On WebArena with Gemini-2.5-Flash, ReasoningBank reached success rate 48.8 versus 40.5 for No Memory, while also reducing steps from 9.7 to 8.3; MaTTS then uses parallel or sequential test-time scaling so that richer experiences produce better memory, and better memory guides more effective scaling [2509.25140].

ArcMemo sharpens the same idea into concept-level memory. Instead of storing instance-bound traces, it extracts reusable concepts in either open-ended situation–suggestion form or typed program-synthesis form. At inference, selected concepts are injected into the induction prompt, and memory can be updated every $k$ problems via a generic write–read loop. On ARC-AGI, ArcMemo-PS achieved official Oracle@2 of 59.33 versus 55.17 for the no-memory baseline, a 7.5% relative gain, and the paper reports that dynamically updating memory during test time outperforms an otherwise identical fixed-memory setting [2509.04439].

Memory also appears as a scaling mechanism in multi-agent inference. ReM-MoA defines a per-instance, cross-layer ranked reasoning memory
$$
\mathcal{M}_l = \{(r_{l,j}, s_{l,j}, \rho_{l,j})\}_{j=1}^N,\quad
\mathcal{M}_{\le l} = \bigcup_{k=1}^{l} \mathcal{M}_k,
$$
where proposer traces are comparatively scored by a Reviewer Agent. Curated diversified routing then exposes different agents to Top, Bottom, and Contrastive subsets of prior reasoning. In depth scaling on MATH with width $N=3$, Standard MoA declines from 70.0 at $L=3$ to 61.0 at $L=9$, whereas ReM-MoA rises from 68.0 to 81.0 and the distilled-reviewer variant rises to 84.0 [2606.24437].

At the training level, memory can serve as intrinsic motivation. In the sub-1B regime, Memory-R$^+$ maintains separate success and failure memories over response embeddings, defines exploitation as distance to the centroid of successful retrieved responses,
$$
R_{\text{exploit}}(q,a) = -\|\phi(a) - c_s(q)\|_2,
$$
and exploration as dissimilarity to retrieved failed responses,
$$
R_{\text{explore}}(q,a) = 1 - \max_{b \in B(M^f, q)} \mathrm{CS}(\phi(a), b).
$$
These normalized intrinsic rewards are combined with external correctness rewards under GRPO. The reported effect is improved sample efficiency and collapse avoidance for tiny LLMs on GSM8K and AI-MO, with Memory-R$^+$ consistently avoiding the reward-mode and response-length collapse modes observed under simpler reward shaping [2504.02273].

## 6. Evaluation challenges, mechanistic issues, and controversies

Reasoning memory is not uniformly beneficial, and a recurrent theme is that memory quality and access policy matter at least as much as capacity. The foundational survey already emphasized several persistent problems: discrete memory access is hard to train, soft addressing can become blurry and interfere across slots, and long-term credit assignment remains difficult even with differentiable retrieval [1702.06186]. Later work on exemplar banks makes the same point empirically: in collaborative few-shot reasoning, random exemplar selection can often beat similarity-based retrieval, and in some tasks the inclusion of any exemplars distracts both weak and strong models rather than helping them [2503.05944].

A separate mechanistic result suggests that the relation between reasoning and memorization is itself partially organized in representation space. The reasoning–memorization study identifies per-layer linear reasoning features in the residual stream, computes them by difference-in-means between reasoning-intensive and memory-intensive task sets, and shows that additive steering along this direction can improve both reasoning-heavy and memory-heavy benchmarks depending on the sign of the intervention. The paper’s claim is not that memory should be removed, but that the balance between rule-based generalization and recall-like behavior can be modulated through a single residual direction [2503.23084].

Personalization introduces another failure mode: memory may change not just the answer but the justification trajectory. DRIFTLENS operationalizes this as memory-induced reasoning drift, maps each reasoning step to a value ontology, and compares the no-memory and memory-injected symbolic trajectories using normalized DTW and the Sequence Recurrence Index. Across four LLMs and 10 user-attribute categories, persona memory induces medium-to-large drift above the pragmatic-noise floor even when answers remain fluent and plausible, with Trans status and Disability among the highest-drift categories on most model-metric panels [2607.02374]. This extends the notion of reasoning memory from capacity and retrieval into safety and fairness: a memory system can appear helpful while silently changing which values structure the reasoning.

The evaluation problem therefore remains unsettled. Many strong results are obtained on synthetic QA, story reasoning, constrained dialogs, or task-specific long-horizon benchmarks; several papers explicitly note that realistic datasets with longer narratives, richer relations, partial observability, and real-world noise are still needed [1702.06186, 2602.13530, 2605.09874]. A plausible implication is that the field is converging on two criteria for mature reasoning memory: first, memory must be structurally aligned with the inferential operations demanded by the task, whether those are temporal ordering, procedural execution, constraint satisfaction, or reflective correction; second, memory must be evaluated არა only by retrieval fidelity or end accuracy, but also by stability, interference, auditability, and the degree to which it genuinely reduces recomputation rather than merely relocating it.

Source: https://www.emergentmind.com/topics/reasoning-memory