---
title: Internal Recall in Language & Memory Systems
url: https://www.emergentmind.com/topics/internal-recall
type: topic
---

# Internal Recall in Language & Memory Systems

Searching arXiv for recent papers on internal recall and closely related mechanisms in language models.
In the cited literature, **internal recall** denotes the retrieval of previously encoded information from a system’s own internal state rather than from an external memory store. In large language models, it refers to the recovery of preceding context, memorized sequences, or factual associations from parameters and hidden representations; in online learning, it denotes the amount of past reward information available to the learner; and in autobiographical memory research it denotes episodic, event-specific recollection operationalized as **internal details** in the Autobiographical Interview framework [2501.13491] [2205.14519] [2604.22016]. Across these settings, the central question is how much information is retained internally, how that information is re-accessed, and which architectural or cognitive constraints determine success or failure.

## 1. Conceptual scope and major uses of the term

In work on autoregressive transformers, internal recall is typically framed as a **parametric** phenomenon: the model retrieves information already encoded in its weights or hidden states. This usage includes the recovery of memorized token sequences, the recall of factual triplets, and the reconstruction of earlier context from later cues [2210.03588] [2408.03247] [2501.13491]. In this literature, the relevant mechanisms are not external retrieval systems but internal computations such as attention, MLP updates, neuron activations, residual-stream directions, and layer-wise transformations.

A second use appears in learning theory, where **bounded recall** means that the learner’s output at time $t$ is a function only of the last $M$ observed reward vectors rather than of an unrestricted internal state. Formally, an $M$-bounded-recall algorithm has the form
$$
x_t = f_t(r_{t-M},\dots,r_{t-1}),
$$
with the stationary case using a time-invariant $f$ [2205.14519]. Here internal recall is a resource constraint: it specifies exactly what part of the past remains usable.

A third use comes from cognitive and autobiographical memory research. There, **internal details** are event-specific episodic details tied to a particular time and place, in contrast to **external details**, which include semantic information, repetitions, or off-target material [2604.22016] [1605.05682]. This usage treats internal recall as recollection in Tulving’s sense of autonoetic consciousness and event-specific re-experiencing.

These usages are not equivalent, but they share a common structural concern: a system has encoded information internally, and recall depends on the organization of that internal state, the path by which it is queried, and the constraints imposed by architecture, optimization, or cognition.

## 2. ReCall, self-referencing causal cycles, and the reversal curse

A central formalization of internal recall in autoregressive LLMs is given by **self-referencing causal cycles** or **ReCall**. For a sequence $\mathcal{S}_{\text{seq}}=[e_1,\dots,e_n]$, an autoregressive model learns
$$
P_\mathcal{M}(e_1,\dots,e_n)\approx \prod_{t=1}^n P_\mathcal{M}(e_t\mid e_1,\dots,e_{t-1}),
$$
which makes forward continuation straightforward but backward recovery difficult [2501.13491]. This asymmetry underlies the **reversal curse**: a model that easily predicts a later line from an earlier one may fail to recover the earlier line from the later one.

The paper formalizes this asymmetry by contrasting forward prediction,
$$
S_r=\argmax_{s\in\mathcal{S}} P_\mathcal{M}(s\mid S_l),
$$
with the reverse problem,
$$
S_l^\star=\argmax_{s\in\mathcal{S}} P_\mathcal{M}(s\mid S_r),
$$
which is not directly modeled by an autoregressive LM [2501.13491]. Using Bayes’ rule, the desired reverse inference can be written as
$$
S_l=\argmax_{s\in\mathcal{S}} P_\mathcal{M}(S_r\mid s)\,P_\mathcal{M}(s),
$$
but the paper argues that this global search is computationally infeasible and architecturally misaligned for next-token models [2501.13491].

ReCall proposes a constructive alternative. If a token or token-sequence from an earlier segment reappears later, it can act as a **cycle token** that creates a usable loop in the left-to-right causal graph. With
$$
S_r' := S_r \oplus e_1,
$$
the later occurrence of $e_1$ allows the model to continue forward from that token and regenerate an approximation to the left segment,
$$
S_l' := [e_2,\dots,e_i],
$$
so that
$$
S_l = e_1 \oplus S_l' \approx [e_1,\dots,e_i].
$$
The paper describes repeated spans such as titles, names, or phrases as natural cross-references in the model’s internal “library” [2501.13491].

The controlled experiments are correspondingly simple and mechanistic. A small decoder-only transformer with approximately **90k parameters**, **2 layers**, **8 attention heads**, **embedding dimension 36**, **Adam**, **learning rate 0.001**, **batch size 1024**, and **cross-entropy loss** was trained on synthetic sequences explicitly containing cycles [2501.13491]. In deterministic few-token settings such as training on $[e_1,e_2,e_3,e_1]$ and testing on $[e_3,e_1,e_2]$, the model reached **100% validation accuracy** on both “All” and “Last” in almost all configurations [2501.13491]. In stochastic settings with sequences of the form $(e_1,\{e_{2_i}\}_{i=1}^n,e_3,e_1)$, the correct next token after the cycle token was predicted with probability approximately **$1/n$**, showing that cycles constrain recall to a small candidate set rather than the full vocabulary [2501.13491].

This yields a specific interpretation of internal recall in autoregressive LLMs: recall need not require explicit backward inference if repeated token sequences create constructive paths that let forward prediction approximate reversal.

## 3. Layer-wise and neuron-level mechanisms of recall in transformers

Beyond sequence-level cycles, several works locate internal recall in specific layers, neurons, and positions. In memorized sequence completion, the recall of idioms and memorized factual statements is described as a **two-stage computation**: early layers move the correct token to the top of the hidden distribution, while upper layers increase its confidence [2210.03588]. The paper concludes that memorized information is stored and retrieved in the **early layers** of the network, with upper layers acting mainly as confidence amplifiers [2210.03588].

A more explicit neuron-level account is given by **Knowledge Neurons**. For a fact $(s,r,o)$, neurons in transformer FFNs are identified by integrated-gradients attribution over multiple paraphrased factual prompts, and the **KN Score**
$$
\text{KN Scores} = \frac{1}{|\omega|}\sum_{\omega_i^l\in\omega}\omega_i^l
$$
is used as a scalar measure of internal factual recall [2408.03247]. On 2-hop factual reasoning, the paper reports that moving from single-hop to 2-hop contexts produces substantial drops in KN Scores for both first-hop and second-hop facts; for Mistral-7B, the changes are **$-10.84\%$** and **$-11.77\%$**, respectively [2408.03247]. Correct answers are therefore often obtained without full activation of the relevant fact circuits. The paper classifies these cases as **shortcuts**, and reports that for Mistral-7B, **44.25%** of correct 2-hop answers under no CoT fall into shortcut categories rather than full multi-hop recall [2408.03247].

The causal role of these neurons is tested directly. Enhancing both first-hop and second-hop knowledge neurons raises Mistral-7B accuracy by **15.11 points** and rescues **31.05%** of previously failed cases, while suppressing them reduces accuracy by **49.53 points** and flips **77.29%** of previously correct cases to wrong [2408.03247]. These interventions show that internal recall is not merely correlated with output correctness; it is causally implicated in it.

A distinct but related account argues that factual recall in autoregressive transformers is not solely subject-centered. In a relation-focused analysis, the decisive locus is the **last relation token** in **middle–late layers**, where the model aggregates **relational attributes** and extracts the object [2408.15091]. Causal tracing yields **Average Indirect Effect of Relation** values around **2.23%** at the last relation token in GPT2-XL, versus approximately **0.04–0.06%** at other positions [2408.15091]. Using a vocabulary lens, the paper defines an **attributes rate**
$$
\mathbf{AR}^l = \frac{|\mathbb{A}_N^l \cap \mathbb{A}_{\text{relation}(r)}|}{|\mathbb{A}_N^l|}
$$
and reports a Spearman correlation of **0.97** between negative object rank and this relation-attribute enrichment [2408.15091]. This supports a view in which subject and relation each contribute distinct internal states to factual recall.

A more theoretical alternative is the **geometric** account of factual recall. In a controlled single-layer setting with random bijections $g_r:[N]\to[N]$, subject embeddings are constructed as linear superpositions of relation-specific attribute vectors, and a small ReLU MLP acts as a relation-conditioned selector rather than as an associative key–value store [2605.12426]. The single-hop theorem gives an embedding dimension
$$
d = 4R\log(N) + 1
$$
with MLP width $R$ for exact recall [2605.12426]. In the multi-hop setting, the same work proves a capacity–depth trade-off without chain-of-thought, and shows that with CoT a 1-layer transformer can solve $k$-hop recall with
$$
d = \tilde O(R+k)
$$
and MLP width $\tilde O(R)$ [2605.12426]. Empirically, gradient descent is reported to discover precisely this predicted structure, and zero-shot transfer to new bijections after re-initializing subject embeddings suggests that the MLP has learned a generic selection mechanism rather than memorizing particular facts [2605.12426].

Taken together, these works depict internal recall in transformers as a layered phenomenon: early candidate promotion, neuron-level factual circuits, relation-conditioned extraction, and, in some settings, geometric superposition in embedding space.

## 4. Multi-hop, multilingual, and multimodal recall

When reasoning requires more than a single factual association, internal recall becomes a dynamic pathway rather than a one-step lookup. In **ACE**, multi-hop factual recall is described as a sequence in which **implicit subjects function as query neurons**, activating downstream **value neurons** across transformer layers [2510.07896]. The FFN output is written as
$$
F_i^l = \sum_{k=1}^N m_{i,k}^l \cdot fc2_k^l,
$$
with
$$
m_{i,k}^l = \sigma(fc1_k^l \cdot (h_i^{l-1}+A_i^l)),
$$
so each neuron becomes an explicit key–value pair [2510.07896]. The paper defines distribution-based importance scores for value neurons and query scores based on inner products with subkeys, then shows that peak query layers precede peak value layers by one or two layers [2510.07896]. Ablating top query neurons in key layers causes **46.2%** and **61.9%** drops in multi-hop performance, and the proposed editing method **ACE** improves multi-hop KE by **9.44%** on GPT-J and **37.46%** on Qwen3-8B [2510.07896].

A parallel line of work studies multilingual factual recall and argues that multilingual LLMs often use an **English-centric** internal pipeline even for non-English prompts [2505.20546]. Logit-lens analysis on Llama-3.2-3B shows that in correct multilingual cases the English answer token rises to rank 1 around **layer 21**, and only later is converted into the target-language token [2505.20546]. Errors split into two kinds: in **78.3%** of incorrect examples the correct English answer never becomes top-ranked in middle layers, whereas in **21.7%** it does become top-ranked but final translation into the target language fails [2505.20546]. Two vector interventions target these stages separately: a **recall vector** extracted from 5-shot ICL runs and injected at **layer 3** with scale **2**, and a **translation difference vector** applied at **layer 21** [2505.20546]. Combined, they raise Chinese accuracy from **2.39%** to **41.83%**, and non-English overall accuracy from **20.07%** to **41.22%** [2505.20546]. This suggests that multilingual inconsistency often reflects failures to route non-English prompts into a reliable internal recall path rather than absence of stored knowledge.

The speech–text multimodal case shows a different limitation. In SpiritLM, factual recall mechanisms known from text models remain largely intact for text input, but only **partially carry over** to speech input [2605.22170]. Using Causal Mediation Analysis and Average Indirect Effect heatmaps, the paper reports strong, localized causal peaks for text-to-text recall in mid-layer MLPs around subject tokens, but much **smaller** and more **diffuse** signals for speech-to-text recall [2605.22170]. The conclusion is not that factual mechanisms vanish in speech, but that the text-style circuits are only partially engaged when the input modality is speech [2605.22170].

A recurring intervention across these settings is **Chain-of-Thought**. In the Knowledge Neurons study, few-shot CoT raises Mistral-7B’s proportion of full multi-hop recall cases from **55.75%** to **91.23%**, while reducing shortcut cases from **44.25%** to **8.77%** [2408.03247]. In the geometric theory, CoT changes the complexity class of multi-hop recall by reducing it to repeated single-hop selection [2605.12426]. These results jointly suggest that CoT often improves internal recall not merely by making outputs longer, but by altering the internal path by which intermediate states are formed and reused.

## 5. Bounded recall, internal replay, and continual adaptation

Outside next-token factual retrieval, internal recall also names the retention and reuse of past information under explicit memory constraints. In online learning with full information, an $M$-bounded-recall algorithm is one whose action at time $t$ depends only on the last $M$ reward vectors [2205.14519]. This constraint sharply changes what is achievable: the minimax per-round regret becomes
$$
\Theta\!\left(\sqrt{\frac{\log d}{M}}\right),
$$
and naive constructions that rerun mean-based algorithms such as Hedge on the last $M$ rounds incur **constant regret per round** [2205.14519]. The paper’s positive result is **AverageRestart**, a stationary bounded-recall algorithm that averages over virtual restarts of a base learner within the last $M$ rounds and attains the optimal rate [2205.14519]. It also proves that bounded-recall algorithms must be **order-aware**: any stationary symmetric bounded-recall algorithm incurs constant regret per round [2205.14519]. In this setting, internal recall is a formal memory window, and the theory characterizes precisely how much learning remains possible under that limit.

In continual learning, **Automatic Recall Machines** reinterpret internal recall as **internal replay**. The task model $f_\theta$ itself is treated as an implicit memory of past data, and replay samples are generated on the fly by gradient ascent in input space, using only the current model and its pre-update snapshot, with no replay buffer and no separate generator network [2006.12323]. The recall objective includes a divergence term
$$
\operatorname{D}(f_{\theta'}(\hat{x}), f_\theta(\hat{x}))
$$
to find inputs on which the current update causes maximal non-local change, together with regularization terms and a distillation loss on the generated recall batch [2006.12323]. On CIFAR10-5, ARM with **100 recalled images** and **no sample memory** reached approximately **26.4%** accuracy with forgetting around **8.1%**, compared with experience replay using **100 stored images**, which reached approximately **22.4%** accuracy with forgetting around **66%** [2006.12323]. On MiniImageNet-20, ARM with **100 recalled images** achieved approximately **5.53%** accuracy and forgetting around **11.9%**, close to experience replay with **500 stored images** at approximately **5.6%** accuracy and forgetting around **55%** [2006.12323]. The work therefore treats internal recall as a practical mechanism for preserving prior behavior through internally generated counterfactual inputs.

These frameworks differ substantially from LLM factual recall, but they illuminate a general distinction. Internal recall can denote either a **capacity limit** on how much past state remains usable or an **internal reconstruction mechanism** that recreates missing data or context from the model’s own parameters. Both meanings are central to understanding learning under memory constraints.

## 6. Human episodic recall, quantitative laws, and broader significance

In human memory research, internal recall is most closely aligned with **episodic recollection**. A large-scale reanalysis of Murdock’s free-recall data found substantial individual differences in three measures: total recall, recency–primacy balance, and initial recall position, with effect sizes $\eta^2$ ranging from **0.09** to **0.26** [1605.05682]. These measures were reported to be relatively uncorrelated, and variance in initial recall position was proposed as a possible index of executive control; smaller variance was associated with higher recall [1605.05682].

A complementary quantitative account proposes a **fundamental law of memory recall**. If $M$ items are actually stored in memory, the average number recalled is predicted by
$$
R(M)\approx \sqrt{\frac{3\pi}{2}M},
$$
or approximately $2.17\sqrt{M}$ [1905.02403]. The derivation models recall as a deterministic walk on a random similarity graph defined by sparse overlapping memory representations, with termination when the walk enters a cycle [1905.02403]. Empirically, the law was supported using a large-scale free-recall and recognition experiment in which $M$ was independently estimated from recognition accuracy [1905.02403]. This gives a starkly different but structurally related picture of internal recall: retrieval is limited not by storage alone but by the path structure of an internal associative search.

Autobiographical recall adds phenomenology and embodiment. In interviews with **806** Holocaust survivors, sentence-level gaze features were used to distinguish **internal** episodic details from **external** semantic details, and to predict temporal context such as pre-war, wartime, liberation, post-war, and present-day [2604.22016]. Vertical gaze features in the **5 seconds preceding sentence onset** differed strongly between internal and external sentences; for example, **Gaze Y** had $t=18.75$, $p<0.001$, and **Gaze Y SD** had $t=10.23$, $p<0.001$ [2604.22016]. Sequence models trained within subjects could predict temporal context from gaze segments, and the paper reports that gaze windows entirely preceding sentence onset were sufficient for prediction [2604.22016]. This suggests that internal recall states can have anticipatory behavioral signatures before verbalization.

The cue structure of autobiographical recall can also be manipulated. In a within-subject study with **18** participants, **MemoryDiorama** compared **Photo-Only**, **Static Diorama**, and **MemoryDiorama** conditions for recall from personal photographs [2604.06773]. MemoryDiorama produced significantly more **internal details**, with means **29.61**, **31.44**, and **38.28** across the three conditions and an ANOVA of $F(2,34)=5.50$, $p=.0085$, $\eta_p^2=.24$ [2604.06773]. The increase was especially pronounced for **perceptual details**, with means **4.11**, **4.33**, and **7.22**, and $F(2,34)=5.98$, $p=.0059$, $\eta_p^2=.26$ [2604.06773]. Subjective **visual details** ratings were also higher in MemoryDiorama, with means **5.17**, **5.22**, and **6.06**, and $F(2,34)=4.59$, $p=.0172$, $\eta_p^2=.21$ [2604.06773]. The ratio of internal to total details did not significantly differ, which suggests that augmented cues increased overall elaboration rather than selectively increasing episodic specificity [2604.06773].

Across artificial and biological systems, internal recall therefore appears in at least three general forms. It can be a **computational workaround** for directional causality, as in ReCall; a **localized internal mechanism** involving layers, neurons, and residual-stream directions, as in transformer factual recall; or a **resource-bounded and cue-dependent retrieval process**, as in online learning and human memory [2501.13491] [2408.03247] [2205.14519] [2604.22016]. A plausible implication is that progress on the topic will depend less on treating recall as a single faculty than on identifying the precise internal path by which a system reconstructs what it already stores.

Source: https://www.emergentmind.com/topics/internal-recall