In-Context Factual Recall in Transformers

• IC-recall is a transformer capability that leverages prompt context to retrieve stored factual associations, typically formatted as subject–relation–object triples.
• Research indicates that IC-recall involves distinct phases of relation identification and answer retrieval, with variations across model architectures and modalities.
• Empirical studies using causal mediation and associative memory analyses reveal that attention heads and MLP components collaboratively drive accurate factual recall even amid contextual competition.

Searching arXiv for the cited papers to ground the article in current literature. In-context factual recall (IC-recall) denotes a class of behaviors in which a transformer uses the current prompt context to identify a factual association and then surface the corresponding attribute. Across recent work, this includes subject–relation prompts such as “The capital of Italy is ___,” contexts of $(\text{subject}, \text{answer})$ examples generated from a hidden relation, and retrieval-augmented settings in which context is used to select demonstrations or factual evidence (Modica et al., 21 May 2026, Huang et al., 26 May 2026, Li et al., 2024). The common question is not only whether the correct fact can be produced, but also how the context activates the internal pathways that encode, store, and retrieve that fact.

1. Definitions and formal scope

A standard formalization treats a factual association as a triple such as subject–relation–object or subject–relation–attribute, for example $(\text{Italy}, \text{capital of}, \text{Rome})$, and operationalizes factual recall as producing the correct object token when prompted with the subject and relation (Modica et al., 21 May 2026). In text-only mechanistic work, prompts are often partitioned into a prefix $\texttt{p}$, subject tokens $\texttt{s}$, relation tokens $\texttt{r}$, and a final prediction position $\texttt{t}$; the model must make the correct attribute $a$ high-scoring at $\texttt{t}$ (Chughtai et al., 2024).

A second, more explicit IC-recall formalization appears in work on fine-tuning dynamics. There, a transformer receives a context of $(\text{subject}, \text{answer})$ pairs generated from a hidden relation $r^*$, followed by a query subject, and must first infer the hidden relation and then retrieve the corresponding answer from parametric memory (Huang et al., 26 May 2026). This formulation isolates two operations: relation identification from context and answer retrieval from stored factual triplets.

The literature also distinguishes the retrieval of stored knowledge from retrieval from external tools. In the multimodal SpiritLM study, factual recall concerns parametric factual knowledge stored in model weights, and the “in-context” aspect is the internal activation dynamics triggered by a particular text or speech prompt rather than a contrast between prompt-provided facts and stored facts (Modica et al., 21 May 2026). By contrast, retrieval-oriented relation extraction work treats recall as the generation of entity-pair queries that guide retrieval from a training corpus, followed by in-context reasoning over retrieved demonstrations (Li et al., 2024).

A related but distinct use of “recall” appears in long-form factuality evaluation. There, recall means whether a generated response covers the relevant reference facts that should be included, rather than whether internal circuits successfully retrieve a parametric association (Jafari et al., 3 Apr 2026). This suggests that “IC-recall” has acquired both a mechanistic meaning and an evaluation meaning, and the two should not be conflated.

2. Mechanistic accounts in text-only transformers

Mechanistic studies converge on the view that factual recall is not implemented by a single monolithic pathway. In the “additive motif” account, factual recall at the answer position is the sum of several distinct contributions: subject heads that attend to subject tokens and write attribute directions mostly in the subject-relevant set $(\text{Italy}, \text{capital of}, \text{Rome})$0; relation heads that attend to relation tokens and write directions mostly in the relation-relevant set $(\text{Italy}, \text{capital of}, \text{Rome})$1; mixed heads that contribute separate additive updates from subject and relation tokens; and MLPs at the answer position that broadly boost many attributes in $(\text{Italy}, \text{capital of}, \text{Rome})$2 (Chughtai et al., 2024). Because residual stream addition and unembedding are approximately linear in this analysis, these components can constructively interfere on the correct answer even when no single component is sufficient on its own (Chughtai et al., 2024).

This account is compatible with a more circuit-level three-step description of factual recall. For prompts such as “the capital of France is,” task-specific attention heads first extract the topic token “France” from context and pass it forward; a subsequent MLP then amplifies or erases individual head contributions so that the topic token stands out in the residual stream; and a deeper MLP generates a component that redirects the residual stream toward the correct answer, analogous to an implicit function such as $(\text{Italy}, \text{capital of}, \text{Rome})$3 (Lv et al., 2024). In that work, a regression-based decomposition of MLP outputs separates an intercept-like component from head-weighting terms, making it possible to interpret some MLPs as head activators and others as function-applying modules (Lv et al., 2024).

These mechanistic results challenge a narrow “single fact circuit” story. The additive account shows that subject-based evidence, relation-based evidence, mixed subject–relation evidence, and MLP-based boosts all contribute non-trivially to the final correct logit (Chughtai et al., 2024). The circuit account likewise finds that attention and MLP components play different roles across layers rather than reducing to one clean lookup step (Lv et al., 2024).

A further complication is architectural variation. A cross-architecture analysis reports that GPT- and most LLaMA-family models exhibit strong factual-recall contributions in early MLP layers, whereas Qwen-based models show unusually strong contributions from the earliest attention modules; DeepSeek distillations follow the pattern of their base architecture (Choe et al., 10 Sep 2025). This suggests that even within autoregressive transformers, the locus of factual recall is architecture-dependent rather than universal.

3. Associative memories, emergence, and training dynamics

Theoretical work casts factual recall as associative memory. In one line of analysis, a shallow transformer with a single self-attention layer followed by an MLP can achieve perfect factual recall on a synthetic task whenever either the total number of self-attention parameters or the MLP parameters scales, up to log factors, linearly with the number of facts (Nichani et al., 2024). The construction shows that value matrices or the MLP can each act as an associative memory, and that storage capacity scales linearly with parameter count for both linear and MLP associative memories (Nichani et al., 2024).

A complementary fine-tuning analysis studies IC-recall in a one-layer transformer equipped with a pre-constructed MLP associative memory over $(\text{Italy}, \text{capital of}, \text{Rome})$4 triplets. Fine-tuning on IC-recall data converges to a pairwise attention pattern: the EoS token distributes attention equally within each subject–answer pair but unequally across different pairs, thereby identifying the hidden relation, while the retrieved relation token then attends to the query subject to recover the answer (Huang et al., 26 May 2026). The same work proves that only polylogarithmically many fine-tuning samples, in the number of stored knowledge triplets, are needed for near-perfect IC-recall, and reports that the pairwise attention pattern also emerges when the MLP memory is pretrained rather than explicitly constructed (Huang et al., 26 May 2026).

Another toy setting decomposes in-context recall into two separate prediction mechanisms. One mechanism uses discrete symbolic labels to perform associative recall of which sequence should be resumed; the second, largely agnostic to those labels, performs a “Bayesian-style” next-token prediction based on previous tokens and context (Daniels et al., 2 Jul 2025). Training dynamics reveal that the second capability emerges much earlier than the first, and OLMo checkpoints on an in-context translation task show a similar gap between performance on the first token of a recalled task and performance on subsequent tokens (Daniels et al., 2 Jul 2025). This suggests that retrieval of the correct latent object and continuation under that object can have distinct phase transitions.

Theoretical and synthetic analyses therefore agree on a broad point: IC-recall is not reducible to one undifferentiated capability. It contains at least a routing or identification stage and a content-retrieval or continuation stage, and these stages can have different parameter dependencies and different learning dynamics (Huang et al., 26 May 2026, Daniels et al., 2 Jul 2025, Nichani et al., 2024).

4. Competition, shortcuts, and contextual control

A major theme in IC-recall research is competition between parametric knowledge and context. In a world-capitals setting, models are prompted with a counterfactual prefix such as “The capital of Poland is London. Q: What is the capital of Poland? A:” and must choose between the memorized answer and the in-context answer (Yu et al., 2023). The tendency to output the memorized answer increases with country frequency and with the co-occurrence frequency of the country and its true capital in pretraining, while increasing the frequency of the in-context city makes the model less likely to adopt that city and more likely to revert to the memorized answer (Yu et al., 2023). The same study localizes attention heads that favor memorized versus in-context answers and shows that scaling a single memory head at runtime can push Pythia-1.4B from 26% in-context and 43% memorized to 86.2% in-context and 4% memorized on the capitals task (Yu et al., 2023).

Work on knowledge neurons shows that even correct multi-hop answers need not arise from faithful stepwise recall. Two-hop reasoning instances are categorized as TT, TF, FT, or FF depending on whether the first-hop and second-hop fact neurons are strongly recalled; non-TT cases are treated as shortcut-like pathways (Wang et al., 2024). Directly enhancing the recall process of the relevant knowledge neurons improves reasoning performance, whereas suppressing it causes large degradations, demonstrating a causal link between factual recall and reasoning success (Wang et al., 2024). Chain-of-Thought prompting increases knowledge-neuron activation for the required facts and reduces shortcut behavior, particularly in few-shot CoT settings (Wang et al., 2024).

A broader robustness benchmark studies whether models can preserve correct internal knowledge when external context contains counterfactual information. On the QA-A setting, in which the answer-bearing part of the context is edited to be wrong, Llama2-13B drops from 96.59% accuracy with original context to 12.80% with edited context on EventKG, despite achieving 62.21% with no context (Liu et al., 2023). The benchmark’s Misleading Rate asks: among questions the model answered correctly with no context, what fraction become wrong once counterfactual context is added. On EventKG QA-A, reported M-Rates range from 80.87% to 84.97% across the evaluated models (Liu et al., 2023). This indicates that context can overwrite correct parametric recall at very high rates.

These studies jointly show that IC-recall is not simply “using context more.” Depending on prompt construction, models may rely on memorized knowledge, be overwritten by context, or answer correctly through shortcut-like internal pathways. This suggests that factual faithfulness and factual correctness are separable properties.

5. Multimodal IC-recall and the speech–text boundary

Recent multimodal work extends factual-recall analysis from text-only models to speech–LLMs. The SpiritLM study examines whether factual-recall mechanisms known from text-only LMs carry over to a multimodal model that jointly represents text and speech tokens using a shared transformer stack built on a Llama 2 text backbone, with modality declaration tokens “T” and “S,” a Llama2 BPE tokenizer for text, and HuBERT-based discrete speech tokens for speech (Modica et al., 21 May 2026). The experiments compare Text $(\text{Italy}, \text{capital of}, \text{Rome})$5 Text and Speech $(\text{Italy}, \text{capital of}, \text{Rome})$6 Text while keeping the output object in text so that the target token is identical across modalities (Modica et al., 21 May 2026).

The core method is Causal Mediation Analysis with clean, corrupted, and corrupted-with-restoration runs. If $(\text{Italy}, \text{capital of}, \text{Rome})$7 is the clean prompt, $(\text{Italy}, \text{capital of}, \text{Rome})$8 is a prompt with the subject corrupted, and $(\text{Italy}, \text{capital of}, \text{Rome})$9 is a patched internal component, the indirect effect is defined as

$\texttt{p}$0

and the Average Indirect Effect (AIE) averages this quantity across prompts (Modica et al., 21 May 2026). In the speech setting, subject-token alignment is handled by CTC-based forced alignment using a HuBERT-LARGE CTC ASR model and a mapping from aligned frames to HuBERT speech tokens, allowing speech-token corruption and token-wise comparison with text (Modica et al., 21 May 2026).

The empirical result is that SpiritLM in Text $\texttt{p}$1 Text behaves almost identically to the text backbone: strong causal signals appear around subject tokens in mid transformer layers, especially MLP sublayers, with some signal at the final token position in upper layers (Modica et al., 21 May 2026). In Speech $\texttt{p}$2 Text, however, the overall AIE is much smaller, the signal is more diffuse, and although there is still a detectable bump around subject-token positions in MLP and attention layers, the effect is “drastically” reduced (Modica et al., 21 May 2026). The same MLPs and attention heads appear to mediate recall in both modalities, but speech does not excite them as strongly, which the authors attribute to a modality semantic gap and to the limitations of discrete HuBERT tokens (Modica et al., 21 May 2026).

For IC-recall, the implication is that multimodal models need not be modality-neutral even when they share parameters. The stored factual memory may remain largely text-shaped, while the ability of speech context to trigger that memory is weaker and less localized. This suggests that the context-to-memory mapping, rather than the storage site alone, is central to multimodal factual recall.

6. Measurement, self-awareness, and benchmarked variation

Benchmark work emphasizes that recall quality cannot be characterized by a single prompt or a single scalar. BELIEF and BELIEF-ICL extend knowledge probing to multi-prompt settings by measuring accuracy, consistency, and reliability over large sets of prompt paraphrases and entity aliases (Zhao et al., 2024). MyriadLAMA expands LAMA-UHN to 4100 relational templates, 21,140,500 derived triples, and 6,492,800 prompts, enabling prompt-ensemble evaluation rather than single-template probing (Zhao et al., 2024). For Llama3-8B on MyriadLAMA, moving from zero-shot to 4-random, 4-relation, and 4-template ICL increases Acc@1 from 0.3427 to 0.5205, 0.6871, and 0.7268, while consistency rises from 0.0240 to 0.2156, 0.3659, and 0.4015 (Zhao et al., 2024). This indicates that same-relation and same-template demonstrations strongly stabilize IC-recall, but the same benchmark also shows a large oracle gap, revealing that prompt-based probing remains a lower bound on stored knowledge (Zhao et al., 2024).

Another measurement direction asks whether models know, at generation time, whether a fact will be recalled correctly. “Factual self-awareness” is defined as a linearly decodable signal in the residual stream at the final prompt token that predicts whether the model’s own next-token distribution will place the correct attribute among its top predictions (Tamoyan et al., 27 May 2025). A linear probe

$\texttt{p}$3

is trained to predict known versus forgotten samples from residual representations (Tamoyan et al., 27 May 2025). On Gemma-2 2B, the best probe achieves AUC ROC 0.896, accuracy 0.820, and $\texttt{p}$4 over baseline; on Pythia-70M, $\texttt{p}$5, indicating that a usable self-awareness direction does not emerge at that scale (Tamoyan et al., 27 May 2025). The signal peaks in intermediate layers, is robust to quotation marks and random unrelated prefixes, but degrades substantially under few-shot contexts, especially when the same entity is repeated across demonstrations (Tamoyan et al., 27 May 2025). This suggests that IC-recall includes a meta-level feature—awareness-at-generation of likely recall success—that is itself context-sensitive.

A related but distinct benchmarked use of recall appears in long-form factuality evaluation. There, factual recall is defined over a reference set of atomic facts $\texttt{p}$6 extracted from retrieved evidence for a query, and the recall score counts how many of those facts are covered in the generated response (Jafari et al., 3 Apr 2026). The framework further defines an importance-aware weighted recall

$\texttt{p}$7

and reports that current long-form models perform substantially better on precision than on recall (Jafari et al., 3 Apr 2026). This use of “recall” is evaluative rather than mechanistic, but it reinforces the broader point that factual systems must be judged not only by whether individual claims are correct, but also by whether relevant facts are actually surfaced.

Taken together, these benchmarks indicate that IC-recall varies along at least three axes: whether the fact can be elicited at all, whether it is robust across paraphrases and contexts, and whether the model internally encodes that it is about to succeed or fail. A plausible implication is that future work on factual recall will continue to combine mechanistic tracing, prompt-ensemble evaluation, and context-sensitive confidence estimation rather than relying on any single notion of “knowing a fact.”