---
title: 'Memory Inception: Latent Steering in LLMs'
url: https://www.emergentmind.com/topics/memory-inception-mi
type: topic
---

# Memory Inception: Latent Steering in LLMs

Memory Inception (MI) denotes a family of memory-centric mechanisms in which control, inference, or adaptation is displaced from visible inputs or parameter updates into an explicit memory substrate. In its most explicit recent usage, MI is a training-free steering method for large language models that inserts text-derived key-value (KV) banks at selected attention layers, so that reminders, personas, heuristics, or policies are represented as latent side memories rather than as visible prompt text [2605.06225]. Closely related work uses the same or a near-equivalent framing for recursive working-memory/long-term-memory consolidation in neural inference [2310.09297], dynamically evolving explicit memory in few-shot continual learning hardware [2207.06810], and the theoretical reuse of prior inference trajectories as reusable structure in Memory-Amortized Inference [2512.05990].

## 1. Terminological scope and major usages

The term is not uniform across the literature. In some papers it is the formal name of a method; in others it is an interpretive label for architectures whose primary operation is to write into, consolidate within, retrieve from, or attack a memory mechanism. This suggests that MI is best understood as a memory-native design pattern rather than a single standardized algorithm.

| Usage | Core mechanism | Source |
|---|---|---|
| Memory Inception | Text-derived KV banks injected at selected LLM layers | [2605.06225] |
| “Memory Inception” process in PMI | Perception writes to working memory, then consolidates into long-term memory | [2310.09297] |
| MI-style continual learning | Frozen controller plus dynamically evolving explicit memory on PCM IMC | [2207.06810] |
| Memory-Amortized Inference | Prior search trajectories reused as structural memory | [2512.05990] |
| Visual Inception / Inception attacks | Memory poisoning or multi-turn jailbreaks against memory mechanisms | [2604.16966], [2504.20376] |
| Approximate acronym overlap | “MI” denotes Multi-function Inception-V4, not Memory Inception | [1811.11996] |

A common misconception is that MI names a single established subfield. The record provided here does not support that interpretation. Instead, the clearest convergence is conceptual: MI-style systems move the locus of control from direct prompt wording or full-network retraining toward an explicit, persistent, and selectively accessible memory representation.

## 2. MI as latent-memory steering for large language models

In the LLM setting, MI was introduced as a training-free steering interface positioned between instruction prompting and activation steering [2605.06225]. Prompting provides strong control, but guidance tokens remain visible and are cached at every layer. Activation steering is compact, but it acts through residual-space additions and is less natural for large structured reminders. MI instead treats steering as selective KV allocation in latent attention space.

The core intuition is that a reminder, persona, heuristic card, or policy need not be materialized throughout the visible transcript if it only matters at a small number of attention sites. MI therefore encodes guidance text into latent KV banks and injects those banks only at selected layers, heads, or KV groups. The model’s own queries can then route to those memory slots where they are useful. In this sense MI is text-faithful like prompting, but hidden and compact like activation steering; unlike CAA, it does not compress behavior into a single global residual vector, but preserves richer structured guidance as reusable latent content [2605.06225].

At a decoder attention site, the starting point is standard attention:
$$
Attn(q_t,K^x_{\le t},V^x_{\le t}) = softmax\!\left(\frac{q_t {K^x_{\le t}}^\top}{\sqrt{d_h}}\right)V^x_{\le t}.
$$
MI augments the candidate context with one or more memory banks
$$
\mathcal{B}^{(b)}=\{(k^{(b)}_m,v^{(b)}_m)\}_{m=1}^{M_b}, \qquad
K^{(b)}=[k^{(b)}_1,\ldots,k^{(b)}_{M_b}]^\top, \qquad
V^{(b)}=[v^{(b)}_1,\ldots,v^{(b)}_{M_b}]^\top,
$$
so that the cache at a selected site becomes
$$
K_t^\star=concat(K^x_{\le t},K^{(1)},\ldots,K^{(B)}), \qquad
V_t^\star=concat(V^x_{\le t},V^{(1)},\ldots,V^{(B)}),
$$
with output $o_t^\star=Attn(q_t,K_t^\star,V_t^\star)$ [2605.06225].

The same paper also gives a bank-level mixture interpretation:
$$
o_t^\star=\sum_{b=0}^{B}\widetilde{\pi}_t^{(b)}o_t^{(b)}, \qquad
\widetilde{\pi}_t=softmax([\widetilde{\beta}_t^{(0)},\ldots,\widetilde{\beta}_t^{(B)}]).
$$
To prevent larger banks from winning purely by cardinality, MI uses size-normalized evidence
$$
\beta_t^{(b)} = \widetilde{\beta}_t^{(b)}-\log M_b
= \log\!\left(\frac{1}{M_b}\sum_{m=1}^{M_b}\exp(s^{(b)}_{t,m})\right)+c_t^{(b)}.
$$
The appendix reportedly shows that bank-wise attention can be exactly decomposed into within-bank softmax plus bank-level routing [2605.06225].

## 3. Construction of KV banks, layer selection, and storage trade-offs

MI is training-free. There is no finetuning, no learned prefix, and no learned adapter [2605.06225]. A bank is constructed at inference time from source text by running that text through the frozen base model, collecting hidden states at token positions, keeping the positions aligned to the relevant descriptor content unless the experiment intentionally keeps the whole wrapper, and projecting the retained hidden states through the model’s own key and value matrices:
$$
k^{(b)}_{\ell,u,m}=W^{(\ell)}_{K,u}Norm_\ell(h^{(b)}_{\ell,m}), \qquad
v^{(b)}_{\ell,u,m}=W^{(\ell)}_{V,u}Norm_\ell(h^{(b)}_{\ell,m}).
$$
Banks can include token-level slots from the original descriptor and contextualized variants of the same descriptor, which makes them more robust than a single compressed residual vector [2605.06225].

Selectivity is central. MI does not inject guidance at every layer; it first identifies where the reminder matters. The selector scores candidate attention units by a target-vs-reference alignment margin:
$$
a_{\ell,u} = \max_j \frac{q_{\ell,u}^{\top}k^+_{\ell,u,j}}{\sqrt{d_h}}
- \max_j \frac{q_{\ell,u}^{\top}k^-_{\ell,u,j}}{\sqrt{d_h}}.
$$
In the more detailed diagnostic selector, a unit receives
$$
U_{\ell,u} = \overline{a}_{\ell,u} + \xi\,\overline{m}^{+}_{\ell,u} - \chi\,\overline{m}^{x}_{\ell,u},
$$
and layers are ranked via
$$
U_{\ell} = \operatorname{AggTopK}_{u\in\mathcal{U}_{\ell}}(U_{\ell,u}; k).
$$
The selector artifact is frozen before held-out evaluation [2605.06225].

Because Llama-style and Qwen-style decoders use RoPE, a memory key stored at one absolute position should not remain tied to that construction position. MI therefore stores keys in canonical pre-RoPE coordinates and uses
$$
s^{mem}_{t,m} = \frac{\langle R_t^{-1}q_t,\;R_{\delta_m}\tilde k_m\rangle}{\sqrt{d_h}}
= \frac{\langle \bar q_t,\;R_{\delta_m}\tilde k_m\rangle}{\sqrt{d_h}}.
$$
The default is $\delta_m=0$, making memory slots position-agnostic. For grouped-query attention, banks are stored per KV head and expanded to the query heads that share the KV group [2605.06225].

The systems rationale is a KV-cache trade-off. If visible prompting stores guidance tokens at every layer, storage scales like $LT$ for $L$ layers and $T$ guidance tokens. MI instead stores a bank with $S_{\mathrm{bank}}$ slots only at $L_{\mathrm{ctrl}}$ selected layers:
$$
\mathrm{KV\ ratio} = \frac{L T}{L_{\mathrm{ctrl}} S_{\mathrm{bank}}}.
$$
For Qwen3-30B-A3B, the paper states $L=48$ and $L_{\mathrm{ctrl}}=5$; the equal-token/slot case yields $48/5=9.6$, and with content compression plus selected-layer placement the reported effective reductions reach up to $118\times$ for long structured guidance [2605.06225]. MI is therefore not “free” in an absolute sense, but it sharply reduces the amount of guidance content that must be resident across the full layer stack.

Implementation-wise, the ordinary prompt cache is left untouched, while the memory bank is implemented as a side KV bank consumed only at selected layers and units. In the Qwen3 setup, the intervention is integrated at the selected-layer attention path rather than by mutating vLLM’s native paged KV cache [2605.06225].

## 4. Experimental behavior: steering, updates, and structured reasoning

The experimental program for LLM MI is organized around personality steering, updateable guidance, and structured reasoning proxies [2605.06225]. Personality steering uses the IPIP-50 Big Five questionnaire and open-ended generation. In self-evaluation, the paper reports target shift relative to the plain model plus mean absolute non-target drift, summarized as $\mathrm{DAS}=\Delta_{\mathrm{target}}-D_{\mathrm{non\text{-}target}}$. In generation, GPT-4o-mini judges the presence of the target trait, opposite-trait pressure, coherence, and other trait signals, with an analogous metric $\mathrm{GAS}=\Delta_{\mathrm{target}}-D_{\mathrm{non\text{-}target}}$.

On Llama-3.1 self-evaluation, the average DAS is 0.775 for MI versus 0.645 for prompting and 0.185 for CAA. On Qwen3 self-evaluation, MI again beats CAA substantially, 0.270 versus 0.115, though prompting has the highest raw DAS at 0.355. In generation, MI improves over CAA on both backbones and is particularly strong on Qwen3, where the average GAS is 0.440 for MI versus 0.260 for prompting and -0.926 for CAA [2605.06225]. These results define the paper’s main claim: MI provides the best overall control–drift trade-off on matched personality-steering tasks while remaining competitive with prompting.

Updateable guidance is implemented as dialogue shift over 204 branching cases from 34 scenario roots, with 8, 16, and 24 turns and two branches per length [2605.06225]. On Qwen3 dialogue shift, MI reports overall post-shift alignment/quality of 0.816 / 7.75, compared with 0.438 / 8.54 for prompt-init and 0.526 / 8.44 for CAA. At 24 turns, MI still leads alignment on the matched-budget setup, with 0.651 versus 0.344 for prompt-init and 0.497 for CAA. In the Llama experiment, the picture is tighter: CAA slightly outperforms MI on overall alignment, 1.554 versus 1.516, while MI remains close [2605.06225]. The paper explicitly treats this as a limitation pattern rather than a universal win.

The structured reasoning proxies are HARDMath and PHYSICS. For PHYSICS, MI averages 76.9% overall versus 75.4% for prompting and 73.9% for the plain model, and it wins 10 of 12 subject$\times$mode cells. On HARDMath, prompt remains strongest overall at 67.3%, with MI at 63.8% and plain at 60.8% [2605.06225]. The paper’s interpretation is correspondingly nuanced: MI is especially useful for persistent, structured, or expensive-to-store guidance, but it is not uniformly superior to visible prompting on tasks where exact heuristic wording or raw prompt-faithful control can dominate.

The same section also reports that long-context persistence favors MI: the long-context persistence study shows that MI degrades less than prompting under long opposite-style prefills, especially on difficult traits like dismissive/anxious [2605.06225]. Bank quality, however, matters. Noisy, overspecific, or overly broad banks can fail, and the selector requires calibration examples rather than being a zero-design interface.

## 5. Recursive working memory, explicit memory, and amortized memory reuse

Outside LLM steering, the closest architectural analogue is the PMI framework, which explicitly describes a human-like “Memory Inception” process [2310.09297]. PMI factorizes computation into perception, memory, and inference:
$$
H = \mathcal{P}(X), \qquad
M^t = \mathcal{M}(H^{t-1}, M^{t-1}), \qquad
U = \mathcal{I}(H^{t-1}, M^t).
$$
Its memory module separates a limited-capacity working memory,
$$
M_w \in \mathbb{R}^{N \times D_m},
$$
from a higher-order long-term memory,
$$
M_l \in \mathbb{R}^{C \times N \times D_m}.
$$
Current perception is written into working memory through differentiable competitive write access based on multi-head sparse cross-attention, then consolidated into long-term memory by outer-product association:
$$
M_{l}^{t} = {LN}_{3} \left( (M_{w}^{t} \otimes M_{l}^{t-1}) + M_{l}^{t-1} \right).
$$
Inference retrieves separately from working memory and long-term memory and fuses the two sources:
$$
U^t = \alpha U_w^t + (1-\alpha)U_{wl}^t.
$$
The paper’s explicit claim is that this constitutes “memory within memory”: current interpretation is built upon prior memory traces, and those traces are updated by current interpretation [2310.09297].

PMI’s empirical results support the functional significance of this recursion. On bAbI-20k, PMI-TR achieves 2.55 ± 0.11% mean error and 2.32% best error, outperforming vanilla Transformer at 22.1%, TR+HSW at 3.6 ± 0.46%, DNC at 12.8 ± 4.7%, and LSTM at 27.3 ± 0.8%. On equilateral triangle detection, PMI-TR reaches 97.9% accuracy, about 8.1% improvement over the standard Transformer. On language modeling, PMI-TR reports 0.96 BPC on Enwik8, 16.5 test PPL on WikiText-103, and 31.04 test PPL on PG-19 [2310.09297].

A more literal memory substrate appears in in-memory few-shot continual learning with a dynamically evolving explicit memory [2207.06810]. There, a stationary controller, specifically a ResNet-12, is pretrained and metalearned in software, while a separate explicit memory stores one vector per class and expands online. Support examples are mapped to 1-bit quantized bipolar vectors; query examples are mapped to 8-bit quantized vectors; the controller is trained so that classes are mapped to 256-dimensional quasi-orthogonal vectors. The explicit memory is implemented on a Hermes PCM IMC core with a 256 × 256 unit-cell array. New classes allocate a fresh PCM column, later examples are superposed into the same class vector by in-situ progressive crystallization, and inference is performed by in-memory similarity search with
$$
\hat{y} = \arg\max_c s_c.
$$
On CIFAR-100 and miniImageNet, when continually learning 40 novel classes from only five examples per class on top of 60 old classes, the hardware accuracy remains within a range of 1.28%--2.5% compared to the state-of-the-art full-precision baseline software model [2207.06810]. In MI terms, adaptation resides in memory rather than in weight updates.

At the most theoretical end, Memory-Amortized Inference presents MI as the reuse of prior search trajectories after they have been consolidated into stable structure [2512.05990]. The framework’s formal definition is
$$
\Phi_{t+1} = \mathcal{F}(\Phi_t, \Psi_t), \qquad
\Phi_t \approx \mathcal{R}(\Phi_{t+1}, \Psi_t),
$$
with the “topological trinity” of Search $\to$ Closure $\to$ Structure. The paper’s final claim is that MAI is the operational form of Memory Inception: recursive search creates cycles, cycles close into structure, and structure becomes the substrate for future inference [2512.05990]. A plausible implication is that the term “Memory Inception” spans both concrete architectural mechanisms and broader theories of memory-based amortization.

## 6. Attack surfaces, non-standard usages, and conceptual limits

The same memory-native logic that makes MI useful for control and continual adaptation also creates attack surfaces. In multimodal agent systems, “Visual Inception” describes a long-term-memory poisoning attack in which a poisoned image acts as a sleeper agent inside an agentic recommender’s memory bank [2604.16966]. In a mock e-commerce agent environment, the attack achieves GHR = 85.1% and ASR-M = 82.3%. The proposed dual-process defense, CognitiveGuard, reduces GHR from 85.1% to 9.7% and ASR-M from 82.3% to 8.3%, while maintaining RCS = 0.89 [2604.16966]. Here the “inception” mechanism is delayed activation through memory retrieval rather than latent steering.

A related but distinct use appears in multi-turn jailbreaks against text-to-image systems with memory mechanisms [2504.20376]. There, “Inception” segments an unsafe prompt into benign-looking chunks and uses recursion to further subdivide chunks that remain unsafe, relying on memory to reassemble the malicious semantics across turns. The attacked system is formalized as
$$
\mathcal{I} = \mathcal{F}_{out}(\mathcal{M}(Mem(\mathcal{F}_{in}(\mathcal{Q})))),
$$
and on UnsafeDiff the method reaches ASR = 0.40 versus 0.26 for the second-best baseline, DACA, a +0.14 margin [2504.20376]. The broader lesson is that memory mechanisms amplify both alignment and vulnerability.

Another source of confusion is acronym overlap. In the medical-image-classification literature, “MI” denotes Multi-function Inception-V4 rather than Memory Inception, and the memory-efficient variant is CMI, the compressed multi-function Inception-V4 [1811.11996]. Standard Inception-V4 is decomposed into 149 convolutional blocks with RELU everywhere; MI keeps the same 149 blocks but replaces uniform RELU with activations drawn from $\{\text{RELU}, \text{SIG}, \text{TANH}, \text{ELU}\}$; CMI reduces block count and model size. For example, CMI\(_1\) has 58 convolutional blocks and model size 129 MB, CMI\(_2\) has 85 blocks and 190 MB, CMI\(_3\) has 112 blocks and 252 MB, while MI / original Inception-V4 is 323 MB [1811.11996]. This is an important terminological boundary: not every paper using the abbreviation “MI” is part of the Memory Inception lineage.

Across these usages, several limits recur. MI is not universally better than prompting; prompting often remains strongest when the objective is simply the largest visible-text movement or when exact heuristic wording matters [2605.06225]. MI is not parameter-free in the broader sense of systems design, because selector calibration, bank construction, and attention-path integration remain necessary [2605.06225]. And MI is not always a benign capability: persistent, structured, and retrievable memory can be exploited by poisoning and multi-turn jailbreaks [2604.16966], [2504.20376]. The cumulative evidence therefore supports a narrow but stable conclusion: Memory Inception is best understood as the relocation of guidance, knowledge, or attack payloads into explicit memory structures that can be selectively written, consolidated, queried, or reactivated later.

Source: https://www.emergentmind.com/topics/memory-inception-mi