---
title: 'ENGRAM-R: Inference-Time Memory Layer'
url: https://www.emergentmind.com/topics/engram-r
type: topic
---

# ENGRAM-R: Inference-Time Memory Layer

ENGRAM-R is an inference-time memory layer for large reasoning models that is designed to reduce the cost of test-time reasoning by reusing retrieved structured memory instead of repeatedly recomputing long chains of thought [2511.12987]. It extends the earlier ENGRAM conversational-memory framework, which organizes dialogue-derived evidence into episodic, semantic, and procedural stores, by adding Compact Fact Cards and explicit citation control [2511.12960]. The system is model-agnostic, does not modify model weights, and operates as an external layer that organizes, retrieves, compresses, and constrains evidence before answer generation [2511.12987].

## 1. Conceptual scope and formal setting

ENGRAM-R is formulated for efficient long-horizon reasoning in dialogue. A dialogue is represented as
\[
\mathcal{C} = \{x_t\}_{t=1}^{T}, \qquad x_t = (s_t, u_t, \tau_t),
\]
where \(s_t\) is speaker identity, \(u_t\) is the utterance text, and \(\tau_t\) is a timestamp or temporal marker [2511.12987]. The central problem is that full-context inference scales poorly as dialogue histories grow, even when the required answer depends on a comparatively small set of recurring facts, events, or procedures.

The memory state is typed:
\[
\mathcal{M} = \big(\mathcal{M}_{\mathrm{epi}}, \mathcal{M}_{\mathrm{sem}}, \mathcal{M}_{\mathrm{pro}}\big),
\]
with episodic, semantic, and procedural stores [2511.12987]. This typed decomposition is inherited from ENGRAM, whose motivating claim is that long-horizon conversational consistency improves when memory competition is separated by functional role rather than handled in a single undifferentiated store [2511.12960].

The system’s optimization target is not expressed as a single scalar objective. Instead, the paper evaluates ENGRAM-R along two axes: semantic correctness, measured by an LLM-as-Judge protocol, and efficiency, measured through input tokens, reasoning tokens, and latency [2511.12987]. This framing places ENGRAM-R closer to memory orchestration than to conventional retrieval-augmented generation tuned only for answer quality.

## 2. Lineage from ENGRAM and the five-stage pipeline

Base ENGRAM uses a write-time and query-time pipeline built around a lightweight router, typed extraction, dense retrieval, and prompt construction [2511.12960]. ENGRAM-R retains that backbone and adds two explicit reasoning-oriented extensions: Fact Card rendering and citation control [2511.12987].

At write time, each incoming utterance is routed into one or more memory types with a 3-bit output:
\[
r(u_t)\in\{0,1\}^3 \Rightarrow b_t=\big(b_t^{\mathrm{epi}},\,b_t^{\mathrm{sem}},\,b_t^{\mathrm{pro}}\big).
\]
For each selected type, the system constructs a normalized record and an embedding \(e \in \mathbb{R}^d\) using an encoder
\[
g:\mathcal{U}\to\mathbb{R}^d,
\]
then stores records and vectors in a SQLite relational store keyed by conversation and type [2511.12987].

At query time, ENGRAM-R executes a five-stage pipeline: routing, extraction plus embedding, typed retrieval, Fact Card rendering, and prompt construction with citation control [2511.12987]. The first three stages are inherited from ENGRAM. The last two are the distinctive ENGRAM-R additions, intended to reduce what the paper treats as the “evidence narration burden” of large reasoning models.

A central architectural claim is that ENGRAM-R is not a new trained model. The answering LRM remains frozen, and the system functions purely as an inference-time augmentation layer [2511.12987]. In the reported implementation, the answering model is `gpt-oss-20b`, while the memory layer uses `gpt-4o-mini` as a lightweight backbone for memory operations [2511.12987].

## 3. Typed memory representation and evidence structures

The three memory stores are defined by normalized schemas [2511.12987].

| Type | Schema | Role |
|---|---|---|
| Episodic | \(m^{\mathrm{epi}}=(t,\sigma,\delta,e)\) | Event title, brief summary, time anchor, embedding |
| Semantic | \(m^{\mathrm{sem}}=(f,\delta,e)\) | Fact string, time anchor, embedding |
| Procedural | \(m^{\mathrm{pro}}=(t,c,\delta,e)\) | Procedure title, normalized content, time anchor, embedding |

Here \(t\) denotes a title, \(\sigma\) a summary, \(f\) a fact string, \(c\) normalized procedure content, \(\delta\) a temporal anchor or provenance marker, and \(e\) the embedding [2511.12987]. The typed split is meant to reduce competition among heterogeneous records and to preserve temporal and functional distinctions before final truncation.

ENGRAM-R’s defining representational move is the Fact Card:
\[
\phi(m) = \big[\text{id}(m),\; \text{claim}(m),\; \text{anchor}(m)\big].
\]
Each card contains a stable citation identifier such as \([E1]\), a minimal canonical claim, and an anchor such as a timestamp or provenance marker [2511.12987]. The retrieved memory set is transformed into a fact-card evidence state
\[
\mathcal{F}(q) = \{\phi(m)\mid m\in \tilde{R}(q)\}.
\]

The paper describes Fact Cards as atomic, non-redundant claims tied to explicit provenance [2511.12987]. This representation is intended to compress multi-sentence snippets into concise evidence units while preserving auditability. A plausible implication is that ENGRAM-R treats evidence compression as part of the memory interface itself rather than as an incidental prompt-formatting choice.

## 4. Retrieval, aggregation, and citation-controlled prompting

Typed retrieval begins by embedding the query:
\[
e_q = g(q).
\]
For each memory store \(k \in \{\mathrm{epi}, \mathrm{sem}, \mathrm{pro}\}\), the system retrieves top-\(k\) candidates using dense similarity:
\[
R_k(q)=\operatorname{TopK}\{\mathrm{score}(e_q,m)\mid m\in\mathcal{M}_k\}.
\]
The score is described as cosine similarity [2511.12987]. Retrieval is therefore performed separately within episodic, semantic, and procedural stores rather than over a single mixed pool.

The type-local candidate sets are merged, deduplicated, and truncated to a fixed evidence budget:
\[
\tilde{R}(q)=\operatorname{Truncate}_K\!\Big(\operatorname{Dedup}\big(\!\bigcup\nolimits_k R_k(q)\big)\Big),
\]
with default
\[
K = 25.
\]
The paper states that \(K=25\) was chosen as a knee point from a \(K\)-sweep in the prior ENGRAM work [2511.12987].

ENGRAM-R then replaces ENGRAM’s raw snippet serialization with Fact Cards and builds the final prompt as
\[
P(q)=\mathrm{Template}\!\big(q,\mathcal{F}(q)\big).
\]
A citation-validity constraint is imposed:
\[
\hat{a}\Rightarrow \big\{[E_i]\mid \phi(m_i)\in \mathcal{F}(q)\big\},
\]
meaning any citation emitted in the answer must correspond to an actually retrieved Fact Card [2511.12987].

The prompting template operationalizes this constraint by instructing the model to answer concisely in 1–3 sentences, cite supporting facts with labels such as \([E1]\), refer to the question as “Q1” inside the reasoning chain, and never repeat full fact content [2511.12987]. This design is intended to minimize reasoning-token expenditure by encouraging reference to evidence IDs rather than free-form re-description.

A common misconception is to equate ENGRAM-R with ordinary RAG. The paper argues against that equivalence on architectural grounds. Standard RAG typically returns raw snippets or passages and leaves synthesis unconstrained, whereas ENGRAM-R uses typed memory stores, a fixed evidence budget, compact Fact Cards, and explicit citation control [2511.12987]. Another misconception is that ENGRAM-R trains or edits the answering model; the paper is explicit that the answering-model weights remain frozen [2511.12987].

## 5. Empirical performance on long-horizon reasoning

The main benchmarks are LoCoMo and LongMemEval\(_S\) [2511.12987]. LoCoMo comprises 10 dialogues, about 600 turns per dialogue on average, roughly 16K tokens per dialogue, and up to 32 sessions. LongMemEval\(_S\) uses histories around 115K tokens per problem and focuses on multi-session and temporal reasoning [2511.12987].

On LoCoMo, ENGRAM-R reduces overall input tokens from 28,371,703 to 3,293,478 and reasoning tokens from 1,335,988 to 378,424, while judge accuracy moves from 77.5% for full-context inference to 75.6% [2511.12987]. The trade-off is not uniform across categories. On single-hop questions, accuracy drops from 84.7% to 79.1%; on open-domain questions, from 64.6% to 57.2%. By contrast, ENGRAM-R improves multi-hop accuracy from 72.0% to 74.5% and temporal accuracy from 67.3% to 69.2% while using far fewer tokens [2511.12987].

On LongMemEval\(_S\), the gains are stronger in both efficiency and quality. Overall input tokens fall from 27,483,912 to 1,233,256 and reasoning tokens from 245,125 to 54,301, while judge accuracy rises from 38.0% to 59.8% [2511.12987]. In the multi-session slice, accuracy increases from 36.8% to 66.9%; in the temporal slice, from 39.1% to 52.6% [2511.12987]. These numbers support the paper’s claim that compact typed memory can outperform brute-force full-context prompting on extremely long interaction histories.

Latency measurements are reported in p50 and p95 terms. On LoCoMo, full-context total latency is 7.89s p50 and 17.16s p95; ENGRAM-R reports search latency of 1.04s p50 and 2.67s p95, with total latency of 2.56s p50 and 7.43s p95 [2511.12987]. On LongMemEval\(_S\), full-context latency is 9.62s p50 and 21.47s p95, while ENGRAM-R reports 0.72s/1.18s search latency and 1.88s/5.54s total latency [2511.12987].

The paper therefore treats ENGRAM-R less as a universal answer-quality maximizer than as an efficiency-oriented reasoning layer whose strongest benefits appear in multi-hop, temporal, and very long-horizon settings [2511.12987].

## 6. Ablations, failure modes, and methodological limits

The main ablation compares ENGRAM Base with ENGRAM-R [2511.12987]. Both systems use the same typed stores, router, dense retriever, and \(K=25\); ENGRAM-R adds Fact Cards and citation enforcement. On LoCoMo, ENGRAM Base uses 3,112,497 input tokens, 723,997 reasoning tokens, and achieves 73.5% judge accuracy. ENGRAM-R uses 3,293,478 input tokens, 378,424 reasoning tokens, and reaches 74.6% in the appendix table, though the main LoCoMo table reports 75.6% [2511.12987]. The paper does not explain this discrepancy. The ablation conclusion is nonetheless consistent: Fact Cards and citation control nearly halve reasoning tokens and modestly improve quality overall.

The category-level ablation pattern is asymmetric. The paper reports similar single-hop accuracy with much shorter reasoning, gains of +7.9% on multi-hop and +7.7% on temporal reasoning, and a substantial accuracy drop on open-domain questions [2511.12987]. This suggests that the compact evidence interface is most effective when the primary bottleneck is composition of long-range in-dialogue evidence rather than recovery of broader background knowledge.

Several limitations are explicitly acknowledged. Small evidence budgets may miss decisive facts; large budgets weaken efficiency [2511.12987]. External memory can become stale or drift without refresh policies or validity intervals [2511.12987]. Evaluation relies on LLM-as-Judge, which the paper treats as imperfect and potentially improvable through human assessment [2511.12987]. The paper also does not provide a fully factorized ablation separating typed retrieval, Fact Cards alone, and citation control alone as independent contributors [2511.12987].

A further methodological limit is incomplete implementation transparency. The paper states the use of dense retrieval, typed stores, fixed \(K=25\), SQLite relational memory, and frozen answering-model weights, but does not specify the exact embedding model, exact retrieval implementation library, decoding hyperparameters, or the number of judge runs and confidence intervals [2511.12987]. For researchers, this means the system is conceptually clear but not fully specified at the lower-level implementation boundary.

## 7. Terminology, adjacent “Engram” systems, and scope boundaries

ENGRAM-R belongs to a broader family of systems using the term “Engram,” but the surrounding literature is heterogeneous. The most immediate predecessor is ENGRAM, a lightweight memory system for conversational agents that uses the same three canonical memory types—episodic, semantic, and procedural—organized through a single router and retriever [2511.12960]. ENGRAM-R extends this framework with a reasoning-focused evidence interface rather than replacing it.

Other contemporaneous uses of “Engram” refer to different architectures. In large language model pretraining, Engram denotes a hash-keyed conditional-memory module that modernizes classic \(N\)-gram embedding for \(O(1)\) lookup and is analyzed as a new sparsity axis complementary to MoE [2601.07372]. In autoregressive image generation, an Engram module with 2D spatial \(n\)-gram hashing is studied mechanistically and found to behave not as a content-addressed retriever but as a gated architectural side-pathway [2605.13179]. In personalization, “User as Engram” stores per-user content as local edits to a hash-keyed memory table while keeping reasoning skill in a shared adapter [2606.19172]. These systems are architecturally distinct from ENGRAM-R, which is an inference-time memory orchestration layer for large reasoning models rather than a trainable conditional-memory block or a parametric personalization method.

This terminological spread matters because “ENGRAM-R” can be misread as a generic synonym for Engram-style retrieval. In the literature summarized here, however, ENGRAM-R has a specific meaning: typed retrieval, fixed-budget evidence aggregation, Fact Card rendering, and explicit citation control for efficient long-horizon reasoning [2511.12987]. Its contribution is not a new memory substrate in the pretraining sense, nor a claim about mechanistic content-addressed recall in the conditional-memory sense, but a concrete demonstration that memory organization itself can reduce reasoning-token expenditure while preserving or improving accuracy in long-horizon dialogue tasks [2511.12987].

Source: https://www.emergentmind.com/topics/engram-r