ENGRAM-R: Inference-Time Memory Layer

• ENGRAM-R is an inference-time memory layer that organizes dialogue evidence into episodic, semantic, and procedural stores for efficient long-horizon reasoning.
• It introduces Fact Cards and explicit citation control to compress multi-sentence evidence into compact, audit-friendly units.
• Empirical results show that ENGRAM-R reduces token usage and latency while improving multi-hop and temporal reasoning accuracy in extended dialogues.

ENGRAM-R is an inference-time memory layer for large reasoning models that is designed to reduce the cost of test-time reasoning by reusing retrieved structured memory instead of repeatedly recomputing long chains of thought (Patel et al., 17 Nov 2025). It extends the earlier ENGRAM conversational-memory framework, which organizes dialogue-derived evidence into episodic, semantic, and procedural stores, by adding Compact Fact Cards and explicit citation control (Patel et al., 17 Nov 2025). The system is model-agnostic, does not modify model weights, and operates as an external layer that organizes, retrieves, compresses, and constrains evidence before answer generation (Patel et al., 17 Nov 2025).

1. Conceptual scope and formal setting

ENGRAM-R is formulated for efficient long-horizon reasoning in dialogue. A dialogue is represented as

$$\mathcal{C} = \{x_t\}_{t=1}^{T}, \qquad x_t = (s_t, u_t, \tau_t),$$

where $s_t$ is speaker identity, $u_t$ is the utterance text, and $\tau_t$ is a timestamp or temporal marker (Patel et al., 17 Nov 2025). The central problem is that full-context inference scales poorly as dialogue histories grow, even when the required answer depends on a comparatively small set of recurring facts, events, or procedures.

The memory state is typed: $$\mathcal{M} = \big(\mathcal{M}_{\mathrm{epi}}, \mathcal{M}_{\mathrm{sem}}, \mathcal{M}_{\mathrm{pro}}\big),$$ with episodic, semantic, and procedural stores (Patel et al., 17 Nov 2025). This typed decomposition is inherited from ENGRAM, whose motivating claim is that long-horizon conversational consistency improves when memory competition is separated by functional role rather than handled in a single undifferentiated store (Patel et al., 17 Nov 2025).

The system’s optimization target is not expressed as a single scalar objective. Instead, the paper evaluates ENGRAM-R along two axes: semantic correctness, measured by an LLM-as-Judge protocol, and efficiency, measured through input tokens, reasoning tokens, and latency (Patel et al., 17 Nov 2025). This framing places ENGRAM-R closer to memory orchestration than to conventional retrieval-augmented generation tuned only for answer quality.

2. Lineage from ENGRAM and the five-stage pipeline

Base ENGRAM uses a write-time and query-time pipeline built around a lightweight router, typed extraction, dense retrieval, and prompt construction (Patel et al., 17 Nov 2025). ENGRAM-R retains that backbone and adds two explicit reasoning-oriented extensions: Fact Card rendering and citation control (Patel et al., 17 Nov 2025).

At write time, each incoming utterance is routed into one or more memory types with a 3-bit output: $$r(u_t)\in\{0,1\}^3 \Rightarrow b_t=\big(b_t^{\mathrm{epi}},\,b_t^{\mathrm{sem}},\,b_t^{\mathrm{pro}}\big).$$ For each selected type, the system constructs a normalized record and an embedding $e \in \mathbb{R}^d$ using an encoder

$g:\mathcal{U}\to\mathbb{R}^d,$

then stores records and vectors in a SQLite relational store keyed by conversation and type (Patel et al., 17 Nov 2025).

At query time, ENGRAM-R executes a five-stage pipeline: routing, extraction plus embedding, typed retrieval, Fact Card rendering, and prompt construction with citation control (Patel et al., 17 Nov 2025). The first three stages are inherited from ENGRAM. The last two are the distinctive ENGRAM-R additions, intended to reduce what the paper treats as the “evidence narration burden” of large reasoning models.

A central architectural claim is that ENGRAM-R is not a new trained model. The answering LRM remains frozen, and the system functions purely as an inference-time augmentation layer (Patel et al., 17 Nov 2025). In the reported implementation, the answering model is gpt-oss-20b, while the memory layer uses gpt-4o-mini as a lightweight backbone for memory operations (Patel et al., 17 Nov 2025).

3. Typed memory representation and evidence structures

The three memory stores are defined by normalized schemas (Patel et al., 17 Nov 2025).

Type	Schema	Role
Episodic	$m^{\mathrm{epi}}=(t,\sigma,\delta,e)$	Event title, brief summary, time anchor, embedding
Semantic	$m^{\mathrm{sem}}=(f,\delta,e)$	Fact string, time anchor, embedding
Procedural	$s_t$0	Procedure title, normalized content, time anchor, embedding

Here $s_t$1 denotes a title, $s_t$2 a summary, $s_t$3 a fact string, $s_t$4 normalized procedure content, $s_t$5 a temporal anchor or provenance marker, and $s_t$6 the embedding (Patel et al., 17 Nov 2025). The typed split is meant to reduce competition among heterogeneous records and to preserve temporal and functional distinctions before final truncation.

ENGRAM-R’s defining representational move is the Fact Card: $s_t$7 Each card contains a stable citation identifier such as $s_t$8, a minimal canonical claim, and an anchor such as a timestamp or provenance marker (Patel et al., 17 Nov 2025). The retrieved memory set is transformed into a fact-card evidence state

$s_t$9

The paper describes Fact Cards as atomic, non-redundant claims tied to explicit provenance (Patel et al., 17 Nov 2025). This representation is intended to compress multi-sentence snippets into concise evidence units while preserving auditability. A plausible implication is that ENGRAM-R treats evidence compression as part of the memory interface itself rather than as an incidental prompt-formatting choice.

4. Retrieval, aggregation, and citation-controlled prompting

Typed retrieval begins by embedding the query: $u_t$0 For each memory store $u_t$1, the system retrieves top-$u_t$2 candidates using dense similarity: $u_t$3 The score is described as cosine similarity (Patel et al., 17 Nov 2025). Retrieval is therefore performed separately within episodic, semantic, and procedural stores rather than over a single mixed pool.

The type-local candidate sets are merged, deduplicated, and truncated to a fixed evidence budget: $u_t$4 with default

$u_t$5

The paper states that $u_t$6 was chosen as a knee point from a $u_t$7-sweep in the prior ENGRAM work (Patel et al., 17 Nov 2025).

ENGRAM-R then replaces ENGRAM’s raw snippet serialization with Fact Cards and builds the final prompt as

$u_t$8

A citation-validity constraint is imposed: $u_t$9 meaning any citation emitted in the answer must correspond to an actually retrieved Fact Card (Patel et al., 17 Nov 2025).

The prompting template operationalizes this constraint by instructing the model to answer concisely in 1–3 sentences, cite supporting facts with labels such as $\tau_t$0, refer to the question as “Q1” inside the reasoning chain, and never repeat full fact content (Patel et al., 17 Nov 2025). This design is intended to minimize reasoning-token expenditure by encouraging reference to evidence IDs rather than free-form re-description.

A common misconception is to equate ENGRAM-R with ordinary RAG. The paper argues against that equivalence on architectural grounds. Standard RAG typically returns raw snippets or passages and leaves synthesis unconstrained, whereas ENGRAM-R uses typed memory stores, a fixed evidence budget, compact Fact Cards, and explicit citation control (Patel et al., 17 Nov 2025). Another misconception is that ENGRAM-R trains or edits the answering model; the paper is explicit that the answering-model weights remain frozen (Patel et al., 17 Nov 2025).

5. Empirical performance on long-horizon reasoning

The main benchmarks are LoCoMo and LongMemEval$\tau_t$1 (Patel et al., 17 Nov 2025). LoCoMo comprises 10 dialogues, about 600 turns per dialogue on average, roughly 16K tokens per dialogue, and up to 32 sessions. LongMemEval$\tau_t$2 uses histories around 115K tokens per problem and focuses on multi-session and temporal reasoning (Patel et al., 17 Nov 2025).

On LoCoMo, ENGRAM-R reduces overall input tokens from 28,371,703 to 3,293,478 and reasoning tokens from 1,335,988 to 378,424, while judge accuracy moves from 77.5% for full-context inference to 75.6% (Patel et al., 17 Nov 2025). The trade-off is not uniform across categories. On single-hop questions, accuracy drops from 84.7% to 79.1%; on open-domain questions, from 64.6% to 57.2%. By contrast, ENGRAM-R improves multi-hop accuracy from 72.0% to 74.5% and temporal accuracy from 67.3% to 69.2% while using far fewer tokens (Patel et al., 17 Nov 2025).

On LongMemEval$\tau_t$3, the gains are stronger in both efficiency and quality. Overall input tokens fall from 27,483,912 to 1,233,256 and reasoning tokens from 245,125 to 54,301, while judge accuracy rises from 38.0% to 59.8% (Patel et al., 17 Nov 2025). In the multi-session slice, accuracy increases from 36.8% to 66.9%; in the temporal slice, from 39.1% to 52.6% (Patel et al., 17 Nov 2025). These numbers support the paper’s claim that compact typed memory can outperform brute-force full-context prompting on extremely long interaction histories.

Latency measurements are reported in p50 and p95 terms. On LoCoMo, full-context total latency is 7.89s p50 and 17.16s p95; ENGRAM-R reports search latency of 1.04s p50 and 2.67s p95, with total latency of 2.56s p50 and 7.43s p95 (Patel et al., 17 Nov 2025). On LongMemEval$\tau_t$4, full-context latency is 9.62s p50 and 21.47s p95, while ENGRAM-R reports 0.72s/1.18s search latency and 1.88s/5.54s total latency (Patel et al., 17 Nov 2025).

The paper therefore treats ENGRAM-R less as a universal answer-quality maximizer than as an efficiency-oriented reasoning layer whose strongest benefits appear in multi-hop, temporal, and very long-horizon settings (Patel et al., 17 Nov 2025).

6. Ablations, failure modes, and methodological limits

The main ablation compares ENGRAM Base with ENGRAM-R (Patel et al., 17 Nov 2025). Both systems use the same typed stores, router, dense retriever, and $\tau_t$5; ENGRAM-R adds Fact Cards and citation enforcement. On LoCoMo, ENGRAM Base uses 3,112,497 input tokens, 723,997 reasoning tokens, and achieves 73.5% judge accuracy. ENGRAM-R uses 3,293,478 input tokens, 378,424 reasoning tokens, and reaches 74.6% in the appendix table, though the main LoCoMo table reports 75.6% (Patel et al., 17 Nov 2025). The paper does not explain this discrepancy. The ablation conclusion is nonetheless consistent: Fact Cards and citation control nearly halve reasoning tokens and modestly improve quality overall.

The category-level ablation pattern is asymmetric. The paper reports similar single-hop accuracy with much shorter reasoning, gains of +7.9% on multi-hop and +7.7% on temporal reasoning, and a substantial accuracy drop on open-domain questions (Patel et al., 17 Nov 2025). This suggests that the compact evidence interface is most effective when the primary bottleneck is composition of long-range in-dialogue evidence rather than recovery of broader background knowledge.

Several limitations are explicitly acknowledged. Small evidence budgets may miss decisive facts; large budgets weaken efficiency (Patel et al., 17 Nov 2025). External memory can become stale or drift without refresh policies or validity intervals (Patel et al., 17 Nov 2025). Evaluation relies on LLM-as-Judge, which the paper treats as imperfect and potentially improvable through human assessment (Patel et al., 17 Nov 2025). The paper also does not provide a fully factorized ablation separating typed retrieval, Fact Cards alone, and citation control alone as independent contributors (Patel et al., 17 Nov 2025).

A further methodological limit is incomplete implementation transparency. The paper states the use of dense retrieval, typed stores, fixed $\tau_t$6, SQLite relational memory, and frozen answering-model weights, but does not specify the exact embedding model, exact retrieval implementation library, decoding hyperparameters, or the number of judge runs and confidence intervals (Patel et al., 17 Nov 2025). For researchers, this means the system is conceptually clear but not fully specified at the lower-level implementation boundary.

7. Terminology, adjacent “Engram” systems, and scope boundaries

ENGRAM-R belongs to a broader family of systems using the term “Engram,” but the surrounding literature is heterogeneous. The most immediate predecessor is ENGRAM, a lightweight memory system for conversational agents that uses the same three canonical memory types—episodic, semantic, and procedural—organized through a single router and retriever (Patel et al., 17 Nov 2025). ENGRAM-R extends this framework with a reasoning-focused evidence interface rather than replacing it.

Other contemporaneous uses of “Engram” refer to different architectures. In LLM pretraining, Engram denotes a hash-keyed conditional-memory module that modernizes classic $\tau_t$7-gram embedding for $\tau_t$8 lookup and is analyzed as a new sparsity axis complementary to MoE (Cheng et al., 12 Jan 2026). In autoregressive image generation, an Engram module with 2D spatial $\tau_t$9-gram hashing is studied mechanistically and found to behave not as a content-addressed retriever but as a gated architectural side-pathway (Wang et al., 13 May 2026). In personalization, “User as Engram” stores per-user content as local edits to a hash-keyed memory table while keeping reasoning skill in a shared adapter (Li, 17 Jun 2026). These systems are architecturally distinct from ENGRAM-R, which is an inference-time memory orchestration layer for large reasoning models rather than a trainable conditional-memory block or a parametric personalization method.

This terminological spread matters because “ENGRAM-R” can be misread as a generic synonym for Engram-style retrieval. In the literature summarized here, however, ENGRAM-R has a specific meaning: typed retrieval, fixed-budget evidence aggregation, Fact Card rendering, and explicit citation control for efficient long-horizon reasoning (Patel et al., 17 Nov 2025). Its contribution is not a new memory substrate in the pretraining sense, nor a claim about mechanistic content-addressed recall in the conditional-memory sense, but a concrete demonstration that memory organization itself can reduce reasoning-token expenditure while preserving or improving accuracy in long-horizon dialogue tasks (Patel et al., 17 Nov 2025).