Text2Mem: Textual Memory Operations

• Text2Mem is a framework that maps textual descriptions into explicit memory operations, facilitating context retrieval and editable, transactional memory updates.
• It spans diverse methodologies including associative memory, optimization-based writing, and dual-layer systems for conversational and multimodal applications.
• The approach balances trade-offs in capacity, compute cost, and fidelity while driving open research in benchmarking, hybrid retrieval, and formal memory control languages.

Searching arXiv for the cited Text2Mem-related papers to ground the article in current records. Text2Mem refers to systems that map textual descriptions of changes into explicit memory operations, and related work uses the term more broadly for mechanisms that read a context once, convert it into a compact memory state, and answer later queries from that state without re-accessing the original context (Li, 23 Jun 2026, Kuratov et al., 14 Mar 2026). In current arXiv literature, the label spans several technical families: explicit associative memories whose core state is writable and forgettable, optimization-based memory writers that adapt a compact state at test time, prompt-to-parameter adaptation schemes that treat a prompt as transient training data, application-layer episodic and semantic memory stacks for conversational or agent systems, and standardized languages for memory control across heterogeneous backends (Zanzotto et al., 18 Feb 2025, Zhu et al., 2024, Jena et al., 20 May 2026, Wang et al., 14 Sep 2025).

1. Scope, precursors, and historical development

Several earlier architectures already instantiate the central Text2Mem pattern: text is transformed into a memory substrate that is later read during generation. In neural machine translation, MemDec augments the decoder with a fixed-size external memory matrix $M_t^b \in \mathbb{R}^{n \times m}$, read and written once at each decoding step with content-based addressing, and improves Chinese–English translation by $4.8$ BLEU upon Groundhog and $5.3$ BLEU upon Moses (Wang et al., 2016). In long-document summarization, Mem2Mem introduces readable and writable external memories on both encoder and decoder sides, compresses an input article into a compact set of sentence representations, performs implicit extraction without labels, and attains results competitive with transformer-based summarization methods while using 16 times fewer parameters (Park et al., 2020).

The same pattern extends beyond pure text. VX2TEXT converts video, audio, speech, and text into language-space embeddings by modality-specific tokenizers, performs multimodal fusion in the language space, and generates open-ended text with a transformer encoder–decoder, outperforming prior methods on captioning, question answering, and audio-visual scene-aware dialog (Lin et al., 2021). A plausible implication is that Text2Mem is not restricted to textual inputs alone; it also covers pipelines in which non-textual signals are first rendered into language-like memory representations before retrieval or generation.

More recent work makes the memory interpretation explicit. MeMo proposes LLMs with explicit multi-layer correlation matrix memories (CMMs), FastMem performs “fast memorization of the prompt” by updating only the last FFN module before inference, GradMem writes long contexts into a fixed set of prefix memory tokens by test-time gradient descent, CALMem implements a dual memory outside the model, Metis structures agent experience into text and code memory, and Text2Mem itself is formalized as a memory operation language in a backend-agnostic control plane (Zanzotto et al., 18 Feb 2025, Zhu et al., 2024, Kuratov et al., 14 Mar 2026, Jena et al., 20 May 2026, Dai et al., 23 Jun 2026, Wang et al., 14 Sep 2025).

2. Associative-memory and transactional substrates

The most explicit architectural instantiation is MeMo. Its memory primitive is the classical correlation matrix memory

$C = \sum_i k_i v_i^{\top},$

with retrieval by

$\hat{v} = C^{\top} k.$

Multi-layer MeMo stacks several such memories,

$$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$

so that higher layers compose tokens into higher-level sequence representations and $C^{(\mathrm{out})}$ stores next-token associations (Zanzotto et al., 18 Feb 2025, Li, 23 Jun 2026).

At the interface level, MeMo exposes explicit architectural operations

$$\mathrm{memo}(S,y), \quad \mathrm{forget}(S,y), \quad \mathrm{retrieve}(S),$$

where $S$ is a sequence and $y$ is the associated next token or value. This is the critical departure from transformer-style implicit memorization: memorization, retrieval, and forgetting are callable operations rather than distributed side effects of gradient descent (Li, 23 Jun 2026).

The version-aware extension of Multi-layer MeMo turns these primitives into a transactional Text2Mem layer. A primitive edit is

$4.8$0

and a high-level operation is an ordered transaction

$4.8$1

The paper’s key claim is that a version-aware operation is rarely a single MeMo association; it is an ordered program of forgetting, memorizing, preserving historical chains, and recording an inverse program (Li, 23 Jun 2026).

This transaction model supports operations such as replace, obsolete, keep-history, rollback, and trace. For multi-token continuations, MeMo decomposes a continuation $4.8$2 into a chain of next-token associations, so a replacement is implemented as forgetting the old chain and memorizing the new one. History preservation is achieved by writing versioned sequences, so the “latest” view can suppress an outdated fact while version-conditioned queries still retrieve the old fact. Rollback is defined by reversing the transaction order and negating the signs:

$4.8$3

Two auxiliary CMMs turn this into a memory system for changes themselves. The Version CMM maps version transitions and slot indices to transaction handles, and the Transaction CMM maps transaction handles to executable change content and inverse programs. The framework also accepts structured diff-level inputs. A structured assertion $4.8$4 is serialized as

$4.8$5

so writing and removing an assertion are simply $4.8$6 and $4.8$7 (Li, 23 Jun 2026). In this formulation, Text2Mem is not merely “text to vector”; it is text to an auditable, reversible, version-aware edit program over explicit memories.

3. Optimization-based writing and parametric fast memory

A second family keeps the base LLM frozen and learns a per-context memory state at test time. GradMem studies the context-removal setting in which the model must answer queries without access to the original context after a write phase. Its memory is a fixed-size matrix of prefix memory tokens,

$4.8$8

prepended to the query during reading. Writing is driven by a self-supervised context reconstruction loss,

$4.8$9

and optimized by $5.3$0 steps of test-time gradient descent on the memory tokens alone:

$5.3$1

The model weights remain frozen; only the memory tokens are adapted for the current context (Kuratov et al., 14 Mar 2026).

This loss-driven write mechanism is empirically stronger than forward-only compression at fixed memory size. On associative key–value retrieval with 8 memory vectors, RMT reaches $5.3$2 at 16 pairs and $5.3$3 at 96 pairs, whereas GradMem with $5.3$4 reaches $5.3$5 at 16 pairs and $5.3$6 at 96 pairs, and GradMem with $5.3$7 reaches $5.3$8 at 16 pairs and $5.3$9 at 96 pairs. The same paper reports competitive transfer to bAbI, Short SQuAD, and WikiText-103, and emphasizes a compute–capacity knob: increasing WRITE steps at inference improves memory quality without retraining (Kuratov et al., 14 Mar 2026).

FastMem takes an explicitly parametric route. Instead of writing into prefix tokens, it updates only the last Transformer block’s FFN before inference to maximize the likelihood of the memorization text under a next-token-prediction objective with KL regularization:

$C = \sum_i k_i v_i^{\top},$0

$C = \sum_i k_i v_i^{\top},$1

All layers except the last FFN are frozen. The paper motivates this choice by treating upper FFNs as key–value memory modules and reports that adapting only the last block avoids severe overfitting while remaining fast (Zhu et al., 2024).

The quantitative gains are large on context-faithfulness tasks. FastMem improves the accuracy of Llama 3-8B-Inst on NQ-SWAP from $C = \sum_i k_i v_i^{\top},$2 to $C = \sum_i k_i v_i^{\top},$3, and reduces the output structure failure rate of Qwen 1.5-4B-Chat from $C = \sum_i k_i v_i^{\top},$4 to $C = \sum_i k_i v_i^{\top},$5. On 8×A800 GPUs with a 2500-token input, the memorization phase for Llama-3-8B-Instruct takes about $C = \sum_i k_i v_i^{\top},$6 s and does not increase peak GPU memory over standard forward passes (Zhu et al., 2024). The contrast with GradMem is instructive: GradMem writes into explicit prefix memory tokens, whereas FastMem writes the prompt into a narrowly localized parameter subset.

4. Application-layer conversational and agent memory

A third family moves Text2Mem entirely outside the model. CALMem is an application-layer dual memory for conversational AI with two subsystems: an episodic memory layer built on sliding-window vector embeddings of conversation history, and a semantic memory layer of agent-writable structured facts (Jena et al., 20 May 2026). Episodic memory indexes every stored message in the background using 1,000-character windows with 200-character overlap and 800-character step size, embeds each chunk with all-MiniLM-L6-v2 into $C = \sum_i k_i v_i^{\top},$7, and stores the vectors in SQLite. Retrieval embeds the query, computes cosine similarity, filters by a threshold of $C = \sum_i k_i v_i^{\top},$8, and returns the top-ranked chunks.

The crucial design point is intra-session retrieval. CALMem explicitly excludes chunks that correspond to messages still present in the active prompt and searches over compacted-away turns from the current session as well as prior sessions. This closes what the paper calls the compaction continuity problem: compacted context is no longer irretrievably lost merely because it left the active window (Jena et al., 20 May 2026).

Memory injection is handled by the MOIM (Message of Injected Memory), whose episodic budget decreases as the context fill ratio rises. For $C = \sum_i k_i v_i^{\top},$9, CALMem injects 5 chunks × 600 chars; for $\hat{v} = C^{\top} k.$0, 3 chunks × 400 chars; for $\hat{v} = C^{\top} k.$1, 2 chunks × 250 chars; and for $\hat{v} = C^{\top} k.$2, episodic injection is suppressed. Dense retrieval markedly outperforms sparse baselines on a manually labeled conversational retrieval set: TF-IDF yields P@5 $\hat{v} = C^{\top} k.$3, BM25 yields P@5 $\hat{v} = C^{\top} k.$4, and CALMem dense retrieval yields P@5 $\hat{v} = C^{\top} k.$5, R@5 $\hat{v} = C^{\top} k.$6, and MRR $\hat{v} = C^{\top} k.$7. Ablation over long sessions shows that episodic memory alone gives context recovery $\hat{v} = C^{\top} k.$8, semantic memory alone gives fact consistency $\hat{v} = C^{\top} k.$9, and CALMem’s dual memory gives context recovery $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$0 and fact consistency $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$1 (Jena et al., 20 May 2026).

Metis applies a related dual-memory idea to self-evolving agents, but with a different distinction: text memory versus code memory. Text memory is structured as

$$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$2

that is, environment facts, common pitfalls, and execution plans. Code memory is a library of validated callable tools, produced only when a plan shows repeated reuse. The paper’s controlled study isolates the trade-off: text memory is cheaper to construct and transfers better, whereas code memory is more execution-efficient but more expensive and more brittle if distilled too early (Dai et al., 23 Jun 2026).

Those trade-offs appear quantitatively in AppWorld. On the official split, Metis improves task goal completion from $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$3 for No Memory to $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$4, while reducing execution tokens from $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$5K to $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$6K. On the resampled split, it improves task goal completion from $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$7 to $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$8 and reduces execution tokens from $$\mathcal{C}_{\mathrm{MeMo}} = \{C^{(1)}, C^{(2)}, \ldots, C^{(L)}, C^{(\mathrm{out})}\},$$9K to $C^{(\mathrm{out})}$0K. The controlled profiling study further shows construction cost asymmetry—text memory requiring $C^{(\mathrm{out})}$1M reflection tokens and 224 rounds versus code memory requiring $C^{(\mathrm{out})}$2M tokens and 562 rounds—and transfer asymmetry, with text dropping from about $C^{(\mathrm{out})}$3 to $C^{(\mathrm{out})}$4 from Oracle to Streaming, whereas code drops from about $C^{(\mathrm{out})}$5 to $C^{(\mathrm{out})}$6 (Dai et al., 23 Jun 2026). In Metis, Text2Mem is therefore a staged process: raw execution traces become structured textual memory first, and only recurrent plans are crystallized into code.

5. Formal memory control languages and execution contracts

The most explicit control-plane formalization is the unified memory operation language Text2Mem. It defines 12 operations organized by stage: Encode; storage operations Update, Label, Promote, Demote, Merge, Delete, Split, Lock, Expire; and retrieval operations Retrieve and Summarize (Wang et al., 14 Sep 2025). Each instruction is represented as a JSON object with fields stage, op, target, args, and optional meta. The schema uses conditional requirements and invariants: for example, Encode requires args.payload, Promote requires at least one of priority, weight_delta, or remind, Expire requires ttl or until, locked items cannot be hard-deleted, and Merge must preserve lineage.

Execution proceeds through a validator–parser–adapter pipeline. The validator checks structural constraints, allowed values, and semantic invariants before execution. The parser converts a valid JSON instance into a typed operation object with normalized parameters such as time, tags, and priority. Adapters then map the typed object either to a SQL prototype backend or to real memory frameworks, invoking embeddings or summarization services when required. Results are returned through a unified execution contract, ExecutionResult, which reports status, affected IDs, retrieved content, diffs, and metadata (Wang et al., 14 Sep 2025).

The paper explicitly frames this move as analogous to text-to-SQL: natural language is no longer interpreted directly against a backend-specific memory API, but first translated into a formal executable language. The planned Text2Mem Bench correspondingly separates NL→schema generation from schema→execution, with proposed metrics including schema validity, slot accuracy, key–value F1, executability, state diffs, ranking shifts, trigger correctness, and cross-backend consistency (Wang et al., 14 Sep 2025). Relative to transaction-aware MeMo, which formalizes low-level edit programs over explicit associative memories, this language operates one abstraction level above: it standardizes how memory commands are specified, validated, and ported across systems.

6. Trade-offs, limitations, and open directions

The literature converges on a recurring set of trade-offs. Explicit memory improves editability, traceability, and locality, but it does not eliminate capacity management or write-time cost. In version-aware MeMo, the scope is explicitly limited to “local changes that can be expressed as MeMo-compatible memory edits”; global corpus-wide shifts are out of scope and would still require training, and the paper gives no formal locality guarantees (Li, 23 Jun 2026). In CALMem, exact brute-force cosine scan is deliberately retained only up to about $C^{(\mathrm{out})}$7k chunks, with HNSW or external vector databases deferred until scale exceeds about $C^{(\mathrm{out})}$8k chunks (Jena et al., 20 May 2026).

Optimization-based writers introduce a different bottleneck. GradMem’s WRITE phase requires $C^{(\mathrm{out})}$9 forward+backward passes over the context, and training relies on second-order meta-learning through the write loop; the paper presents this as the main training cost and notes that one-off queries per context do not amortize the write overhead (Kuratov et al., 14 Mar 2026). FastMem is cheaper because it updates only the last FFN, but it still assumes that the memorized reference is correct and relevant, and the paper explicitly warns that in RAG settings noisy or malicious references may be followed more faithfully after memorization (Zhu et al., 2024).

Metis exposes a representation-level trade-off that is likely general. Text memory is broadly applicable and degrades gracefully because the agent can reinterpret or ignore it; code memory is execution-efficient but expensive to construct and brittle if promoted from single trajectories. These results suggest that a robust Text2Mem system should not decide between text and code at design time alone; it should also condition that decision on recurrence, stability, and validation cost (Dai et al., 23 Jun 2026). A similar inference applies across the field: explicit memories are most attractive when updateability, rollback, and provenance matter, whereas prompt- or parameter-based memories are most attractive when the base model must stay fixed and the same context is reused for many queries.

Open directions are stated directly in several papers. Version-aware MeMo calls for benchmarks on update success, rollback, traceability, locality, and transaction reuse; GradMem proposes better write objectives and more efficient meta-learning; FastMem points to multi-layer PEFT-style variants and per-example FFN copies; CALMem identifies hybrid lexical+dense retrieval, fact expiry, and temporal reasoning; Metis highlights better program synthesis and more principled criteria for crystallization; and the memory operation language paper proposes a larger, multilingual Text2Mem Bench and richer adapters (Li, 23 Jun 2026, Kuratov et al., 14 Mar 2026, Zhu et al., 2024, Jena et al., 20 May 2026, Dai et al., 23 Jun 2026, Wang et al., 14 Sep 2025). Taken together, these directions indicate that Text2Mem has become less a single mechanism than a design space organized around one question: how textual experience should be turned into memory objects, memory state, or memory operations that remain editable, economical, and useful over long horizons.