--- title: Memento 3 Explained url: https://www.emergentmind.com/topics/memento-3 type: topic --- # Memento 3 Explained Memento 3 is a model-based architecture for continual learning in unfamiliar environments that enables a frozen large language model (LLM) to construct, revise, and execute explicit world models through external memory. Its persistent memory is a natural-language rulebook containing hypotheses about entities, action semantics, transition dynamics, hidden state variables, goals, and termination conditions. The rulebook is compiled into executable code for prediction and planning; discrepancies between predicted and observed transitions trigger reflection, rule revision, code repair, recompilation, and exact replay verification. The underlying LLM parameters remain fixed, while the agent’s external learning state, world model, planner, and controller evolve through interaction [2610.11794]. ## 1. Conceptual foundations and objectives Memento 3 extends earlier memory-based Memento systems from policy adaptation to world-model adaptation. Earlier systems primarily retained interaction outcomes, successful actions, or reusable skills. Memento 3 instead records generalized hypotheses about how an environment works. Its central design principle is **Code as Model**: semantic rules are maintained in natural language and operationalized as executable transition models. The system addresses environments in which limited observations are compatible with multiple hypotheses. For example, repeated observations of an object stopping at column 7 are consistent both with a rule that the object stops at a wall and with a rule that it stops whenever it reaches column 7. These hypotheses make different predictions in unobserved states. Exact replay of prior observations can eliminate inconsistent models, but cannot alone identify the correct generalization among all replay-consistent alternatives. Memento 3 therefore distinguishes three categories of knowledge: - **Known mechanics**: rules supported by observations and represented explicitly. - **Unknown mechanics**: aspects deliberately left underspecified. - **Revisable hypotheses**: rules that may be retracted or refined when new evidence becomes available. The architecture is presented as a route to **model-based recursive self-improvement (RSI)**. The recursion is external to the LLM weights: the current model predicts and guides interaction; interaction produces evidence; evidence causes model revision; and the revised model guides subsequent exploration and planning. The LLM itself remains fixed. A further motivation is the problem of self-confirming models. A model may avoid states that would falsify its assumptions. If the set of histories reached under a model’s own policy contains no prediction-error states, the model can remain unfalsified despite being incorrect elsewhere. Memento 3 addresses this problem partly through exploratory actions and, in its population extension, through parallel alternative world models that generate different behaviors. ## 2. Formal environment and learning state The environment is modeled as a deterministic, goal-directed partially observable Markov decision process: $$ M=\langle S,A,O,T,\rho,G\rangle, $$ where $S$ is the underlying state space, $A$ is the action space, $O$ is the observation space, $T$ is the unknown transition function, $\rho$ maps hidden states to observations, and $G$ is the unknown set of goal states. At time $t$, $$ s_{t+1}=T(s_t,a_t), \qquad o_t=\rho(s_t). $$ The agent receives observations, action-interface information, and completion feedback, but not the source code, true transition function, hidden state, or natural-language task description. Its interaction history is $$ H_t=(o_0,a_0,o_1,\ldots,a_{t-1},o_t), $$ with completion feedback retained in the record. The paper also gives a Bayesian reference formulation over a discrete family of candidate models $m=(T_m,\rho_m,G_m)$. The ideal joint belief is $$ b_t(s,m)=\Pr(s_t=s,m^\star=m\mid H_t), $$ where $m^\star$ is the unknown true model. Memento 3 does not explicitly maintain this full posterior. Instead, it approximates the remaining hypothesis space using one or more LLM-generated and verified rulebook–program pairs. Let $\Psi$ denote the space of finite natural-language rulebooks and $\Theta$ the space of executable world-model programs. The persistent hypothesis is $$ h_t=(\psi_t,\theta_t), $$ and the complete external learning state is $$ Z_t=(H_t,\psi_t,\theta_t). $$ The three components have distinct roles: - $H_t$ records empirical evidence. - $\psi_t$ expresses generalized semantic hypotheses. - $\theta_t$ operationalizes those hypotheses for simulation and planning. For a rulebook $\psi$, $[\psi]\subseteq\Theta$ denotes the set of programs that faithfully implement its stated hypotheses. This permits a distinction between semantic changes and implementation repairs. A rule-level change modifies the environmental hypothesis itself; an implementation repair changes code while leaving the rulebook intact. The LLM parameters remain constant: $$ \phi_{t+1}=\phi_t=\phi. $$ The mutable learning state is therefore external to the foundation model. ## 3. Rulebooks and executable world models An executable world model $\theta$ defines an internal state space, an initializer, a transition predictor, an observation renderer, and a predicted goal set: $$ \iota_\theta:O\to\widehat S_\theta, $$ $$ f_\theta:\widehat S_\theta\times A\to\widehat S_\theta, $$ $$ \rho_\theta:\widehat S_\theta\to O, $$ $$ \widehat G_\theta\subseteq\widehat S_\theta. $$ Replay begins with $$ \widehat s_0^\theta=\iota_\theta(o_0) $$ and proceeds according to $$ \widehat s_{i+1}^\theta=f_\theta(\widehat s_i^\theta,a_i). $$ The predicted next observation is $$ \widehat o_\theta(H_i,a_i) = \rho_\theta\!\left(f_\theta(\widehat s_\theta(H_i),a_i)\right). $$ The internal state reconstruction is essential because the latest observation may not expose hidden variables such as inventory, velocity, switches, or opponent phase. The model must therefore maintain a latent operational state reconstructed from the entire interaction history. A rulebook is a semantic representation rather than a mere event log. It can state rules about object interactions, glyph orientation, cursor behavior, transformations, goal conditions, and dynamics such as Pong ball–paddle interactions. It may also retain competing hypotheses or explicitly leave uncertain mechanics unresolved. Memento 3 uses a rulebook–code pair rather than code alone because executable code can reproduce observed transitions while implementing an unintended rule. A program might contain special cases that fit the recorded data but fail to express the intended generalization. Rulebook–code faithfulness checking is intended to prevent this form of opaque overfitting. The paper gives a minimum-description-length interpretation of model selection. Among verified rulebook–program pairs, the LLM’s simplification preference is treated as an approximation to selecting a pair minimizing the description lengths of the rulebook and its implementation: $$ (\psi_t,\theta_t) \approx \arg\min_{(\psi,\theta)\in\mathcal V_t} \left[ \mathcal L(\psi)+\mathcal L(\theta\mid\psi) \right]. $$ This is an interpretive formulation rather than an explicitly maintained Bayesian optimization procedure. Replay removes hypotheses that contradict observations, while simplicity provides a practical bias among hypotheses that remain consistent. ## 4. Observation, reflection, revision, and verification Memento 3 operates through a continual loop executed after real environmental interactions. ### Observation and prediction error The agent selects an action, executes it, receives the next observation and completion feedback, and extends the history: $$ H_{t+1}=H_t\mathbin{\Vert}(a_t,o_{t+1}). $$ The current executable predicts the result of the action. The observed transition is compared with the prediction. A discrepancy may indicate: - an incorrect environmental rule; - an incorrect goal or termination hypothesis; - a missing hidden state variable; - a code defect; - an overly narrow generalization; - an underspecified mechanic that the implementation prematurely fixed. In ARC-AGI-3, discrepancy detection is cell-exact: predicted rendered frames must match all $64\times64$ grid cells of the observed frame. Execution stops at the first mismatch so that the counterexample can be incorporated before further interaction. ### Reflective diagnosis The fixed LLM chooses among three diagnoses: $$ \mathcal D= \{\mathsf{retain},\mathsf{revise\_rulebook},\mathsf{repair\_code}\}. $$ The diagnosis is sampled or selected according to the LLM’s reflection process: $$ d_{t+1}\sim \pi_\phi(\cdot\mid H_{t+1},\psi_t,\theta_t). $$ The alternatives have distinct meanings: - **Retain**: the existing rulebook–code pair remains plausible. - **Revise rulebook**: the semantic environmental hypothesis is wrong or incomplete. - **Repair code**: the rulebook is acceptable, but the implementation is not faithful. If revision is selected, the rulebook is updated using the accumulated history. Revision may retract an earlier rule, refine an overly broad rule, introduce a hidden variable, resolve competing hypotheses, or record an unresolved alternative. ### Compilation and code repair A rulebook-guided compiler produces an executable candidate: $$ \widetilde\theta_{t+1} \sim C_\phi(\cdot\mid \theta_t,\widetilde\psi_{t+1},H_{t+1}). $$ If the diagnosis is `retain`, the existing executable is retained. Otherwise, the LLM generates new code or repairs the existing implementation. The candidate must pass two independent checks. First, the LLM assesses semantic faithfulness: $$ \widetilde\theta_{t+1}\in[\widetilde\psi_{t+1}]. $$ Second, deterministic replay verifies every recorded transition. For a candidate $\theta$, $$ \mathrm{verify}(\theta;H_t) = \prod_{i