Memini: Continual Knowledge Updating in LLMs

• Memini is a continual learning architecture that shifts adaptation from static LLM parameters to a dynamic external memory layer.
• It employs a two-variable, Benna–Fusi inspired dynamical system on graph edges, enabling immediate updates, consolidation, and selective forgetting.
• The design supports online, training-free operation where recent associations are quickly accessible and unsupported ones naturally decay.

Memini is a proposal for continual knowledge updating in LLM systems that reconceptualizes external memory as a self-organizing learning substrate rather than a passive datastore (Pattichis et al., 6 May 2026). In this formulation, an LLM’s parametric knowledge remains fixed after training, while continual adaptation is shifted into a persistent external memory organized as a directed associative graph whose edge states evolve autonomously through coupled fast and slow dynamics inspired by the Benna–Fusi model of synaptic consolidation. The central claim is that new associations should become immediately usable, repeated evidence should gradually consolidate, and unsupported associations should fade, all through the same local dynamical rule rather than through explicit archival, pruning, or controller-driven memory management (Pattichis et al., 6 May 2026).

1. Conceptual framing and problem setting

Memini is motivated by a standard deployment mismatch: LLMs are trained once, then deployed into environments in which facts appear, disappear, and change salience over time (Pattichis et al., 6 May 2026). Directly updating model weights is described as expensive, risky, or inaccessible for downstream users of API-based models, and it reintroduces the stability–plasticity problem: rapid adaptation can overwrite older knowledge, while slow adaptation misses new information (Pattichis et al., 6 May 2026). External-memory systems avoid modifying the backbone model, but Memini argues that most such systems remain static at a deeper level because their organization is typically governed by explicit rules or an external controller rather than by autonomous internal learning dynamics (Pattichis et al., 6 May 2026).

Against that background, Memini proposes that external memory should follow a biological-memory-style principle. The paper characterizes biological memory as using coupled multi-timescale dynamics such that new associations are immediately usable, repeated associations are strengthened, and unsupported associations fade (Pattichis et al., 6 May 2026). Memini translates that principle into an LLM-facing memory layer: the LLM reads and writes text, but the memory performs the learning (Pattichis et al., 6 May 2026).

This places Memini within continual learning, but not in the usual sense of continual parameter adaptation. Learning is relocated from the parametric backbone to the non-parametric memory layer. A plausible implication is that Memini is especially targeted at frozen-model or API-only regimes, because the architecture is explicitly described as online, local, training-free, with no gradient-based optimization and no retraining of the LLM (Pattichis et al., 6 May 2026).

2. Associative graph representation

Memini represents knowledge as a directed graph

$G = (V, E),$

where nodes are entities or concepts extracted from an incoming document stream and directed edges represent associations between them (Pattichis et al., 6 May 2026). The graph is associative rather than symbolic in the knowledge-graph sense: edges are unlabeled, so meaning is carried by the pattern, direction, and strength of connectivity rather than by explicit typed relations (Pattichis et al., 6 May 2026). Direction is semantically consequential, because $A \to B$ and $B \to A$ are distinct associations.

Each directed edge $(A,B)$ stores two internal variables:

• $w_{\text{fast}}$, the accessible retrieval-facing component
• $w_{\text{slow}}$, a hidden consolidation component

The minimal memory unit is therefore

$(A, B, w_{\text{fast}}, w_{\text{slow}}).$

The full memory state at time $t$ is given as the evolving graph plus the two state variables on each edge:

$$G(t) = (V(t), E(t)), \qquad \{(w_{\text{fast}}(e,t),\, w_{\text{slow}}(e,t))\}_{e \in E(t)}.$$

(Pattichis et al., 6 May 2026)

This representation is neither a typed knowledge graph nor a conventional key-value store. Compared with standard RAG or vector-store retrieval, Memini does not merely index passages for similarity search; it maintains an evolving association structure whose future retrieval trajectories depend on prior evidence (Pattichis et al., 6 May 2026). Compared with key-value memory, it privileges distributed associative traversal over exact slot lookup. Compared with knowledge-graph memory, it uses graph structure without typed symbolic relations, and the paper explicitly argues that prior graph memories often leave graph weights static even when traversal is sophisticated (Pattichis et al., 6 May 2026).

When a new document arrives, the system extracts entities and their co-occurrences. A co-occurrence between $A$ and $A \to B$0 updates the corresponding directed edge online; nonexistent edges are effectively initialized at zero and then updated (Pattichis et al., 6 May 2026). Retrieval reads from this graph state, but does not itself alter it. The paper emphasizes that memory evolves from the document stream rather than from retrieval activity (Pattichis et al., 6 May 2026).

3. Multi-timescale edge dynamics

The technical core of Memini is a two-timescale Benna–Fusi-style dynamical system attached to every directed edge. Using the simplest chain model with $A \to B$1 variables per edge, the state of $A \to B$2 evolves as

$A \to B$3

$A \to B$4

where $A \to B$5 and $A \to B$6 are decay constants with $A \to B$7, $A \to B$8 is the coupling strength, and $A \to B$9 is the external co-occurrence input (Pattichis et al., 6 May 2026).

The input is defined by co-occurrence:

$B \to A$0

and

$B \to A$1

Thus $B \to A$2 is driven by direct input, its own decay, and coupling toward $B \to A$3, whereas $B \to A$4 receives no direct input and changes only via slow decay and coupling from $B \to A$5 (Pattichis et al., 6 May 2026). This asymmetry is central. A single mention can create a sharp transient rise in $B \to A$6, but only repeated evidence keeps the fast variable elevated long enough to pull up $B \to A$7. Once $B \to A$8 is large, it feeds back to sustain $B \to A$9 between exposures (Pattichis et al., 6 May 2026).

The paper identifies four behaviors emerging from this mechanism and treats them as the main justification for using two coupled variables rather than a single decaying weight.

First, Memini yields immediate episodic sensitivity: after a single co-occurrence, $(A,B)$0 rises rapidly, making the association retrievable almost at once (Pattichis et al., 6 May 2026).

Second, it yields gradual consolidation: repeated evidence accumulates in the slow variable through the coupling term $(A,B)$1, and the resulting $(A,B)$2 then sustains future accessibility through $(A,B)$3 (Pattichis et al., 6 May 2026).

Third, it yields selective forgetting: associations with sparse or old support decay toward zero, while reinforced ones persist. The paper explicitly states that weakening is not caused by explicit negative evidence or “anti-updates.” There is no special update when $(A,B)$4 appears without $(A,B)$5; forgetting emerges from the decay terms

$(A,B)$6

(Pattichis et al., 6 May 2026)

Fourth, it supports adaptation to change over time: because edge states are history-dependent, older unsupported associations stop dominating, while newer recurring associations become stronger (Pattichis et al., 6 May 2026). The paper notes that the resulting memory is trajectory-dependent, so the same facts arriving in different temporal orders can produce different associative organization.

A notable design choice is that retrieval reads only $(A,B)$7, not $(A,B)$8 (Pattichis et al., 6 May 2026). The stated reason is to preserve a recency signal. A recently mentioned association and an older consolidated association may have the same current $(A,B)$9, but the edge with higher $w_{\text{fast}}$0 will remain retrievable longer because the hidden slow variable continues to sustain the fast one (Pattichis et al., 6 May 2026). This allows episodic and consolidated memory to be differentiated dynamically rather than by introducing separate memory stores.

For practical simulation in the appendix, the authors use forward Euler updates with $w_{\text{fast}}$1 per document, clamp both variables to be non-negative, and illustrate the dynamics with

$w_{\text{fast}}$2

(Pattichis et al., 6 May 2026)

4. Retrieval and online operation

Memini retrieves context by spreading activation over the current graph state. Query entities are mapped to seed nodes initialized as

$w_{\text{fast}}$3

Activation then propagates for a fixed number of steps according to

$w_{\text{fast}}$4

where $w_{\text{fast}}$5 is per-iteration retention decay, $w_{\text{fast}}$6 is a global spreading factor, $w_{\text{fast}}$7 is the dynamic edge weight, and $w_{\text{fast}}$8 imposes a fan-effect penalty on high-degree hubs (Pattichis et al., 6 May 2026). After $w_{\text{fast}}$9 propagation steps, nodes are ranked by final activation and the top-$w_{\text{slow}}$0 associated passages are returned to the LLM as context (Pattichis et al., 6 May 2026).

The paper notes that this resembles graph-based retrieval methods such as spreading-activation RAG, but emphasizes a crucial distinction: Memini’s edge weights are persistent and experience-shaped (Pattichis et al., 6 May 2026). In other graph retrieval systems, weights are often fixed at indexing time or adjusted only within a query. In Memini, the graph has already been reorganized by prior evidence, so the same query posed at different times can traverse different paths and surface different passages (Pattichis et al., 6 May 2026).

Operationally, the update loop is local and online. When a new document arrives, the system:

1. identifies entities or concepts through the LLM or another extraction component;
2. forms candidate directed associations from co-occurring entity pairs;
3. applies the local dynamical update with $w_{\text{slow}}$1 to each observed edge;
4. allows all edges to continue evolving through decay and coupling even when $w_{\text{slow}}$2.

After a single observation, the usual outcome is a transient rise in $w_{\text{slow}}$3 with little change in $w_{\text{slow}}$4; after repeated observations, $w_{\text{slow}}$5 accumulates and the edge becomes durable (Pattichis et al., 6 May 2026). If an old fact is no longer supported, both variables decay. If a conflicting association begins receiving more evidence, its edge strengthens while the older one fades, allowing the graph to shift toward current information without explicit conflict-resolution logic (Pattichis et al., 6 May 2026).

The paper does not provide a formal complexity analysis. It states only that the natural cost is proportional to the number of observed co-occurrence updates plus graph-based retrieval propagation, that edge storage is constant-size per association with two scalars per directed edge plus passage links, and that memory growth depends on how many entities and co-occurrences are admitted into the graph (Pattichis et al., 6 May 2026). It discusses decay-driven disappearance conceptually, but does not specify a full pruning policy beyond the fact that unsupported weights move toward zero (Pattichis et al., 6 May 2026).

5. Empirical evidence and ablations

The empirical section is explicitly limited in scope. Memini is presented as a design paper with an initial illustrative experiment, not as a full benchmark study; there are no standard QA benchmarks, no published-system comparisons, and no retrieval metrics yet (Pattichis et al., 6 May 2026). The appendix instead provides a sanity check on whether the intended dynamics emerge on a real temporal document stream.

The experimental corpus consists of 13 historically versioned Wikipedia articles about COVID-19, spanning four phases: origin, containment, vaccines, and variants/endemic transition (Pattichis et al., 6 May 2026). Versioning is used so that each article is fetched as it existed at the relevant time, thereby avoiding leakage from later edits (Pattichis et al., 6 May 2026). The experiment tracks 20 entities across these phases, detects sentence-level co-occurrences, and generates 124 events over 68 unique entity pairs (Pattichis et al., 6 May 2026).

The main qualitative findings are narrowly targeted but consistent with the design claims: single mentions create transient fast traces; repeated clustered mentions create consolidation; recent repeated associations remain stronger than equally frequent older ones; and the slow variable is necessary to retain repeated-but-no-longer-mentioned associations (Pattichis et al., 6 May 2026).

The critical ablation compares Memini with two baselines. The first is a single-timescale ablation

$w_{\text{slow}}$6

with $w_{\text{slow}}$7 matched to Memini’s effective early decay when $w_{\text{slow}}$8:

$w_{\text{slow}}$9

The second is a uniform-retention baseline that counts events and never decays (Pattichis et al., 6 May 2026).

Pairs are grouped by frequency and recency. The strongest result is reported for repeated, no longer mentioned pairs, for which Memini retains substantially more weight than the matched single-timescale ablation, isolating the contribution of $(A, B, w_{\text{fast}}, w_{\text{slow}}).$0 (Pattichis et al., 6 May 2026). The reported mean final weights are:

• Repeated, no longer mentioned: Memini $(A, B, w_{\text{fast}}, w_{\text{slow}}).$1, single-timescale $(A, B, w_{\text{fast}}, w_{\text{slow}}).$2, uniform $(A, B, w_{\text{fast}}, w_{\text{slow}}).$3
• Few mentions, no longer mentioned: Memini $(A, B, w_{\text{fast}}, w_{\text{slow}}).$4, single-timescale $(A, B, w_{\text{fast}}, w_{\text{slow}}).$5, uniform $(A, B, w_{\text{fast}}, w_{\text{slow}}).$6
• Repeated, recently mentioned: Memini $(A, B, w_{\text{fast}}, w_{\text{slow}}).$7, single-timescale $(A, B, w_{\text{fast}}, w_{\text{slow}}).$8, uniform $(A, B, w_{\text{fast}}, w_{\text{slow}}).$9
• Few mentions, recently mentioned: Memini $t$0, single-timescale $t$1, uniform $t$2

The paper interprets this pattern as showing that Memini preserves old associations when they had repeated support, allows weak old associations to fade, and behaves similarly to simpler methods when consolidation has not had time to matter (Pattichis et al., 6 May 2026). The authors also emphasize the limitations: the corpus is tiny, the groups are small, and the current empirical support concerns the plausibility and necessity of the dynamics rather than demonstrated downstream superiority on end tasks (Pattichis et al., 6 May 2026).

6. Relation to adjacent memory architectures, misconceptions, and limitations

Memini is explicitly distinguished from several existing memory paradigms. Relative to standard RAG or vector stores, it is not merely an external retrieval layer but an evolving associative substrate whose organization changes through evidence accumulation and decay (Pattichis et al., 6 May 2026). Relative to continual learning in model weights, it shifts adaptation into a non-parametric memory layer, importing the Benna–Fusi consolidation principle into external-memory architecture rather than parameter dynamics (Pattichis et al., 6 May 2026).

Within the broader memory-systems literature, Memini also sits alongside but is not reducible to other recent proposals. Mnemosyne is likewise a graph-structured, human-inspired long-term memory architecture, but it is designed for edge-based dialogue systems and relies on memory commitment, redundancy filtering, probabilistic recall, temporal decay, and a fixed-length core summary rather than on Memini’s two-variable per-edge Benna–Fusi dynamics (Jonelagadda et al., 7 Oct 2025). MeMo is a different line of work again: an explicit associative language-model architecture that directly memorizes token sequences in layered correlation-matrix memories and supports subtractive forgetting, rather than an external graph memory for continual knowledge updating (Zanzotto et al., 18 Feb 2025). MemAudit addresses yet another problem: post-hoc auditing of poisoned persistent agent memory through counterfactual causal attribution and structural anomaly detection, rather than autonomous consolidation and forgetting in the memory layer itself (Tan et al., 22 May 2026).

A common source of confusion is terminological. The paper “Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks” studies extractive memorization in neural machine translation and does not mention “Memini”; it concerns memorization detection and mitigation in constrained sequence generation, not continual external-memory updating for LLM systems (Raunak et al., 2022).

The limitations of Memini are substantial and explicitly acknowledged. It remains a conceptual architecture plus a small illustrative validation; there is no evidence yet that it improves end-task QA, retrieval robustness, or large-scale temporal knowledge benchmarks (Pattichis et al., 6 May 2026). It depends on entity extraction and co-occurrence detection, so errors in those stages directly affect memory formation (Pattichis et al., 6 May 2026). Because edges are unlabeled, distinct relation types may be blurred unless graph structure disambiguates them (Pattichis et al., 6 May 2026). The paper does not specify a detailed pruning algorithm, and several practical questions—large-scale retrieval behavior, graph growth control, and formal complexity characterization—are left open (Pattichis et al., 6 May 2026).

Taken together, these features define Memini’s significance. It treats external memory not as a static repository to be searched, but as a local dynamical system that reorganizes itself through use and disuse (Pattichis et al., 6 May 2026). This suggests a particular research program for continual knowledge updating: the central problem is not only how an LLM retrieves from memory, but how memory itself should learn.