Retrodiction of Forgetting: Inferring Memory Decay

• Retrodiction of Forgetting is defined as the retrospective inference of latent memory loss from observable behavior and predictive distributions.
• It employs formal models based on temporal discounting, decay laws, and interference to estimate past knowledge states from present outputs.
• This approach spans educational models, continual learning, and LLM inference to diagnose and reconstruct forgetting trajectories.

Retrodiction of forgetting is the retrospective inference of latent memory loss, temporal discounting, or obsolete knowledge states from present behavior, predictive distributions, or stored state variables. In current research, the term is often implicit rather than explicit: some work infers a model’s overall forgetting tendency from present outputs, some estimates current recall probability from response histories, some diagnoses loss of predictive information from changes in induced future distributions, and some inverts controlled forgetting trajectories in weight space to recover a more knowledgeable precursor state (Tran et al., 28 Dec 2025, Mooney et al., 2018, Sanati et al., 6 Nov 2025, Jang et al., 7 Aug 2025). A useful distinction is between “state retrodiction” (Editor’s term), which infers latent memory strength or accessibility, and “content retrodiction” (Editor’s term), which infers which items, facts, or parameters were attenuated, suppressed, or lost.

1. Conceptual scope and historical framing

Retrodiction of forgetting arises wherever forgetting is treated not only as a forward decay process but also as an inferable hidden cause of present behavior. In educational modeling, the target is often a learner’s current recall probability “for any KC at any point in time,” inferred from correctness, timing, spacing, direction, and question format (Mooney et al., 2018). In LLM inference, the target may be a latent discount factor or an item-level retention profile that explains why current outputs favor recent evidence over older conflicting facts (Tran et al., 28 Dec 2025). In continual learning, the target may be a change in predictive competence, a shift in internal guidance, or an obsolete but still internally stored fact (Sanati et al., 6 Nov 2025, Sun et al., 2024, Elbadry et al., 9 May 2026).

Two broad traditions structure the topic. One treats forgetting as decay-like, often using exponential or power-law forms. Another treats forgetting as interference-like, where later items, tasks, or competitors block or overwrite retrieval. The literature explicitly contains both views: one line models recall with adaptive power-law forgetting curves and recurrent updates (Mooney et al., 2018), whereas another explains power-law-like forgetting by retroactive interference among later memories rather than passive decay (Georgiou et al., 2019). More recent geometric accounts intensify the interference view by arguing that time alone produces little forgetting unless competing memories exist in a crowded representational space (Barman et al., 27 Mar 2026).

A recurring theme is that retrodiction becomes meaningful only when forgetting is represented by explicit variables, equations, or trajectories. This is why recent work on agent memory, continual learning, and weight-space inversion is especially relevant: these systems expose state variables, decay laws, or transition maps that make retrospective explanation possible (Wei et al., 26 Jan 2026, Jang et al., 7 Aug 2025).

2. Formal models for inferring forgotten state

A prominent formalization treats inference itself as a discounted memory process. In one LLM account, ideal Bayesian accumulation is replaced by temporal discounting: $$p_t(\theta \mid \mathcal{D}_{1:t}) \propto p(D_t \mid \theta)\,[p_{t-1}(\theta \mid \mathcal{D}_{1:t-1})]^\gamma,$$ where $\gamma \in (0,1]$ is the temporal discount factor. Forgetting is then retrodicted by fitting an optimal $\gamma^*$ that minimizes average predictive KL divergence: $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$ The same framework gives an item-level retention law

$$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$

or, in unnormalized form, $R(\Delta t)=e^{-\lambda \Delta t}$. Under this model, present outputs support backward inference about which earlier observations had low effective weight (Tran et al., 28 Dec 2025).

In educational memory modeling, retrodiction appears as estimation of latent recall from practice history. The Recurrent Power Law model defines

$$s' = e^{-s}, \qquad \tau' = e^{-\tau}, \qquad p_{cr} = (1 + s' r)^{-\tau'},$$

with $r$ the retention interval. Successful and unsuccessful recalls then update the latent forgetting-curve parameters $s$ and $\tau$, so the model maintains a history-dependent estimate of the learner’s current forgetting state rather than a static score (Mooney et al., 2018).

A more general learning-theoretic formalism defines forgetting as lack of self-consistency in predictive distributions over futures. Its operational quantity is

$\gamma \in (0,1]$0

where forgetting is diagnosed when self-generated, expected updates alter the learner’s own induced future distribution. This does not reconstruct forgotten content directly, but it provides a principled retrospective diagnostic of information loss over an interval (Sanati et al., 6 Nov 2025).

Agent-memory architectures make the hidden variables even more explicit. FadeMem represents each memory as

$\gamma \in (0,1]$1

with strength evolving under adaptive decay: $\gamma \in (0,1]$2 Because $\gamma \in (0,1]$3 depends on semantic relevance, access frequency, and recency, later weakening can be retrodicted as low importance, insufficient reinforcement, contradiction suppression, redundancy penalty, fusion, or pruning, rather than as undifferentiated loss (Wei et al., 26 Jan 2026).

3. Empirical settings, observables, and retrodictive signals

The methodology of retrodicting forgetting depends strongly on domain and observable. In LLM inference work, the proposed benchmark family includes Temporal Recall, Concept Drift, and Associative Memory. The intended measurements are predictive KL divergence, task-specific accuracy under distribution shifts, and curve-fitting error against established human forgetting laws. This setup is explicitly designed to support fitting decay profiles and estimating whether adaptation failures reflect stale beliefs, excessive recency, or wrong inference schemes (Tran et al., 28 Dec 2025).

In multi-task reinforcement learning, forgetting is observed through evaluation reward and solution rate after task switches. The experiments use MiniGrid environments with a PPO agent, and one protocol trains on SimpleCrossing until the agent solves it in evaluation $\gamma \in (0,1]$4 of the time, then switches to an Empty grid until the agent only achieves a solution rate of $\gamma \in (0,1]$5 or less. The resulting traces show decreasing forgetting in six seeds and periodic forgetting in four seeds, and the paper’s main retrodictive lesson is that present performance is ambiguous unless one conditions on which task intervened, how often a task had been repeated, and whether cross-task effects are asymmetric (Speckmann et al., 3 Mar 2025).

In attentive knowledge tracing, forgetting is operationalized less as a latent memory state than as recency-weighted attenuation of old interactions. FoLiBi injects a head-specific linear bias into attention,

$\gamma \in (0,1]$6

so that reduced present influence of a past interaction can be attributed more cleanly to sequence lag rather than to entangled content similarity. This is a weaker form of retrodiction, but it supports retrospective explanation of why old evidence counted less in current predictions (Im et al., 2023).

In continual learning for LLMs, optimizer motion can itself be used as a proxy signal. FOREVER defines model-centric time by

$\gamma \in (0,1]$7

and recent instability by

$\gamma \in (0,1]$8

These quantities are not direct forgetting measures, but they are loggable latent clocks and hazard proxies from which forgetting pressure can be retrospectively reconstructed more plausibly than from raw step count (Feng et al., 7 Jan 2026).

Weight-space work pushes retrodiction further by treating forgetting itself as the signal to invert. KNOW prediction defines

$\gamma \in (0,1]$9

and trains KNOWN with the residual loss

$\gamma^*$0

Here the goal is not merely to diagnose forgetting, but to infer a synthetic, less-forgotten, and even counterfactually more knowledgeable weight state from a controlled forgetting trajectory (Jang et al., 7 Aug 2025).

4. Distinguishing forgetting from neighboring phenomena

A major development in the literature is the claim that not all apparent forgetting is actual loss of stored capability. The most explicit example is pseudo-forgetting, where performance on old tasks drops under the original instruction but can be restored through partial external correct rationale or Task-Agnostic Prefix Prompting. The paper defines

$\gamma^*$1

and argues that larger values indicate that the prompt has become worse at guiding the model toward the appropriate rationale. In this view, low present accuracy may retrodict a failure of activation or rationale guidance rather than destruction of the underlying task competence (Sun et al., 2024).

A related but distinct phenomenon is temporal knowledge drift. Here the issue is not sequential fine-tuning but the fact that the world changed after training. The paper argues that drift is encoded as a residual-stream direction geometrically orthogonal to both correctness and uncertainty, and reports a linear drift probe with AUROC $\gamma^*$2–$\gamma^*$3, while methods based on token entropy, semantic entropy, CCS, and SAPLMA remain near chance at $\gamma^*$4–$\gamma^*$5. Five tests support the orthogonality claim, including weight cosines $\gamma^*$6, score correlations $\gamma^*$7, and bidirectional null-space projection $\gamma^*$8. This makes it possible to retrodict whether a wrong answer reflects stale but once-valid internal knowledge rather than generic confabulation (Elbadry et al., 9 May 2026).

Interference-based theories sharpen the distinction further by arguing that forgetting need not be time-driven at all. The retroactive interference model represents memories by multi-dimensional valences and derives

$\gamma^*$9

with asymptotic form

$$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$0

In this account, older surviving memories become more stable because they have survived repeated competition, so retrodiction should target interference structure and effective dimensionality rather than a universal decay constant (Georgiou et al., 2019).

Geometric memory models make the same point in a different language. One paper reports that with the same temporal decay function but no competitors, the fitted forgetting exponent is $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$1, whereas with $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$2 distractors it rises to $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$3, close to the human benchmark $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$4. The same work reports effective dimensionalities of $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$5, $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$6, and $$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$7 for MiniLM, BGE-base, and BGE-large respectively, suggesting that forgetting risk is governed by local competition in low-effective-dimensional semantic space rather than by elapsed time alone (Barman et al., 27 Mar 2026).

5. Identifiability, ambiguity, and methodological limits

Retrodiction of forgetting is often underdetermined. The clearest practical example is repeated-learning convolution modeling, where later retention is generated by

$$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$8

or, for repeated impulses,

$$\gamma^* = \arg\min_{\gamma \in (0,1]} \frac{1}{T}\sum_{t=1}^T \mathrm{KL}\!\left(\hat{p}_{\text{LLM},t}\,\|\, p_{\text{Bayes},t}(\cdot;\gamma)\right).$$9

The paper explicitly notes that different kernel choices can fit the same sparse test points while implying different full curves, and that denser observations are needed to distinguish trajectories. This implies that present recall may be consistent with multiple histories of repetition, decay, and input duration (Xie et al., 2019).

Even when the forward process is analytically tractable, inverse identification may remain partial. In the retroactive interference note, the exact retention curve

$$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$0

and the exact joint retention probability

$$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$1

show that later retention depends on latent dimensionality, spacing, and order geometry, but the paper is explicit that later retention patterns generally cannot recover a unique past memory state or exact presentation history (Katkov, 2023).

The empirical basis of some retrodictive proposals is also uneven. One LLM paper specifies discounted-Bayes fitting, temporal-recall benchmarks, concept-drift probes, and Probabilistic Memory Prompting, but its empirical section does not actually report the promised forgetting-benchmark results. As stated in the source description, there are no actual estimated $$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$2 values, no fitted $$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$3 values, no explicit temporal degradation curves, and no direct human-vs-LLM forgetting-curve plots. This limits how strongly its formal retrodictive machinery can be validated (Tran et al., 28 Dec 2025).

Continual-learning theory adds another caution: replay metadata are not sufficient. In over-parameterized continual linear regression, forgetting on past tasks is

$$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$4

and the paper proves that sample replay can increase forgetting. In the worst case, forgetting without replay is $$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$5, but with replay it can become $$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$6; in the average-case setting, even random replay can increase forgetting in expectation. The retrodictive implication is that one must know not just replay budget, but which samples were replayed and how replay altered the effective projection geometry (Mahdaviyeh et al., 4 Jun 2025).

A still broader limit appears in logical forgetting. In propositional settings, forgetting removes variables while preserving consequences over the retained alphabet, yet the resulting representation can become larger rather than smaller. The paper proves that deciding whether forgetting from a Horn formula can be expressed within a given size bound is $$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$7-hard and in $$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$8, while the unrestricted propositional version is $$w_i = \frac{\exp(-\lambda (t-i))}{\sum_{j=1}^t \exp(-\lambda (t-j))},$$9-hard and in $R(\Delta t)=e^{-\lambda \Delta t}$0. This suggests that post-forgetting representations may carry “ghosts” of lost structure, but not in a uniquely invertible way (Liberatore, 2020).

6. Significance and emerging directions

A consistent outcome across recent work is that forgetting is increasingly treated as a structured, and sometimes adaptive, process rather than a single failure mode. One LLM account explicitly reframes forgetting as a feature rather than a bug, arguing that graded discounting can balance stability against plasticity in changing environments (Tran et al., 28 Dec 2025). Agent-memory work makes the same point computationally: important memories decay more slowly, retrieval strengthens memory, contradictory memories are suppressed, and related traces can be fused into stronger abstractions while irrelevant details fade (Wei et al., 26 Jan 2026).

This suggests a broader program for retrodiction. Instead of asking only whether a model is wrong now, one asks whether present behavior is best explained by temporal discounting, interference, activation failure, structured obsolescence, or representational compression. The most promising approaches therefore combine explicit latent variables, causal logging, and domain-specific observables: predictive-distribution comparisons in general learners, stateful memory variables in agents, drift probes in residual space, and controlled forgetting trajectories in parameter space (Sanati et al., 6 Nov 2025, Elbadry et al., 9 May 2026, Jang et al., 7 Aug 2025).

A plausible implication is that future retrodictive systems will need provenance-like memory traces rather than final outputs alone. Several papers already point in that direction by emphasizing access timestamps, layer assignment history, fusion events, contradiction categories, or optimizer-path length as explanatory variables (Wei et al., 26 Jan 2026, Feng et al., 7 Jan 2026). Where those logs are absent, retrodiction remains possible only in a model-based and often non-identifiable sense.

In that sense, retrodiction of forgetting is less a single technique than a convergence point between memory modeling, continual learning, educational tracing, agent architectures, and representational analysis. Its central question is stable across domains: given what is observable now, what can be inferred about how memory weakened, which information remained latent, which information was overwritten or compressed, and whether present error reflects forgetting at all?