---
title: Designed Forgetting in Computational Systems
url: https://www.emergentmind.com/topics/designed-forgetting
type: topic
---

# Designed Forgetting in Computational Systems

Designed forgetting is the intentional incorporation of forgetting into computational systems as a controlled capability rather than an accident of optimization or storage failure. In recent work, the term spans several levels of abstraction: a sociotechnical umbrella over erasure, unlearning, and exclusion in data systems; selective attenuation or deletion in neural parameters and external memories; context-conditioned suppression of outputs at inference time; and architectural designs in which forgetting is supported directly by the model or memory substrate [2602.21180, 2405.20620, 2603.15033, 2410.00382]. Across these strands, the shared objective is to remove, bound, or de-emphasize information in ways that preserve utility, adaptability, privacy, or safety while avoiding uncontrolled catastrophic forgetting.

## 1. Definitions, scope, and conceptual boundaries

Designed forgetting is distinguished from both unintentional forgetting and catastrophic forgetting. The survey literature defines unintentional forgetting as incidental information loss, and catastrophic forgetting as the abrupt loss of previously learned tasks during sequential training, whereas designed forgetting is an intentional model component or procedure that removes or down-weights specific information by design and is intended to improve generalisation, adaptability, or privacy [2405.20620]. In data-systems terms, Makovska et al. further argue that forgetting cannot be reduced to a single operation and should instead be treated as a sociotechnical practice differentiated by **agency**, **temporality**, **reversibility**, and **scale** [2602.21180].

Within that taxonomy, **erasure** is the operational removal or disabling of access to data artifacts such as rows, files, indexes, caches, or logs; **unlearning** is a technical intervention designed to remove or bound a data point’s influence on model parameters or downstream outputs; and **exclusion** is upstream non-collection or omission, described as a form of forgetting by design [2602.21180]. This distinction is foundational because record deletion does not by itself alter learned weights, feature transformations, or predictive behavior.

Several subfields instantiate the same general idea with domain-specific emphases. In generative language models, graceful forgetting aims to enhance learning plasticity by selectively discarding irrelevant pre-trained knowledge that would otherwise induce negative transfer [2505.19715]. In multimodal unlearning, “benign memory forgetting” denotes targeted erasure of private knowledge while preserving general visual understanding and unrelated memory [2511.20196]. In LLM agents, designed forgetting is defined as intentional, policy-driven removal or de-emphasis of stored memories according to value, recency, safety, and usage feedback [2604.20300]. In memory-augmented vision models, “unlearning by design” means that forgetting is an intrinsic architectural operation rather than a post-hoc patch [2603.15033].

A recurring implication is that designed forgetting is not a single algorithmic family. It is a design objective that can be realized through decay, deletion, masking, bottlenecking, retrieval gating, adversarial balancing, or governance procedures, depending on whether the system is a database, a classifier, a generative model, a robot, or an agent memory.

## 2. Mechanistic taxonomy

The contemporary literature organizes designed forgetting into several recurring mechanisms. The survey literature groups forgetting methods into **regularization-based**, **data-driven (unlearning)**, and **architectural** families, while agent-memory work refines this into passive decay-based, active deletion-based, safety-triggered, and adaptive reinforcement-based forgetting [2405.20620, 2604.20300].

| Mechanism | Operation | Representative papers |
|---|---|---|
| Erasure and exclusion | Remove artifacts or omit collection | [2602.21180] |
| Parameter-space unlearning | Update weights to suppress influence | [2505.19715], [2410.22374] |
| Architectural unlearning by design | Detach instance memory from core weights | [2603.15033] |
| Representation-space forgetting | Transform latent features under an information bottleneck | [2601.21564] |
| External-memory decay and pruning | Decay, delete, fuse, or gate stored memories | [2601.18642], [2604.20300], [2604.11306], [2607.09306] |
| Context-conditioned suppression | Trigger forgetting from prompt context at inference time | [2410.00382] |
| Selective adapter masking | Constrain forgetting to targeted memory regions | [2511.20196] |

Decay-based mechanisms are the most explicit descendants of cognitive models. FSFM adopts Ebbinghaus-style exponential retention, \( \mathrm{Retention}(t)=e^{-\lambda t} \), and extends it to memory-specific decay rates \( \mathrm{Retention}_i(t)=e^{-\lambda_i t}\times g(\mathrm{frequency},\mathrm{relevance},\dots) \) [2604.20300]. FadeMem generalizes this with a dual-layer memory hierarchy in which short-term and long-term stores use different exponents \( \beta_i \), adaptive decay rates \( \lambda_i(t) \), and reinforcement upon access [2601.18642]. Shin et al. implement a memory operating system with \( M(t)=S\exp(-\lambda t) \) and retrieval gating by threshold, thereby producing a “skeleton–wallpaper” partition in which enduring traces persist and surface detail decays [2607.09306].

Deletion-based mechanisms are more discrete. H\(^2\)-EMV assigns each episodic-memory node an expiration time \( \tau_n \), extends that lifetime by an LLM-estimated relevance factor \( \alpha_n \), and deletes the node if it remains expired after extension [2604.11306]. MUNKEY stores instance-specific memorization in an external memory bank \( M=\{(k_i,v_i)\}_{i=1}^N \) and unlearns by simple set difference, \( M_u=M\setminus\{(k_i,v_i)\mid i\in D_f\} \), avoiding weight updates altogether [2603.15033].

Other mechanisms are selective rather than global. LWF ranks self-generated forgetting examples by a Fisher-weighted forgetting-confidence score and periodically unlearns only the top-scoring fraction during fine-tuning [2505.19715]. SMFA first isolates a Memory Forgetting Adapter and then masks only those forgetting updates that conflict with a retaining anchor and dominate in magnitude, so that unrelated visual understanding is preserved [2511.20196]. Representation Unlearning fixes the encoder and learns a low-dimensional transform \( f_\phi \) that maximizes mutual information with retained data while suppressing information about forgotten data [2601.21564].

## 3. Mathematical formulations

A central formal distinction in the literature is between deleting artifacts and deleting influence. Makovska et al. state the standard criterion for approximate unlearning as
\[
\|\hat\theta-\theta_{D\setminus\{z\}}\|\le \epsilon
\]
or, in probabilistic models,
\[
D_{KL}(p(\cdot\mid\hat\theta)\,\|\,p(\cdot\mid\theta_{D\setminus\{z\}}))\le \delta,
\]
where \( \hat\theta=\mathcal U(D,z) \) is the output of an unlearning procedure and \( \theta_{D\setminus\{z\}} \) is the retrain-from-scratch reference [2602.21180]. This formalism makes explicit that credible forgetting is relative to a gold-standard model that never saw the removed datum.

Fisher-based approaches model forgetting as targeted deviation from a downstream optimum. In LWF, the diagonal Fisher Information Matrix on the learning task,
\[
F_i = E_{x\sim p_{\mathrm{data}}}\Bigl[\Bigl(\frac{\partial}{\partial \theta_i}\log p_\theta(x)\Bigr)^2\Bigr],
\]
weights parameter importance, and the forgetting-confidence score for a self-generated example \(x\) is
\[
FC(x)=\frac12\sum_i F_{L,i}\cdot\bigl(\theta_{\mathrm{base},i}-\alpha\,\partial\ell(x)/\partial\theta_i-\theta^*_{L,i}\bigr)^2.
\]
Examples with high \(FC(x)\) are treated as stronger candidates for unlearning because their one-step update moves parameters farther from the learning-task optimum [2505.19715].

Memory-decay systems instead formalize forgetting in time. H\(^2\)-EMV sets initial expiration as
\[
\tau_n = t_{\mathrm{end}(n)}+\Delta t_{\ell(n)}\gamma_{\ell(n)},
\]
queries an LLM for a relevance multiplier \( \alpha_n \in \{0,1,2,\dots,\infty\} \), then extends \( \tau_n\leftarrow \tau_n+\alpha_n\Delta t_{\ell(n)} \) and deletes the node if it remains expired [2604.11306]. FadeMem uses
\[
v_i(t)=v_i(\tau_i)\exp\!\bigl(-\lambda_i(t)(t-\tau_i)^{\beta_i}\bigr)
\]
with \( \beta_i=0.8 \) for long-term memory and \( \beta_i=1.2 \) for short-term memory, while importance
\[
I_i(t)=\alpha\,\mathrm{rel}(c_i,Q_t)+\beta\,\frac{f_i}{1+f_i}+\gamma\,\exp\!\bigl(-\delta\,(t-\tau_i)\bigr)
\]
governs promotion and demotion across layers [2601.18642]. Tran et al. cast in-context inference itself as a discounted Bayesian update,
\[
p_t(\theta\mid\mathcal D_{1:t})\propto p(D_t\mid\theta)\cdot[p_{t-1}(\theta\mid\mathcal D_{1:t-1})]^\gamma,
\]
with exponential weights \( w_i(t)=\exp[-\lambda(t-i)] \), thereby interpreting forgetting as a functional memory process rather than a defect [2601.09726].

Architectural unlearning introduces yet another formal pattern: move instance-specific information out of the core parameters. In MUNKEY, the memory bank \(M=\{(k_i,v_i)\}_{i=1}^N\) stores frozen keys \(k_i=g_\phi(x_i)\) and learnable exemplar tokens \(v_i\); forgetting is exactly the removal of the corresponding key–value entry [2603.15033]. Representation Unlearning replaces weight modification with the objective
\[
\max I(Z';X_r)-\beta I(Z';X_f),
\]
implemented by learning a transformation \( f_\phi:Z\to Z' \) in latent space while keeping the backbone encoder fixed [2601.21564]. These formulations differ operationally, but each makes forgetting an explicit design variable.

## 4. System architectures and algorithmic recipes

In generative language modeling, LWF is a fine-tuning framework built around self-knowledge elicitation, Fisher scoring, and periodic unlearning. The procedure is: fine-tune the base model on learning data to obtain \( \theta_L^* \); compute diagonal Fisher on the learning data; generate self-knowledge from forgetting-task prompts; compute \(FC(x)\) for each generated example; sort by descending \(FC(x)\); select the top \(K=|\mathcal D_{\mathrm{self}}|/N_u\); and interleave ordinary fine-tuning batches with periodic gradient-ascent unlearning on one selected example every \(N_u\) steps [2505.19715]. The framework is notable because it avoids architecture-incompatible unlearning machinery and uses only reversed-sign loss for the unlearning step.

In robotic episodic memory, H\(^2\)-EMV organizes experience into a rooted tree \( \mathcal H_t=(V,E) \) with scene graphs at level 1, event summaries at level 2, goal summaries at level 3, and recursive LLM-driven summaries above that. Each node stores a natural-language summary, child pointers, a parent pointer, and a time span. Designed forgetting operates top-down over this hierarchy: when a node expires, an LLM estimates relevance conditioned on current natural-language rules \( \mathcal R \); the node’s lifetime is extended or the node is replaced by a “forgotten” placeholder; and user feedback that reveals missing details is converted into revised forgetting rules \( \mathcal R_{t+1} \) by another LLM prompt [2604.11306].

Agent-memory systems broaden the design space. FSFM specifies an `UltraSafeMemoryManager` over a vector store with an `ImportanceScoringEngine` and a `SelectiveForgettingMechanism`; memories are scored by content quality, business value, temporal recency, and safety penalty, then subjected to passive decay, active deletion, safety-triggered removal, and capacity pruning [2604.20300]. FadeMem instead combines continuous decay with LLM-guided conflict resolution and memory fusion: similar memories are classified as compatible, contradictory, subsumes, or subsumed; contradictory older memories are suppressed by temporal competitive decay; and clusters of related memories are merged into a single fused memory if factual preservation exceeds a threshold [2601.18642]. Shin et al. place similar retention logic in a Lifelong Selective Memory operating system on top of a hyperbolic base model, where each trace is assigned learned salience \(S\) and decay rate \( \lambda \), and recurrently retrieved traces are periodically consolidated more deeply in the manifold [2607.09306].

In designed-forget architectures for classifiers and multimodal models, forgetting is localized structurally. Forgetting Neural Networks insert forgetting layers whose per-neuron attenuation factors \( \varphi_j(t)=\exp(-t/\tau_j) \) scale activations during the unlearning phase; the paper studies rank forget rate, ordered forget rate, top-30 forget rate, and random forget rate [2410.22374]. MUNKEY couples a ViT-Tiny backbone to an external memory bank of fixed keys and learnable exemplar tokens, trains with stochastic pathway dropout so the model cannot ignore either the image or token pathway, and thereby supports direct zero-shot forgetting through key deletion [2603.15033]. SMFA confines multimodal forgetting to adapter space: a refusal-oriented fine-tuning phase produces the Memory Forgetting Adapter, a second few-shot retain-set fine-tuning phase yields a retaining anchor, and a dual-criterion binary mask removes only those adapter entries whose forgetting gradient conflicts with preservation and dominates in relative magnitude [2511.20196].

A separate line moves forgetting to inference. In-context knowledge unlearning fine-tunes an LLM so that the presence of prompt-side unlearning tokens `<<UNL>>u<</UNL>>` causes the answer to be the special token “forgot” for target knowledge while retaining normal answers for unrelated queries [2410.00382]. This is designed forgetting at test time rather than during parameter or memory update. A plausible implication is that designed forgetting can be implemented either as a persistent state change or as a reversible control signal, depending on the deployment requirement.

## 5. Evaluation protocols and empirical findings

Empirical work evaluates designed forgetting along at least four axes: forgetting effectiveness, retained utility, efficiency or storage savings, and residual leakage. The exact metrics vary by domain. LWF reports answer accuracy on domain-specific QA tasks including gsm8k, QASC, SST-5, dental, and psychol; H\(^2\)-EMV uses QA accuracy, tree-size \(N_f\), and query-time token count \(C_{qa}\); FSFM measures storage usage, latency, throughput, signal-to-noise ratio, and dangerous or sensitive retention; MUNKEY reports Test Accuracy, Retain Accuracy, Forget Accuracy, Membership Inference Attack AUROC, and Average Gap; and Representation Unlearning reports Forget Accuracy, Retain Accuracy, \( \Delta \)CE, MIA AUC, speed-up versus retraining, and GPU memory footprint [2505.19715, 2604.11306, 2604.20300, 2603.15033, 2601.21564].

Several concrete results establish that selective forgetting can improve downstream behavior rather than merely reduce memory. On Llama3.2-1B, LWF with mixed forgetting of all non-target tasks yields average relative improvements of \(+6.95\%\) on gsm8k, \(+5.54\%\) on QASC, \(+2.10\%\) on SST-5, \(+1.46\%\) on dental, and \(+7.93\%\) on psychol; high-\(FC\) filtering outperforms low-\(FC\) filtering, and periodic unlearning outperforms both “ahead” and “random” schedules [2505.19715]. On TEACh with 5-episode histories, H\(^2\)-EMV reports memory-size reduction of approximately \(45\%\) \((94.9\to52.2\) nodes\()\), query-time compute reduction of approximately \(35\%\) \((10.2\mathrm{k}\to9.4\mathrm{k}\ \mathrm{tokens})\), maintained first-round QA accuracy \(S_c^1\approx20\%\), and second-round QA accuracy \(S_c^2\approx34\%\), described as a \(70\%\) relative gain due to learned relevance [2604.11306]. FSFM reports storage usage at \(70\%\) capacity versus \(100\%\) for baseline, average latency \(8.5\pm0.2\) s versus \(11.1\pm0.35\) s, throughput \(58.5\pm1.2\) versus \(45.0\pm1.5\) q/min, dangerous retention \(0.0\%\), sensitive retention \(54.1\%\), a \(+29.2\%\) net gain in content quality, and \(+8.49\%\) improvement in effective access efficiency [2604.20300].

Architectural designs also show strong unlearning behavior. MUNKEY, evaluated on CIFAR-10, CIFAR-100, Tiny ImageNet, PathMNIST, BloodMNIST, and DermaMNIST, achieves the lowest Average Gap across both \(10\%\) and \(2\%\) forget rates, with MIA values typically very close to \(0.5\) on \(D_f\), unlearning runtime of approximately \(0\) s, and one-time training overhead of only approximately \(5\%\) per epoch relative to a vanilla ViT [2603.15033]. Representation Unlearning on TinyImageNet reports speed-ups of approximately \(235\times\) in the standard regime and \(754\times\) in the zero-shot regime, with peak GPU memory of approximately \(1\,986\) MB and \(1\,971\) MB versus approximately \(8\,318\) MB for retraining [2601.21564]. In multimodal privacy removal, SMFA is reported to reduce text-memory ROUGE-L on the forget set close to zero at a \(5\%\) forget ratio on LLaVA-OneVision while preserving over \(90\%\) of ROUGE-L on the retain set and retaining at least \(95\%\) of original image-understanding scores [2511.20196].

Inference-time forgetting shows a different trade-off profile. On TOFU and Age, in-context knowledge unlearning with LoRA fine-tuning achieves up to \(100.0\%\) Forget and \(95.0\%\) Retain in-domain for Llama2-13B on TOFU, while out-of-domain retention can drop sharply, for example to \(31.6\%\) on TOFU world facts for the same model [2410.00382]. FNNs on MNIST and Fashion-MNIST reach approximately \(92\)–\(93\%\) test accuracy with MIA score in \([0.40,0.60]\) for the best variant, and BCL in continual learning reaches \(99.5\%\pm0.1\), \(98.7\%\pm0.1\), and \(97.3\%\pm0.2\) retained accuracy on split-MNIST and permuted-MNIST settings, with larger gains on split-CIFAR100 when the full game-theoretic mechanism is used [2410.22374, 2109.14035].

## 6. Misconceptions, governance, and open problems

A common misconception is that deletion of records is equivalent to forgetting. The data-systems literature explicitly rejects that equivalence: erasure acts on stored representations, whereas unlearning must propagate deletion through learned parameters and downstream behavior [2602.21180]. A second misconception is that successful suppression at output level necessarily implies internal erasure. The “pretend to forget” analysis in in-context knowledge unlearning shows the opposite: for forget samples, intermediate layers still place highest probability on the true answer, and only the last layer switches to output “forgot” [2410.00382].

The literature also documents collateral risks. LWF reports “superficial forgetting,” where format mismatches such as free-form versus multiple-choice QA can collapse output style if those examples are selected as forgetting data; it also notes approximately one extra epoch of overhead because computing \(FC\) requires one backward pass per self-generated sample [2505.19715]. SMFA is motivated by the observation that naive unlearning can severely degrade general visual understanding, especially on challenging ophthalmic images [2511.20196]. MUNKEY trades iterative unlearning cost for storage of one exemplar token per training sample, and Representation Unlearning depends on Gaussian approximations and Neural-Collapse proxies that may fail outside the demonstrated regime [2603.15033, 2601.21564].

Governance concerns are not peripheral. Makovska et al. argue that designed forgetting should be evaluated not only by compliance or utility retention but also by **transparency**, **accountability**, and **epistemic justice** [2602.21180]. Their proposed design principles include forgetting-by-design from day 1, layered mechanisms combining exclusion, erasure, and unlearning, semantic dependency awareness, participatory decision-making, evaluative metrics beyond accuracy, reversibility and auditability where lawful and safe, and scalable infrastructure such as sharding and incremental retraining. They also emphasize that forgetting can simultaneously protect rights and enable silencing.

Open research directions are similarly consistent across papers. LWF calls for stronger metrics of knowledge conflict, reduced overhead through approximate Fisher or subspace methods, and theoretical analysis of inter-task interference [2505.19715]. Agent-memory systems point to reinforcement-learning-based policy optimization, cross-modal forgetting, meta-learning of forgetting policies, and integration with differential privacy or federated protocols [2604.20300]. Multimodal unlearning suggests extension to continual or online unlearning and theoretical analysis of high-dimensional masking criteria [2511.20196]. Representation-space and architectural methods raise the question of how far forgetting can be shifted away from parameter updates without sacrificing verifiability [2601.21564, 2603.15033].

Taken together, these results position designed forgetting as a general systems principle: forgetting is engineered, parameterized, measured, and governed. The central challenge is no longer whether systems can forget, but how selectively, efficiently, verifiably, and justly they should do so.

Source: https://www.emergentmind.com/topics/designed-forgetting