Geometric Forgetting Hypothesis Overview

• Geometric Forgetting Hypothesis is a claim that forgetting results from misalignment in geometric structures like subspaces and manifolds rather than uniform decay.
• It explains catastrophic forgetting by linking update directions to overlapping gradient or tangent spaces, validated through metrics like principal angles and recoverability correlations.
• The hypothesis spans multiple domains—including continual learning, unlearning, and semantic memory—offering actionable insights for designing models that mitigate information loss.

Searching arXiv for recent and directly relevant papers on the geometric forgetting hypothesis and closely related formulations. arxiv_search(query="Geometric Forgetting Hypothesis forgetting geometry subspace representation forgetting", max_results=10, sort_by="relevance") to=arxiv_search 娱乐平台招商 彩神争霸苹果 天天中彩票是不是 նկար {"query":"Geometric Forgetting Hypothesis forgetting geometry subspace representation forgetting", "max_results": 10, "sort_by": "relevance"} to=arxiv_search {"query":"Geometric Forgetting Hypothesis forgetting geometry subspace representation forgetting", "max_results": 10, "sort_by": "relevance"} The geometric forgetting hypothesis is the claim that forgetting is governed primarily by the geometry of representations, gradients, subspaces, manifolds, or retrieval neighborhoods rather than by undifferentiated decay. In recent arXiv work, that claim appears in several non-equivalent forms: as a principal-angle law for catastrophic forgetting in LoRA continual learning, as motion off a task-preserving Banach submanifold in neural ODEs, as orthogonality to retain-gradient subspaces in machine unlearning, as accessibility collapse caused by subspace misalignment in continual vision models, and as interference among nearby semantic embeddings that yields power-law forgetting and false memory (Steele, 10 Feb 2026, Bayram et al., 3 Sep 2025, Zhou et al., 21 Nov 2025, Trivedi et al., 11 Jun 2026, Barman et al., 27 Mar 2026). Taken together, these formulations suggest not a single doctrine but a family of hypotheses in which geometry specifies what is forgotten, what remains recoverable, and why retrieval or retention fails.

1. Conceptual scope and representative formulations

Across the literature, the hypothesis does not refer to one universal geometric object. In continual learning, the relevant object may be a gradient subspace; in unlearning, a retain-gradient tangent or orthogonal complement; in dynamical systems, a manifold of mapping-preserving controls; in semantic memory, a retrieval neighborhood induced by cosine similarity; and in operator learning, the geometry of the physical domain itself (Steele, 10 Feb 2026, Zhou et al., 21 Nov 2025, Bayram et al., 3 Sep 2025, Barman et al., 27 Mar 2026, Xia et al., 7 May 2026). This diversity matters because different papers use “forgetting” to mean loss increase on old tasks, loss of exact endpoint preservation, residual discrepancy relative to retraining, reduced basin of attraction, exponential damping of a memory kernel, or decaying retrieval under competition.

Domain	Geometric object	Representative formulation
LoRA continual learning	Minimum principal angle between task gradient subspaces	$$F_{i,t} \leq \alpha \cdot (1 - \cos^2(\theta_{i,t})) + \beta$$ (Steele, 10 Feb 2026)
Neural ODE sequential tuning	Banach submanifold of mapping-preserving controls	$\bigcap_{i=1}^j U(x^i,y^i)$ with tangent space $\bigcap_{i=1}^j \ker(u^k,x^i)$ (Bayram et al., 3 Sep 2025)
Gradient-based unlearning	$H$-orthogonal complement of the retain-gradient span	retain-invariant iff $\Delta\theta \in T_r(\theta)^\perp$ (Zhou et al., 21 Nov 2025)
Continual vision recoverability	Recovery subspace and principal-angle drift	$k_t \approx 8$, with recoverability strongly predicted by drift (Trivedi et al., 11 Jun 2026)
Semantic memory	Retrieval neighborhoods in low effective dimension	interference among competitors yields power-law forgetting (Barman et al., 27 Mar 2026)

A minimal common core nevertheless recurs. Forgetting is treated as a failure of alignment between a current state and a task-relevant geometric structure: a prior-task gradient span, a task-preserving manifold, a recoverable subspace, a retraining-consistent direction, or a semantic neighborhood. This suggests that the broad hypothesis is best understood as a structural thesis about representation and access, not as a claim that all forgetting laws share the same equation.

2. Subspace geometry in continual learning

The most explicit continual-learning formulation appears in LoRA-based adaptation. In that setting, forgetting on task $i$ after learning task $t$ is defined at the loss level as $F_{i,t} = L_i(\theta_t) - L_i(\theta_{t-1})$, and the paper proposes the bound $$F_{i,t} \leq \alpha \cdot (1 - \cos^2(\theta_{i,t})) + \beta$$, where $\bigcap_{i=1}^j U(x^i,y^i)$0 is the minimum principal angle between the task gradient subspaces $\bigcap_{i=1}^j U(x^i,y^i)$1 and $\bigcap_{i=1}^j U(x^i,y^i)$2, $\bigcap_{i=1}^j U(x^i,y^i)$3, and $\bigcap_{i=1}^j U(x^i,y^i)$4 captures non-geometric sources of forgetting (Steele, 10 Feb 2026). The associated geometric mechanism is a decomposition of the new update $\bigcap_{i=1}^j U(x^i,y^i)$5 into components parallel and orthogonal to the earlier task subspace, so that first-order interference depends on projected overlap with prior-task directions.

That same paper emphasizes that the proposed law is a “geometric bound with empirically validated parameterization,” not a strict exact theorem, and that the specific $\bigcap_{i=1}^j U(x^i,y^i)$6 parameterization is empirical. It also reports a regime-dependent “approximate rank-invariance” in high-angle settings: in synthetic tasks the correlation between interference and forgetting is $\bigcap_{i=1}^j U(x^i,y^i)$7, predicted-actual forgetting correlation is $\bigcap_{i=1}^j U(x^i,y^i)$8, $\bigcap_{i=1}^j U(x^i,y^i)$9, and rank CV across ranks $\bigcap_{i=1}^j \ker(u^k,x^i)$0–$\bigcap_{i=1}^j \ker(u^k,x^i)$1 is $\bigcap_{i=1}^j \ker(u^k,x^i)$2; on Split-CIFAR100 and sequential GLUE, rank sweep CVs are $\bigcap_{i=1}^j \ker(u^k,x^i)$3 and $\bigcap_{i=1}^j \ker(u^k,x^i)$4, respectively (Steele, 10 Feb 2026). The same study reports that rank-forgetting correlation is $\bigcap_{i=1}^j \ker(u^k,x^i)$5 in a low-angle regime and $\bigcap_{i=1}^j \ker(u^k,x^i)$6 in a high-angle regime, reconciling prior claims that higher rank worsens forgetting with its own high-angle rank-invariance observation.

An older but conceptually related line treats forgetting through curvature and Fisher geometry. In a sequentially trained Hopfield network, the diagonal Fisher term is $\bigcap_{i=1}^j \ker(u^k,x^i)$7, interpreted as curvature of the log-probability or energy landscape with respect to a weight, and local consolidation rules reduce plasticity for high-curvature directions, thereby attenuating interference between overlapping memories (Deistler et al., 2018). The same work argues that in the sparse-coding limit this curvature can be approximated from local synaptic information, linking global information-geometric structure to local statistics.

Taken together, these results suggest a specific continual-learning version of the hypothesis: forgetting is anisotropic. It is concentrated in directions defined by subspace overlap or high curvature, while other directions remain comparatively safe for further learning.

3. Manifolds, tangent spaces, and orthogonality in non-forgetting and unlearning

In neural ODEs, the geometric forgetting hypothesis is formulated as an exact constraint-manifold statement rather than a subspace-overlap law. For a finite training set $\bigcap_{i=1}^j \ker(u^k,x^i)$8, the controls preserving already learned endpoint mappings form $\bigcap_{i=1}^j \ker(u^k,x^i)$9, and the main theorem states that this space is a Banach submanifold of $H$0 of finite codimension under memorization, linearized controllability, and transversal intersection assumptions (Bayram et al., 3 Sep 2025). In this picture, forgetting occurs when an update leaves the manifold, whereas non-forgetting is motion tangent to it. The corresponding tangent characterization is $H$1, so allowable infinitesimal updates are exactly those annihilated by the differential of the old-task endpoint constraints.

A closely related first-order formulation appears in machine unlearning. There the retain-gradient subspace is $H$2, equipped with an optimizer-induced SPD metric $H$3, and the paper proves that an update is retain-invariant if and only if it lies in $H$4 (Zhou et al., 21 Nov 2025). Any forget gradient $H$5 is decomposed into tangential and normal components, $H$6, and the projected direction $H$7 is optimal among all first-order retain-invariant trust-region moves. The same paper derives the split update $H$8, making orthogonality the operative notion of disentanglement.

Representation-misdirection unlearning pushes the same idea into concept-vector geometry. There, forgetting is implemented by linear operations on forget representations $H$9 relative to a concept direction $\Delta\theta \in T_r(\theta)^\perp$0: additive steering $\Delta\theta \in T_r(\theta)^\perp$1 and ablative removal $\Delta\theta \in T_r(\theta)^\perp$2 change concept-relevant odds through the alignment $\Delta\theta \in T_r(\theta)^\perp$3 between latent and unembedding directions (Dang et al., 29 Jan 2026). The same paper reports that such target vectors can produce controllable side behaviors in truthfulness, sentiment, refusal, and in-context learning tasks. This suggests that some unlearning procedures do not erase a concept so much as steer access to it along a chosen representational axis.

These works converge on a common geometric motif. Forgetting is minimized not by uniformly shrinking updates but by respecting a constraint geometry: stay on the task-preserving manifold, or move in the orthogonal complement of the retain-sensitive tangent space, or manipulate only the intended concept direction.

4. Representation-space auditing, recoverability, and retraining consistency

A separate line of work argues that output-level forgetting can systematically overestimate what has been forgotten. One paper formalizes “output forgetting” as agreement of output distributions with retraining and “retraining-consistent representation forgetting” as joint agreement of representation distributions and transformation distributions relative to the original model (Yong et al., 23 Jun 2026). Its Theorem 1 shows that output agreement can coexist with representation-level residuals: with a linear head $\Delta\theta \in T_r(\theta)^\perp$4, one can shift forget-set representations by a nonzero $\Delta\theta \in T_r(\theta)^\perp$5 so that outputs remain unchanged while hidden distributions differ. Empirically, the same paper reports structured mismatch, not diffuse error: forget/retain asymmetry, directional mismatch, and residual discrepancy concentrated along retraining-related directions.

The accessibility-collapse framework sharpens that diagnosis in continual learning. In a sequentially trained ResNet-18 on Split CIFAR-100, active Task 0 accuracy collapses from $\Delta\theta \in T_r(\theta)^\perp$6 to $\Delta\theta \in T_r(\theta)^\perp$7, but recoverability from frozen representations remains substantial, with mean recoverability $\Delta\theta \in T_r(\theta)^\perp$8 and accessibility gap $\Delta\theta \in T_r(\theta)^\perp$9 averaging $k_t \approx 8$0 (Trivedi et al., 11 Jun 2026). The paper introduces Recovery Subspace Dimensionality $k_t \approx 8$1, defined as the minimum rank of the linear probe weight matrix preserving at least $k_t \approx 8$2 of full probe accuracy; across ten tasks, $k_t \approx 8$3 with mean $k_t \approx 8$4 and $k_t \approx 8$5. It further reports that principal-angle drift strongly predicts recoverability with $k_t \approx 8$6, and a geometric regression model explains $k_t \approx 8$7 of recoverability variance. The paper’s conclusion is explicit: catastrophic forgetting is primarily an accessibility and manifold-alignment problem rather than information destruction.

The same retraining-relative logic appears in localized collateral forgetting. In approximate unlearning, discrepancy between the unlearned model and the retrained reference is highly non-uniform and grows with geometric proximity to the forget set in representation space (Dolgova et al., 29 May 2026). Theoretical results show that for gradient ascent the excess squared loss scales with $k_t \approx 8$8, while for random-labeling methods the discrepancy is governed by the query’s projection onto the forget span. The proposed mitigation, Local Teacher Distillation, replaces random targets with soft labels from a small teacher trained only on retained neighbors of the forget set.

A related representational result concerns temporal knowledge drift rather than task forgetting in the standard sense. There, temporal drift is encoded as a direction in residual-stream space geometrically orthogonal to both correctness and uncertainty, with a linear probe achieving AUROC $k_t \approx 8$9–$i$0 while entropy-based and correctness-based baselines remain near chance at $i$1–$i$2 (Elbadry et al., 9 May 2026). This broadens the article’s theme: stale knowledge, accessibility failure, and apparent forgetting can occupy distinct representational axes that are invisible at the output layer.

Taken together, these studies suggest that a geometric forgetting hypothesis should not be evaluated only by loss or accuracy after training. It must also ask whether the forgotten information is still present but misaligned, localized, or hidden in directions that standard output metrics do not audit.

5. Interference, semantic neighborhoods, and attractor basins

In semantic memory systems, the geometric forgetting hypothesis becomes a theory of retrieval neighborhoods. One paper shows that power-law forgetting arises from interference among competing memories in embedding space rather than from decay alone: with competitors, the forgetting exponent is $i$3, close to the human benchmark $i$4, whereas the identical decay function without competitors yields $i$5 (Barman et al., 27 Mar 2026). The same work reports that production embedding models with nominal dimension $i$6–$i$7 concentrate variance in only about $i$8 effective dimensions, placing them in an interference-vulnerable regime, and that cosine similarity on unmodified pre-trained embeddings reproduces the Deese–Roediger–McDermott false alarm rate $i$9 versus human $t$0. Its conclusion is explicit: time alone does not produce forgetting in that system; competition does.

A companion theorem-oriented paper makes the same point at the level of model class. For semantically continuous kernel-threshold memories, it derives four structural results: semantically useful representations have finite effective rank; finite local dimension implies positive competitor mass in retrieval neighborhoods; under growing memory, retention decays to zero, yielding power-law forgetting curves under power-law arrival statistics; and for associative lures satisfying a $t$1-convexity condition, false recall cannot be eliminated by threshold tuning (Barman et al., 28 Mar 2026). Empirically, the same paper tests vector retrieval, graph memory, attention-based context, BM25 filesystem retrieval, and parametric memory, and argues that systems escaping interference entirely do so by sacrificing semantic generalization.

Earlier attractor-network work supplies an older geometric precursor. In Parisi’s bounded-synapse “memory which forgets,” recent patterns remain recognizable, but the basin of attraction of recognized patterns decreases exponentially with age, making old memories practically inaccessible even when they remain exact-cue recognizable (Marinari, 2018). The authors explicitly call this exponentially tiny basin “a clearly non physiological feature,” so the paper supports a geometric age effect for accessibility but not as a satisfactory biological forgetting law.

Not all papers support exponential or geometric decay as a general rule. A retroactive-interference model of memory retention derives $t$2 in one dimension and $t$3 asymptotically in higher dimensions, thereby challenging constant-hazard geometric forgetting and arguing instead for decreasing hazard with age (Georgiou et al., 2019). In another domain, open quantum systems satisfying a specific ETH-compatible condition yield an exact continuous-time exponential forgetting law at the memory-kernel level, $t$4, under Lindblad dephasing in the observable’s eigenbasis (Knipschild et al., 2019). These results show that “geometric forgetting” can refer either to interference geometry that yields heavy-tailed retention or to exponential memory-kernel damping, depending on domain and formalism.

6. Domain-specific extensions: geometry of tasks, domains, and data

Some work makes geometry the object being forgotten rather than the mechanism of forgetting. In deep operator learning, the Geometric Forgetting Hypothesis is formalized as a depth-wise failure mode: standard operator layers satisfy $t$5, inducing the Markov chain $t$6 for geometry encoding $t$7, and therefore $t$8 by data processing (Xia et al., 7 May 2026). The paper argues that global mixing layers in Fourier and attention-based neural operators progressively lose access to domain geometry, degrading accuracy, stability, and generalization on irregular domains. Its remedy is explicit geometry memory injection, $t$9, which restores geometric constraints at intermediate depths.

In point-cloud continual learning, geometry enters as a property of the data that exacerbates forgetting. One study on incremental 3D object learning argues that catastrophic forgetting is amplified by the irregular and redundant geometric structures of point cloud data, and proposes an adaptive-geometric centroid module plus a geometric-aware attention mechanism to preserve unique local geometric characteristics (Dong et al., 2020). On ModelNet40, ShapeNet, and ScanNet, the full model outperforms variants without adaptive geometry or geometric attention, and those ablations hurt more than removing the score fairness compensation component. This suggests that, in that setting, forgetting is not only class imbalance or replay scarcity but also a failure to preserve the right local geometric structures.

A related but more abstract extension concerns temporal drift in factual knowledge. Temporal drift is encoded as an independent axis in residual-stream space, orthogonal to correctness and uncertainty, so a model can confidently produce outdated answers while remaining invisible to entropy- or correctness-based screening (Elbadry et al., 9 May 2026). Although this is not catastrophic forgetting in the continual-learning sense, it broadens the family resemblance: geometry can separate stale recall from uncertainty, just as it can separate retain-invariant from retain-destructive updates, or accessible from inaccessible old-task structure.

Taken together, these papers suggest that the hypothesis has at least three levels: geometry can be the cause of forgetting, the substrate in which forgetting becomes measurable, or the object that is itself forgotten.

7. Limits, controversies, and open problems

The literature is unusually explicit about its limits. The LoRA principal-angle law is a “geometric bound with empirically validated parameterization,” not a fully deductive exact theorem, and its fitted sign can appear counterintuitive because the observed regime associates larger subspace angles with larger empirical forgetting (Steele, 10 Feb 2026). The neural ODE Banach-submanifold result is exact but highly specialized to smooth controlled dynamics, finite datasets, bounded controls, and transversal intersections (Bayram et al., 3 Sep 2025). Gradient-space unlearning guarantees are local and first-order, with approximation error depending on the quality of the retain-subspace estimate (Zhou et al., 21 Nov 2025). Retraining-consistent representation forgetting is offered as a stronger evaluative lens, not as a universal axiom for every machine-unlearning setting (Yong et al., 23 Jun 2026).

There is also a substantive dispute over forgetting laws. Some papers support exponential or approximately geometric forms, as in memory-kernel damping under decoherence and exponential basin shrinkage with age (Knipschild et al., 2019, Marinari, 2018). Others argue that the more appropriate law is power-law retention arising from retroactive interference, with older memories becoming more resilient because surviving items are selected for high valence in one or more dimensions (Georgiou et al., 2019). The semantic-memory papers add yet another nuance by locating the relevant mechanism in effective dimensionality and competitor mass rather than in time alone (Barman et al., 27 Mar 2026, Barman et al., 28 Mar 2026).

A final open issue concerns universality. Some results are explicitly broad in ambition, such as the claim that any system that organizes information by meaning and retrieves it by proximity should exhibit forgetting and false recall (Barman et al., 27 Mar 2026). Others are tightly domain-bound: ViT-LoRA and RoBERTa-LoRA, neural ODEs, Hopfield networks, class-unlearning benchmarks, point-cloud object learning, or neural operators on irregular PDE domains (Steele, 10 Feb 2026, Bayram et al., 3 Sep 2025, Deistler et al., 2018, Dong et al., 2020, Xia et al., 7 May 2026). Taken together, these papers suggest that the geometric forgetting hypothesis is best viewed neither as a single theorem nor as a loose metaphor. It is a research program: forgetting is to be explained by the geometry of what is stored, what competes, what remains accessible, and what directions learning updates are allowed to occupy.