Representation Drift in Neural Networks

• Representation drift is the evolution of a model's internal features as it adapts to new data domains, often resulting in catastrophic forgetting.
• Mitigation strategies such as regularization, dynamic parameter isolation, and replay-based methods are designed to preserve feature geometry while balancing plasticity and stability.
• Empirical studies on benchmarks like DomainNet and ImageNet-R demonstrate that drift control techniques can improve overall task accuracy despite inherent trade-offs.

Representation drift denotes the phenomenon where the internal feature representations of a machine learning system, typically a deep neural network, evolve as the model is exposed to new domains, distributions, or tasks in a sequential or continual manner. In domain-incremental learning (DIL) and related continual learning paradigms, representation drift can lead to catastrophic forgetting—loss of performance on previously encountered domains—as the system’s latent space is modified to accommodate new input statistics. Representation drift arises due to distributional shifts between domains and the model’s inability to preserve the geometric or statistical structure required for robust performance across all domains seen to date.

1. Theoretical Landscape and Formalization

In the context of DIL, representation drift materializes when a model trained on domain $D_1$ (with data distribution $P_1(x)$) is successively updated on domains $D_2, D_3, \ldots$, whose input distributions $P_t(x)$ differ from $P_1(x)$. Standard optimization on the current domain shifts parameters away from previous optima, thereby distorting the intermediate representation $\phi(x)$ extracted by the model. This effect is rigorously observed in works addressing sequential learning on benchmarks such as DomainNet, ImageNet-R, and CIFAR-10C, where curvature and alignment of latent manifolds change as new domains arrive (Geng et al., 18 Nov 2025, Ghobrial et al., 2022).

Two core mechanisms drive representation drift:

1. Plasticity Pressure: Gradient updates on new domain data reshape feature extractors and classifiers to fit new data, often at the expense of old representations.
2. Stability Constraint: Regularization, architectural isolation, or replay attempt to counteract drift by anchoring representations to their old configurations.

The empirical risk objective in general incremental learning quantifies this trade-off as follows (Xie et al., 2022): $$\mathcal{R}_t(\mathcal{M}) = \mathbb{E}_{(x,y)\sim D_t}\big[ \ell(y, \hat{y}(x;\theta_t)) \big] + \text{(regularization terms)},$$ where regularization is employed to minimize the divergence between the current and previous representations.

2. Empirical Manifestations and Metrics

Representation drift is commonly assessed using the following empirical quantities:

• Average Task Accuracy ($A_T$): Measures final accuracy on all domains after all incremental steps. Sharp drops in $A_{T,i}$ for $i < T$ indicate severe drift.
• Forgetting ($F_T$): Computed as the average performance drop on each domain after training on all others:

$$F_T = \frac{1}{T-1} \sum_{i=1}^{T-1} [ a_{T,i} - a_{i,i} ]$$

Large positive $F_T$ is a quantifiable signature of destructive representation drift (Geng et al., 18 Nov 2025, Xie et al., 2022).

• Latent Space Visualization: t-SNE or eigenvalue spectra of representation features across tasks reveal cluster collapse, loss of inter-class margin, or domain mixing (Garg et al., 2021).

Specific benchmarks highlight the magnitude of drift:

• On DomainNet, LAVA achieves average task accuracy $A_T=73.84\%$ and forgetting $F_T=0.58$ (Geng et al., 18 Nov 2025), while in image classification, catastrophic forgetting from naïve fine-tuning can reduce per-domain accuracy by 15–32 percentage points (Xie et al., 2022, Garg et al., 2021).

3. Algorithms and Architectural Mechanisms for Drift Mitigation

A taxonomy of drift-mitigation strategies encompasses the following:

• Regularization-based Methods (e.g., EWC): Penalize deviation from old weights using empirical Fisher information estimated on source domains. In DIRA, the penalty

$$R(\theta) = \frac{1}{2} \sum_j F_{0,j} (\theta_j - \theta^*_{0,j})^2$$

directly constrains representation drift, enabling robust few-shot adaptation while preserving old performance (Ghobrial et al., 2022).

• Dynamic Parameter Isolation: Partition shared and domain-specific parameters. Examples include domain-specific BatchNorm, adapters, or classifier heads that localize adjustments and minimize cross-domain interference (Mulimani et al., 23 Dec 2024, Garg et al., 2021). In semantic segmentation, dynamic isolation of 21.17% of parameters to domain-specific roles attained $Δm\% \approx 7.8\%$ (forgetting), compared to $Δm\% \approx 23.7\%$ for fine-tuning (Garg et al., 2021).
• Replay-based Methods: Maintain buffered exemplars from past domains to maintain space consistency. On realistic object detection benchmarks, replay with even 10–25% coverage sharply reduces forgetting vs. regularization alone (Neuwirth-Trapp et al., 19 Aug 2025).
• Alignment and Anchoring Mechanisms: LAVA constrains inter-class geometry in latent space via an anchor derived from class-name semantics, preserving relative feature distances across domains, thus controlling drift without strict parameter freezing (Geng et al., 18 Nov 2025).

4. Effects Across Modalities and Tasks

Representation drift is observed across vision, audio, and graph domains:

• Vision: Drift disrupts both low-level texture and high-level semantics in models trained across concrete (photographic, synthetic) and abstract (painting, sketch) domains. Without countermeasures, DIL methods exhibit large negative backward transfer ($F_T \downarrow$) and marked loss of discriminability (Geng et al., 18 Nov 2025, Neuwirth-Trapp et al., 19 Aug 2025).
• Audio: In domain-incremental acoustic scene and sound tagging, freezing all parameters blocks drift but also stymies adaptation; conversely, full fine-tuning yields destructive drift. Partitioning BatchNorm and classifier heads yields superior plasticity/stability trade-offs (Mulimani et al., 23 Dec 2024).
• Graphs: Node, link, and graph-level tasks in Domain-IL regimes display pronounced drift when instance distributions shift with domain metadata; replay-based GEM outperforms regularization at large domain counts (Ko et al., 2022).

5. Benchmarking and Quantitative Evidence

Standard domain-incremental benchmarks, such as DomainNet, ImageNet-R/C, D-RICO, and multi-domain graph datasets, are designed to expose representation drift by maximizing diversity in appearance, sensor, or structural statistics. Drift is manifested quantitatively:

Benchmark	Domains	Classes	Average Accuracy (A_T, %)	Forgetting (F_T)	Replay Impact
DomainNet (Geng et al., 18 Nov 2025)	6	345	73.84	0.58	+6.78pp $A_T$ gain with LAVA
ImageNet-Mix	30	200	78.45	0.26	+7.18pp $A_T$ with LAVA
D-RICO (Neuwirth-Trapp et al., 19 Aug 2025)	15	3	43.43 (Replay-25%)	3.35	Replay-10-25% halves FM vs. FT
OGBN-Proteins (Ko et al., 2022)	8	112	81.0 (GEM)	–	GEM> EWC> MAS on domain-IL (NC)

Catastrophic drift is reduced, but not eliminated, by architectural isolation or replay. The fundamental trade-off surfaces: low representation drift (low $F_T$) often accompanies decreased plasticity, limiting adaptation to new domain idiosyncrasies.

6. Open Challenges and Future Directions

Current approaches exhibit limitations in controlling representation drift as the number of domains, their heterogeneity, or the model complexity increase. Drawbacks include:

• Parameter Bloat: Domain-specific isolation grows parameter count linearly with domains (88–84% sharing in leading methods), complicating memory and inference (Mulimani et al., 23 Dec 2024, Garg et al., 2021).
• Replay and Privacy: Long-term storage of replay exemplars, while effective, conflicts with privacy and deployment constraints (Ko et al., 2022, Neuwirth-Trapp et al., 19 Aug 2025).
• Cross-domain Knowledge Transfer: Mechanistic understanding of how to aggregate and reuse knowledge when domains are highly disjoint remains underexplored (Geng et al., 18 Nov 2025).
• Unlabeled Domain Adaptation: Reliance on domain labels or oracles is restrictive; robust drift mitigation must generalize to domain-agnostic settings or few-shot adaptation (Ghobrial et al., 2022, Mulimani et al., 23 Dec 2024).

A plausible implication is that future methods will likely focus on compressing domain-specific capacity, developing semantically-grounded anchors for geometric consistency, and exploiting self-supervised or meta-learning techniques to enable unsupervised domain detection and drift control. Additional hybrid strategies combining parameter isolation, functional replay, and alignment regularization are emerging as promising research threads for balancing stability and plasticity in the face of complex representation drift.