Domain-Incremental Learning Overview

• Domain-incremental learning is a continual learning approach where the label space stays fixed while input distributions change over time.
• It addresses catastrophic forgetting using techniques like replay, regularization, and domain-specific modules without relying on old-domain data.
• The approach spans diverse applications—including image, video, time-series, audio, and federated learning—highlighting both domain-aware and domain-agnostic inference challenges.

Domain-incremental learning (DIL) is a continual-learning setting in which a model is trained on a sequence of domains while the label space remains fixed and the input distribution changes over time. In this setting, the learner must retain performance on previously observed domains while adapting to newly arriving ones, often without access to old-domain data and often without domain identity at inference. Recent work extends this formulation across image classification, video understanding, time-series classification, audio analysis, medical imaging, natural language processing, and federated learning, and studies variants that are memory-based, rehearsal-free, source-free, few-shot, one-shot, open-world, and streaming (Esaki et al., 2024, Lamers et al., 2023, Shi et al., 2023).

1. Formalization and scope

A common mathematical formulation represents DIL as a sequence of domains. In domain-incremental time-series classification, for example, the sequence is written as

$$\mathbb{D} = \{\mathcal{D}^1, \mathcal{D}^2, \cdots, \mathcal{D}^T\},$$

with samples $$\mathcal{D}^{t}=\{\mathbf{X}_n^t,\mathbf{Y}^t_n\}_{n=1}^{N_t}$$, where the label space is fixed and only the domain or data distribution changes across increments (Liu et al., 15 Jan 2026). Closely related formulations in image and video DIL describe a sequence of tasks or domains $\{D_1,\dots,D_T\}$ or $\{\mathcal{D}_1,\dots,\mathcal{D}_T\}$ with shared classes and sequential access to current-domain data only (Wang et al., 23 Mar 2025, Mulimani et al., 2024).

Within this shared core, the literature distinguishes several operational regimes. Many works treat DIL as task-agnostic at inference, meaning that task or domain identity is unavailable and the model must classify inputs across all seen domains using a single shared output space (Lamers et al., 2023, Esaki et al., 2024). Other works study domain-aware settings in which domain identity is used to select domain-specific parameters, often as an upper reference or as a deployment assumption (Mulimani et al., 2024, Mulimani et al., 2024). Open-world variants go further by evaluating on unseen tasks with task identity unavailable at test time, as in prompt-based lifelong learning for NLP (Dai et al., 2023). Federated formulations add decentralized, non-IID clients and local privacy constraints while preserving the same fixed-label, changing-domain semantics (Sun et al., 2024, Li et al., 2024).

The term also covers markedly different data regimes. Standard DIL assumes sequential access to full current-domain datasets, but recent work introduces one-shot DIL, where adaptation uses exactly one labeled sample from a new domain of an existing class (Esaki et al., 2024), and few-shot DIL, where the base domain is data-rich but all subsequent domains are $N$-way $K$-shot (Paeedeh et al., 29 Jun 2026). Streaming variants emphasize temporally correlated test data and gradual domain shifts rather than i.i.d. evaluation over mixed domains (Swinnen et al., 29 May 2026). This suggests that DIL is less a single benchmark protocol than a family of fixed-label continual-learning problems organized around domain shift.

2. Central technical difficulties

The canonical difficulty in DIL is catastrophic forgetting: after adapting to a new domain, the model loses performance on previously learned domains. This concern appears in application-specific form across home-environment action recognition, chest X-ray analysis, acoustic scene classification, and domain-shifted image benchmarks (Hu et al., 2024, Kim, 18 May 2026, Mulimani et al., 2024, Wang et al., 23 Mar 2025). In memory-based settings, the difficulty is sharpened by the fact that only a small subset of old-domain data is available, making direct empirical risk estimates over previous domains noisy (Shi et al., 2023).

A recurring interpretation is that forgetting is driven by representation drift. DARE explicitly frames DIL in terms of abrupt representation drift at task boundaries and proposes a three-stage training process—Divergence, Adaptation, and REfinement—together with Intermediary Reservoir Sampling to reduce drift in the feature encoder (Jeeveswaran et al., 2024). Duct sharpens this diagnosis by arguing that DIL forgetting is not only a feature-drift problem but also a representation–classifier compatibility problem: sequential updates overwrite both the representation and the classifier, so preserving only one of them is insufficient (Zhou et al., 2024).

Several papers identify modality-specific mechanisms behind this drift. In one-shot DIL, the decisive failure mode is attributed to batch normalization statistics: with only one repeatedly augmented target sample, BN running mean and variance become unreliable, and standard DIL baselines can harm both new-domain and original-domain performance (Esaki et al., 2024). In medical imaging, forgetting interacts with class imbalance, so that naive replay can preserve old-domain information while still underrepresenting minority classes; class-aware memory balancing and class-aware loss reweighting are introduced specifically to address that interaction (Kim, 18 May 2026). In audio DIL, domain mismatch can be expressed strongly through feature statistics, motivating domain-specific BN statistics and affine parameters (Mulimani et al., 2024).

A further difficulty is hidden task structure. Many successful continual-learning strategies assume known task boundaries or task labels, but task-agnostic DIL removes both assumptions (Lamers et al., 2023). This affects not only training but also inference: several audio and challenge settings show that selecting the correct domain-conditioned branch can be a major bottleneck, sometimes more so than the continual adaptation procedure itself (Casciotti et al., 1 Jun 2026, Mulimani et al., 2024). In streaming video, the problem is reformulated again: domain identity is unknown, but consecutive samples are highly correlated, so domain recovery can be attempted online rather than by a static classifier (Swinnen et al., 29 May 2026).

3. Methodological families

Recent DIL methods span replay, regularization, consolidation, parameter isolation, prompt/prototype mechanisms, and test-time adaptation. A useful organizing principle is whether the method preserves old knowledge by storing data, by constraining parameters or representations, by allocating domain-specific capacity, or by reconstructing a domain-dependent routing signal at inference.

Family	Representative mechanism	Example papers
Memory-based	Replay, distillation, adaptive coefficients	(Shi et al., 2023, Kim, 18 May 2026)
Rehearsal-free	Prototypes, disentanglement, consolidation	(Wang et al., 23 Mar 2025, Zhou et al., 2024, Liu et al., 15 Jan 2026)
Domain-specific modules	BN branches, LoRAs, prompts, adapters	(Mulimani et al., 2024, Mulimani et al., 2024, Sun et al., 2024, Paeedeh et al., 29 Jun 2026)
Task-agnostic routing	Clustering, task-ID inference, online TTT	(Lamers et al., 2023, Bravo-Rocca et al., 2023, Swinnen et al., 29 May 2026)

Memory-based DIL remains prominent. Replay can be made class-aware, as in chest X-ray DIL where memory is partitioned by class and replay batches are balanced under a fixed memory budget (Kim, 18 May 2026). UDIL provides a more general theoretical synthesis: it shows that replay, intra-domain distillation on memory, and cross-domain distillation on current-domain data can all be written within a unified bound with adaptive coefficients $\{\alpha_i,\beta_i,\gamma_i\}$, and that fixed schemes such as ER and LwF correspond to special cases (Shi et al., 2023). Task-agnostic projection-based methods add online clustering to maintain representative pools of samples or gradients without known task boundaries, yielding task-agnostic versions of A-GEM and OGD (Lamers et al., 2023).

Rehearsal-free methods instead attempt to encode domain-robust structure in the model itself. DualCD is a rehearsal-free representation-learning framework for time-series DIL that disentangles temporal features into class-causal and spurious parts and then applies intra-class and inter-class causal interventions (Liu et al., 15 Jan 2026). DualCP anchors rehearsal-free image DIL to dual-level concept prototypes, generating coarse-grained and fine-grained semantic prototypes and aligning calibrated image features to them with a Dual Dot-Regression loss (Wang et al., 23 Mar 2025). Duct consolidates knowledge at two levels by merging historical backbones into a single balanced representation space and then transporting old classifier weights into the latest embedding space with semantic optimal transport (Zhou et al., 2024).

Domain-specific capacity allocation is another dominant pattern. In audio DIL, ADIL freezes a domain-shared CNN backbone and adds domain-specific BN parameters and classifier heads for each new domain, with residual combination between the base classifier and the domain-specific classifier (Mulimani et al., 2024). ODIL goes further toward efficiency by updating only BN running statistics with a few unlabeled samples and storing domain-specific statistics for later reuse (Mulimani et al., 2024). In federated rehearsal-free DIL, RefFiL avoids replay by exchanging clustered prompt representations rather than data and adds a domain-specific prompt contrastive loss (Sun et al., 2024). Few-shot DIL via CVLC combines shared and domain-specific Dual Coalescent Projection modules in a CLIP backbone, multimodal prototypes, and latent-space reservation during base-domain training (Paeedeh et al., 29 Jun 2026).

Some recent work departs from the standard anti-forgetting objective altogether. “Remembering by reconstructing” trains one LoRA adapter per domain and intentionally allows each adapter to specialize and forget other domains; at test time, only the mixture coefficients of frozen LoRAs are optimized online by minimizing MAE reconstruction loss on the current video stream (Swinnen et al., 29 May 2026). This reverses the usual logic of DIL: specialization-induced forgetting becomes the signal used for domain recovery.

4. Benchmarks, protocols, and evaluation

DIL evaluation is heterogeneous, and benchmark design is itself an active research topic. Domain streams may be synthetically induced, as in permutation and rotation variants of MNIST (Shi et al., 2023), or drawn from natural subject, location, institution, or dataset shifts, such as HAR/HHAR/ISRUC-S3/Sleep-EDF in time-series DIL (Liu et al., 15 Jan 2026), PneumoniaMNIST with five simulated domains in medical imaging (Kim, 18 May 2026), and CORe50, DomainNet, Office-Home, and CDDB in visual rehearsal-free DIL (Wang et al., 23 Mar 2025, Zhou et al., 2024). Video DIL has only recently been formalized: Video Domain Incremental Learning for home action recognition introduces user, scene, and hybrid split types specifically to assess domain shifts in unconstrained home environments (Hu et al., 2024).

Two metrics recur throughout the literature: average accuracy and forgetting. In UDIL, the average accuracy after learning task $t$ is

$A_t=\frac{1}{t}\sum_{i=1}^{t} R_{t,i},$

with forgetting reported alongside it on standard DIL benchmarks (Shi et al., 2023). Time-series DIL extends this with relative forgetting and performance-aware relative forgetting (PRF): $$\text{RF}=\frac{1}{T-1}\sum_{j=1}^{T-1}\left(\max_{i \in \{1,2,\ldots,T-1\}} \frac{(a^i_j-a^T_j)}{a^i_j} \right),$$

$$\mathcal{D}^{t}=\{\mathbf{X}_n^t,\mathbf{Y}^t_n\}_{n=1}^{N_t}$$0

motivated by the observation that RF can look artificially favorable for weak models that never learned the earlier domains well (Liu et al., 15 Jan 2026). Audio tagging uses task-appropriate metrics such as lwlrap in multi-label DIL (Mulimani et al., 2024).

Benchmark protocols also differ in whether domain identity is supplied at inference. Some methods are evaluated in both domain-aware and domain-agnostic modes, often with substantial performance gaps when the correct branch must be inferred by entropy or other routing heuristics (Mulimani et al., 2024, Mulimani et al., 2024). The DCASE 2026 challenge formalizes the harder domain-agnostic incremental learning setting for sound classification across three domains and ranks systems by overall average class-wise accuracy across domains (Casciotti et al., 1 Jun 2026). Task-agnostic DIL studies such as TADIL treat domain discovery itself as part of the benchmark, combining transformer embeddings, density clustering, nearest-centroid representatives, and drift detection to decide when a new task has appeared online (Bravo-Rocca et al., 2023).

This diversity of protocols complicates direct comparison. Some results assume task boundaries, some assume task identity at test time, some evaluate on continuous streams, and some use upper-reference “joint” or “disjoint” models that violate the no-old-data constraint. A plausible implication is that reported gains in DIL should be interpreted relative to the exact inference and memory assumptions, not only to the dataset name.

5. Application domains and modality-specific formulations

The breadth of recent DIL applications underscores that the fixed-label, changing-domain assumption is common across deployed systems. In human action recognition for home environments, DIL is motivated by unconstrained users and scenes, and the fixed action vocabulary makes the problem distinct from class-incremental learning (Hu et al., 2024). In medical imaging, the diagnostic task remains constant while domains shift because of imaging devices, acquisition protocols, and institutional conditions; replay-based DIL for pneumonia detection is motivated explicitly by privacy restrictions and the operational impracticality of storing all historical radiographs (Kim, 18 May 2026). Source-free unsupervised DIL pushes this logic further by replacing past data with Gaussian latent prototypes and adapting to new unlabeled target domains adversarially (Ambastha et al., 2023).

Audio provides several distinct DIL formulations. Acoustic scene classification across cities and recording conditions has been studied both through normalization-only adaptation with a few unlabeled samples (Mulimani et al., 2024) and through a dynamic architecture that keeps shared acoustic characteristics in a frozen backbone while learning domain-specific BN parameters and classifier heads (Mulimani et al., 2024). A separate challenge track defines domain-agnostic incremental sound classification with ten shared classes across three domains and highlights domain inference itself as a primary source of error (Casciotti et al., 1 Jun 2026). These audio works jointly show that DIL can be framed either as domain-conditioned parameter selection or as domain-agnostic single-model robustness, depending on deployment assumptions.

Time-series DIL treats subjects or users as domains and emphasizes that class-conditional temporal patterns can vary substantially across domains even when class semantics remain fixed. DualCD introduces a rehearsal-free causal disentanglement framework for this setting and also contributes a benchmark over HAR, HHAR, ISRUC-S3, and Sleep-EDF (Liu et al., 15 Jan 2026). Streaming video DIL adds yet another dimension: because consecutive test samples are highly correlated, online test-time training can exploit stream structure to recover the appropriate domain expert (Swinnen et al., 29 May 2026).

Federated and open-world variants broaden the scope further. RefFiL addresses rehearsal-free federated DIL through prompt sharing rather than replay under privacy and resource constraints (Sun et al., 2024), while pFedDIL maintains multiple personalized local models and uses adaptive knowledge matching to decide whether a new task should reuse or diverge from previous local knowledge (Li et al., 2024). In NLP, Diana formulates open-world domain-incremental lifelong learning with prompt-enhanced LLMs, hierarchical prompts, and explicit unseen-task prompts, making test-time task unavailability a first-class requirement rather than a nuisance (Dai et al., 2023). Collectively, these papers indicate that DIL is increasingly treated as a deployment problem rather than a purely synthetic benchmark setting.

6. Open issues, controversies, and research directions

One unresolved issue is the role of domain identity at inference. Some methods and challenge baselines achieve strong retention by storing domain-specific statistics or branches, but then require the correct domain identifier or a domain-selection heuristic at test time (Mulimani et al., 2024, Mulimani et al., 2024). Other works explicitly argue that domain-agnostic inference is the more realistic DIL setting and build clustering, auxiliary classifiers, or online reconstruction criteria to infer the active domain (Bravo-Rocca et al., 2023, Swinnen et al., 29 May 2026). The resulting comparison space is not uniform: a method that presumes a known domain ID is solving a different problem from one that must infer it online.

A second open question concerns what should be stabilized. DARE centers representation drift (Jeeveswaran et al., 2024), Duct argues for joint representation and classifier consolidation (Zhou et al., 2024), one-shot DIL identifies BN statistics as the bottleneck under extreme data scarcity (Esaki et al., 2024), and DualCD frames DIL as a representation-selection problem in which the model must rely on class-causal rather than spurious temporal features (Liu et al., 15 Jan 2026). These diagnoses are compatible, but they imply different algorithmic levers: replay, normalization control, semantic prototypes, parameter isolation, or classifier transport.

The divide between memory-based and rehearsal-free DIL also remains active. Memory-based approaches can be theoretically unified and adaptively weighted, as in UDIL (Shi et al., 2023), but many realistic deployments impose privacy or storage constraints that favor rehearsal-free methods such as DualCP, RefFiL, Duct, or CVLC (Wang et al., 23 Mar 2025, Sun et al., 2024, Zhou et al., 2024, Paeedeh et al., 29 Jun 2026). This suggests that “better” DIL methods may increasingly be judged by the realism of their constraints as much as by raw accuracy.

Finally, recent extensions point toward more demanding formulations rather than closure. Open-world DIL in NLP requires handling unseen tasks without test-time task labels (Dai et al., 2023). Federated DIL requires personalized transfer under client heterogeneity (Li et al., 2024). Cross-dataset generalization is explicitly left open in time-series DIL, where current methods are developed for within-dataset domain increments rather than incompatible variable spaces across datasets (Liu et al., 15 Jan 2026). Streaming video DIL suggests that future systems may combine continual learning, domain routing, and test-time adaptation in a single loop rather than treat them as separate problems (Swinnen et al., 29 May 2026).

In that sense, contemporary DIL research is converging on a more precise view: the problem is not merely to avoid forgetting, but to decide which information should remain shared across domains, which should stay domain-specific, how that structure should be recovered at inference, and how these choices change under memory, privacy, scarcity, and streaming constraints.