MTIL: Multi-Domain Task Incremental Learning

• MTIL is a continual learning framework where models adapt to sequential tasks from distinct domains under limited access to previous data.
• It employs architectural strategies like frozen backbones with task-specific updates to mitigate forgetting and ensure both retention and forward transfer.
• MTIL leverages selective routing, attention mechanisms, and modular expert designs, making it applicable to vision, language, and hybrid learning tasks.

Multi-domain Task Incremental Learning (MTIL) is a continual-learning regime in which tasks arrive sequentially from multiple distinct data domains, and a model must assimilate each new task while retaining competence on previously seen domains under restricted access to past data. In representative formulations, a task sequence $T_1,\dots,T_t$ is associated with per-task training and test sets, only the current task’s training data are accessible at stage $t$, and the objective is sustained performance over the union of seen test distributions (Wu et al., 15 Jan 2025). Across the literature, however, MTIL is not a single protocol but a family of related settings spanning shared-label domain-incremental learning, disjoint-class task streams, task-aware and task-agnostic inference, and frozen-backbone vision-language adaptation (Zhou et al., 2024, Mandalika, 25 May 2026, Panariello et al., 22 Aug 2025).

1. Problem definitions and protocol variants

A canonical MTIL formulation assumes a sequence of tasks $T_1,\dots,T_t$ drawn from different data domains, with task $T_i$ associated with a training set $D^s_i$ and a test set $D^t_i$, where each $D^s_i\subset\mathcal X\times\mathcal Y$. Domains can shift arbitrarily, and the stream may collect subtasks such as class-increments across multiple domains. At time $t$, the learner sees only $D^s_t$, with all previous data unavailable, and seeks a predictor that performs well on $\cup_{i=1}^t D^t_i$ under a cross-entropy objective (Wu et al., 15 Jan 2025). A closely related parameterization appears in factorized-tensor formulations, where a fixed backbone with shared weights $t$0 is augmented per task by $t$1, so that $t$2 (Garg et al., 2023).

The term also subsumes narrower special cases. In Domain-Incremental Learning (DIL), the label space $t$3 is fixed across all domains, $t$4, no past examples are stored, and the goal is a single classifier that predicts over the shared label set without catastrophic forgetting. This is explicitly mapped as $t$5, with tasks identified with domains and no task identity at inference (Zhou et al., 2024). By contrast, some MTIL works assume disjoint class sets $t$6, requiring both task routing and within-task classification when task identity is absent (Mandalika, 25 May 2026, Wang et al., 24 Jun 2025).

Inference assumptions vary materially across the literature.

Representative setting	Inference assumption	Label structure
MSDEM	performance on $t$7 after seeing only $t$8	subtasks across multiple domains
Duct	no task identity at inference	fixed label space $t$9
ITL-IRU	domain-ID known at test time	each domain is a task
MoDER / semantic segmentation	current domain or task known at inference	domain-specific classes or label spaces
CMAP / AFA / ChordPrompt	no task ID given at inference	disjoint class sets across tasks

This heterogeneity is central to MTIL. A common misconception is to treat all reported results as directly comparable; the published protocols differ in whether label spaces are shared, whether task identity is available at test time, and whether “future-task” zero-shot transfer is part of the objective (Hyder et al., 2022, Panariello et al., 22 Aug 2025, Garg et al., 2021).

2. Architectural families

A dominant architectural pattern is the use of a frozen or largely frozen backbone with small task- or domain-specific augmentations. FTN freezes a backbone $T_1,\dots,T_t$0 and adds low-rank tensor corrections $T_1,\dots,T_t$1 plus task-specific batch-normalization parameters and heads; each convolutional layer is updated by

$T_1,\dots,T_t$2

This preserves the source model while allocating only a small per-task factorization (Garg et al., 2023). A closely related strategy appears in low-rank incremental updates, where each layer weight is expressed as a sum of frozen low-rank factors from prior tasks and a new rank-1 or low-rank increment for the current task (Hyder et al., 2022).

Another long-running line uses reparameterization rather than full-copy expansion. In reparameterized-convolution models, a convolution is decomposed into a fixed filter bank $T_1,\dots,T_t$3 and a small task-specific modulator $T_1,\dots,T_t$4, with $T_1,\dots,T_t$5. The shared $T_1,\dots,T_t$6 filters are frozen, and only the task-specific $T_1,\dots,T_t$7 modulators are trained, which the authors argue eliminates cross-task interference “by construction” (Kanakis et al., 2020). Network-latent tensor factorization instead groups convolutional filters across layers into a higher-order tensor and reconstructs each domain’s weights through a shared Tucker core $T_1,\dots,T_t$8 and domain-specific factor matrices $T_1,\dots,T_t$9, thereby exploiting inter-layer correlations rather than treating layers independently (Bulat et al., 2019).

Dynamic expansion is more explicit in expert-based systems. MSDEM uses multiple frozen Vision-Transformer backbones, fuses their class-token features, and instantiates a new expert module $T_i$0 for each new task, with all previous experts frozen once trained (Wu et al., 15 Jan 2025). DE&E places a Mixture-of-Experts head over a frozen, domain-agnostic feature extractor $T_i$1 and maintains expert networks $T_i$2 together with key vectors $T_i$3 for differentiable soft-KNN gating across text, image, and audio inputs (Wójcik et al., 2023). In semantic segmentation, the analogous decomposition is a Domain-Aware Residual Unit with universally shared $T_i$4 convolutions and per-domain adapters and BatchNorm parameters, so that approximately $T_i$5 of weights are shared and about $T_i$6 are domain-specific (Garg et al., 2021).

Recent vision-language MTIL methods keep CLIP frozen and specialize only prompts, gates, or lightweight adapters. ChordPrompt learns banks of visual and textual prompts and two cross-modal Aligner matrices $T_i$7 and $T_i$8, inserting prompts into both encoders at every layer (Wang et al., 24 Jun 2025). CMAP introduces task-specific prompt pools and per-layer gating parameters on top of a frozen CLIP backbone $T_i$9 with embedding dimension $D^s_i$0 (Mandalika, 25 May 2026). AFA inserts a shared LoRA adapter into the final Transformer FFN of both branches and combines it with a Mixture-of-Experts adapter distributed through Transformer FFNs (Dong et al., 12 May 2025).

3. Routing, attention, and selective transfer

As MTIL shifted from task-aware to task-agnostic and zero-shot-aware protocols, routing became a central design problem. MSDEM addresses routing at two levels. First, its Dynamic Expandable Attention attaches a new multi-head attention block $D^s_i$1 for each task and computes

$D^s_i$2

so the current task can selectively weigh the relevance of multiple pre-trained backbones while earlier attention weights remain frozen (Wu et al., 15 Jan 2025). Second, its Dynamic Graph Weight Router defines a learnable relation matrix $D^s_i$3, uses a Gumbel-Softmax relaxation to produce a soft subset $D^s_i$4 of prior experts, and forms $D^s_i$5, followed by a second attention block over the routed representation (Wu et al., 15 Jan 2025).

DE&E uses an explicitly differentiable gating mechanism rather than task-conditioned attention. For an embedding $D^s_i$6, cosine distances to expert keys $D^s_i$7 are converted into a Gaussian-kernel matrix, followed by Sinkhorn/Bregman projection to solve an entropy-regularized optimal transport problem between a uniform row-marginal and a fixed column marginal. The second column of the transport plan yields gating weights $D^s_i$8, which act like a soft indicator of whether expert $D^s_i$9 is among the $D^t_i$0 nearest keys to $D^t_i$1 (Wójcik et al., 2023). This is combined with cosine similarity scores $D^t_i$2 in the ensemble prediction.

Vision-language MTIL has largely reframed routing as prototype matching. CMAP’s “Text-Space Task Routing” forms one frozen text prototype per task,

$D^t_i$3

and routes by maximizing $D^t_i$4. The method supplements this with “Multi-Prototype Visual-Textual Confidence,” combining $D^t_i$5 K-means visual prototypes per class with text-alignment scores under task-adaptive confidence thresholds, and “Symmetric Cross-Modal Gating,” which applies Hard Gumbel gates to the text encoder conditioned on batch image features (Mandalika, 25 May 2026). ChordPrompt similarly stores normalized domain prototypes

$D^t_i$6

and retrieves the prompt bank of the prototype with maximal cosine similarity, falling back to vanilla zero-shot CLIP if the similarity does not exceed a threshold $D^t_i$7 (Wang et al., 24 Jun 2025).

MoDER extends selective transfer from routing to recomposition. It stores a LoRA-based textual expert $D^t_i$8 for every seen class in a foundational hub, retrieves the top-$D^t_i$9 seen classes most similar to an unseen class in vanilla CLIP text space, and forges an update

$D^s_i\subset\mathcal X\times\mathcal Y$0

before applying an $D^s_i\subset\mathcal X\times\mathcal Y$1-smoothing step $D^s_i\subset\mathcal X\times\mathcal Y$2 to generate a refined prototype for the unseen class (Panariello et al., 22 Aug 2025). This suggests a broader MTIL trend: routing is increasingly used not only to preserve old competence but also to improve forward generalization to unseen domains.

4. Forgetting mitigation and optimization strategies

One major MTIL strategy is strict parameter isolation. In FTN, the original backbone is frozen after source training, previously learned factors remain frozen, and task $D^s_i\subset\mathcal X\times\mathcal Y$3 uses only its own $D^s_i\subset\mathcal X\times\mathcal Y$4, batch-normalization parameters, and head. The paper states that, since there is no update to shared or earlier factors once they are learned, FTN “by construction entirely isolates parameter updates for different tasks, guaranteeing zero interference and thus no forgetting” (Garg et al., 2023). ITL-IRU follows the same logic: frozen low-rank basis directions are never changed, and only the new low-rank update and selector matrices are learned for each incoming task, yielding “zero forgetting” in its reported benchmarks (Hyder et al., 2022). Reparameterized convolutions make the same claim in terms of task interference rather than low-rank updates: fixed shared filters plus frozen old modulators prevent gradient collision among tasks (Kanakis et al., 2020).

A second strategy is controlled expansion with frozen history. MSDEM updates only the new expert’s parameters $D^s_i\subset\mathcal X\times\mathcal Y$5, the new attention heads $D^s_i\subset\mathcal X\times\mathcal Y$6, and the new routing row $D^s_i\subset\mathcal X\times\mathcal Y$7, while previous experts, attention blocks, and routing structure remain frozen (Wu et al., 15 Jan 2025). AFA similarly freezes previously learned routers and decomposes learning into two stages: ABFA trains a fresh router with a Mixture-of-Experts adapter on the current task’s few-shot labels, and AFFA then updates only the shared forward adapter through a symmetric contrastive loss to accumulate task-invariant knowledge across tasks (Dong et al., 12 May 2025).

A third strategy is consolidation or distillation rather than strict isolation. Duct constructs a merged representation

$D^s_i\subset\mathcal X\times\mathcal Y$8

to consolidate domain-specific backbone updates, then retrains the new classifier on the merged embedding space and uses optimal transport to estimate compatible old-domain classifiers, merging them as

$D^s_i\subset\mathcal X\times\mathcal Y$9

No replay or distillation losses are added beyond this explicit representation and classifier consolidation (Zhou et al., 2024). In semantic segmentation, forgetting is controlled through a knowledge-distillation retention term

$t$0

combined with differential learning rates between shared and domain-specific parameters (Garg et al., 2021). In object detection, Attentive Feature Distillation uses bottom-up self-attention and top-down box attention to preserve critical regions of feature maps during task transitions, and in difficult domain-plus-category shifts it is combined with exemplar sampling (Liu et al., 2020).

A fourth strategy uses replay without old labels. Kar et al. formulate continual expansion of an $t$1-task NLP model into an $t$2-task model through knowledge distillation on unlabeled data $t$3 drawn from old-task distributions, optimizing

$t$4

With temperature $t$5 and $t$6, this “UKD” procedure is reported to prevent “up to 20% performance drops on old tasks” and to nearly match multi-task retraining while never revisiting old labels (Kar et al., 2023). MoDER uses a different replay substrate—synthetic CLIP image embeddings generated by a light diffusion model—to train per-class textual experts with a sigmoid-style alignment loss and preserve composability across tasks (Panariello et al., 22 Aug 2025).

5. Benchmarks, metrics, and empirical landscape

Empirical MTIL spans several benchmark families rather than a single evaluation suite. MSDEM evaluates on TinyImageNet, CIFAR-100, CIFAR-10, and Birds, using two- and three-domain sequences and one four-domain chain, with “Average” defined as mean accuracy across all tasks and “Last” as accuracy on the final task (Wu et al., 15 Jan 2025). CMAP, ChordPrompt, AFA, and MoDER all use an 11-dataset vision benchmark—Aircraft, Caltech101, CIFAR-100, DTD, EuroSAT, Flowers102, Food101, MNIST, OxfordPet, StanfordCars, and SUN397—totalling 1,201 classes, but metric conventions are not fully uniform. CMAP defines Transfer as forward-generalization to unseen tasks, Last as final retention on seen tasks, and Average as mean accuracy across all tasks (Mandalika, 25 May 2026). MoDER also reports Transfer, Average, and Last, with task identity known at inference in its MTIL protocol (Panariello et al., 22 Aug 2025). AFA states that, in its MTIL reporting, “Average = (Transfer + Last)/2” (Dong et al., 12 May 2025). Segmentation and detection works instead use mIoU, $t$7, or mAP (Garg et al., 2021, Liu et al., 2020).

Within this heterogeneous landscape, several results are recurrently cited because they illustrate the design trade-offs of MTIL. On the MSDEM benchmark, “MSDEM-2 (two ViT backbones) outperforms all baselines (DER, DER++, DER+++refresh, MoE adapters, prompt-based StarPrompt, RanPac, Dap) by large margins in both Average and Last, often $t$8 over the strongest prompt-replay methods.” The same study reports that MSDEM-2 uses approximately $t$9 M trained parameters versus approximately $D^s_t$0 M in StarPrompt-2nd stage, needs approximately $D^s_t$1 GiB GPU versus approximately $D^s_t$2 GiB, and runs approximately $D^s_t$3 it/s versus approximately $D^s_t$4 it/s (Wu et al., 15 Jan 2025).

FTN illustrates the parameter-efficiency extreme. On ImageNet→Sketch with ResNet-50, fine-tuning per domain uses 141 M convolutional parameters with mean top-1 $D^s_t$5, while FTN with $D^s_t$6 uses 36.02 M parameters with mean top-1 $D^s_t$7. On DomainNet with ResNet-34, fine-tuning per domain uses 127.7 M parameters with mean top-1 $D^s_t$8, while FTN with $D^s_t$9 uses 25.22 M parameters with mean top-1 $\cup_{i=1}^t D^t_i$0. Across all listed benchmarks, FTN is reported to match or closely approach single-task performance while adding only a small fraction of the parameters used by full fine-tuning or prior adapter methods (Garg et al., 2023).

On the 11-dataset CLIP-based MTIL benchmark, several families have progressively improved forward transfer and final retention. ChordPrompt reports, under Order I, Transfer $\cup_{i=1}^t D^t_i$1 versus ZSCL $\cup_{i=1}^t D^t_i$2, Average $\cup_{i=1}^t D^t_i$3 versus $\cup_{i=1}^t D^t_i$4, and Last $\cup_{i=1}^t D^t_i$5 versus $\cup_{i=1}^t D^t_i$6, with approximately $\cup_{i=1}^t D^t_i$7 M trainable parameters (Wang et al., 24 Jun 2025). AFA reports Transfer $\cup_{i=1}^t D^t_i$8, Average $\cup_{i=1}^t D^t_i$9, and Last $t$00, while reducing trainable parameters from $t$01 M in BCL to $t$02 M and GPU memory from $t$03 GiB to $t$04 GiB (Dong et al., 12 May 2025). MoDER reports Transfer $t$05, Average $t$06, and Last $t$07, and explicitly claims improvement over the zero-shot CLIP baseline on Transfer (Panariello et al., 22 Aug 2025). CMAP reports that text-space routing, multi-prototype visual-textual confidence, and symmetric gating improve Order-I performance over IAP, with ablations showing $t$08text routing reduces Transfer by $t$09 percentage points, $t$10MPVTC reduces Average by $t$11 percentage points, and $t$12sym-gating reduces Average by $t$13 percentage points and Last by $t$14 percentage points (Mandalika, 25 May 2026).

Other subdomains reveal complementary empirical patterns. Duct reports on ViT-B/16-IN1K that Office-Home improves from Finetune $t$15, $t$16 to Duct $t$17, $t$18; DomainNet improves from L2P $t$19, $t$20 to Duct $t$21, $t$22; CORe50 improves from CODA-Prompt $t$23, $t$24 to Duct $t$25, $t$26 (Zhou et al., 2024). In semantic segmentation, the proposed domain-incremental method achieves CS $t$27 and BDD $t$28 on CS→BDD versus fine-tuning at CS $t$29 and BDD $t$30, with $t$31 (Garg et al., 2021). In object detection, Attentive Feature Distillation improves KITTI→Kitchen from 10.9 under fine-tuning to 36.6 without exemplars and 68.6 with 100 exemplars (Liu et al., 2020). In NLP, UKD maintains MNLI at approximately $t$32 after a five-task chain, compared with approximately $t$33 under classical TKD (Kar et al., 2023).

6. Conceptual issues, misconceptions, and research directions

The first conceptual issue is that MTIL is not synonymous with a single inference model. Some methods require task or domain identity at test time, including ITL-IRU, the semantic-segmentation setting, and the MTIL protocol adopted by MoDER; others explicitly require no task identity and therefore must solve routing and confidence estimation as part of inference, including Duct, CMAP, ChordPrompt, and AFA (Hyder et al., 2022, Garg et al., 2021, Panariello et al., 22 Aug 2025, Zhou et al., 2024). A plausible implication is that “state of the art” claims should be interpreted within protocol families rather than across all MTIL papers indiscriminately.

The second issue concerns what actually causes forgetting. In object detection, the reported finding is that “domain gaps have smaller negative impact on incremental detection, while category differences are problematic,” especially in the fully multi-domain plus category-incremental scenario (Liu et al., 2020). This contrasts with formulations where the label space is fixed and only the input distribution shifts, for which representation consolidation or per-domain adapters can be highly effective (Zhou et al., 2024, Garg et al., 2021). The literature therefore distinguishes at least two sources of difficulty: representation drift under domain shift, and classifier or routing ambiguity under class-set growth.

The third issue is the role of the frozen foundation model. Many methods rely on a strong pre-trained backbone—ImageNet-pretrained ResNets, ViT-B/16, or CLIP—and treat continual learning primarily as structured parameter allocation around that backbone. The papers themselves note the downside: FTN’s future directions include adaptive per-layer rank selection and shared evaluation among tasks (Garg et al., 2023); Duct notes that merging weaker backbones may degrade and that optimal-transport cost grows with the number of classes and tasks (Zhou et al., 2024); DE&E remarks that the performance upper bound is tied to the quality of the frozen feature extractor and that the Sinkhorn-based gating is approximately $t$34 slower than non-differentiable E&E (Wójcik et al., 2023).

Current research directions largely extend the same themes. CMAP argues for exploiting CLIP’s text embedding space rather than relying exclusively on visual routing (Mandalika, 25 May 2026). ChordPrompt proposes deeper cross-modal prompt exchange and domain-adaptive retrieval (Wang et al., 24 Jun 2025). AFA identifies separate mechanisms for forward forgetting and backward forgetting, and notes scaling to hundreds of domains, dynamic expert pruning, or fully joint optimization of both adapters as future work (Dong et al., 12 May 2025). MoDER reframes continual adaptation as modular expert recomposition for unseen classes and future domains (Panariello et al., 22 Aug 2025). Earlier tensor and low-rank approaches point toward adaptive rank control, compression of the shared core, and hybridization with routing or mask-based methods (Bulat et al., 2019, Hyder et al., 2022). Collectively, these directions suggest that MTIL is evolving from a narrow anti-forgetting problem into a broader study of how fixed foundation models, selective routing, and compact per-task modules can jointly support retention, transfer, and zero-shot generalization across arbitrarily shifting domains.