Iterative Transfer Learning

Updated 3 June 2026

Iterative Transfer Learning is a process that incrementally adapts models across tasks through cycles of self-training, feature alignment, and pseudo-label updates.
It is applied in diverse areas such as domain adaptation, text attribute transfer, reinforcement learning, and ensemble methods to address distribution gaps.
Empirical evidence shows that iterative methods significantly improve performance metrics like reduced MSE, higher BLEU scores, and increased RL rewards.

Iterative transfer learning refers to a class of algorithms and procedures that leverage sequential, stepwise adaptation or transfer of models or knowledge between related tasks or domains, typically with iterative refinement over multiple cycles. Unlike one-shot transfer paradigms that initialize from a pretrained source and adapt once to a target, iterative transfer learning updates models or their pseudo-labels, feature mappings, or supporting ensembles in successive rounds, gradually bridging gaps between source and target distributions, domains, or tasks via repeated self-training, alignment, or adaptation steps. This approach is prominent in scenarios such as semi-supervised domain adaptation, sequential multi-task settings, small-data regression, cross-domain reinforcement learning, iterative knowledge distillation, and online or streaming data contexts.

1. Formal Definitions and General Principles

Iterative transfer learning is characterized by the cyclical transfer and updating of models, parameters, or pseudo-labeled data from source to target, often using an explicit iterative protocol. Let $\mathcal{S},\mathcal{T}$ denote source and target domains or tasks, $\mathcal{D}_S$ and $\mathcal{D}_T$ their datasets, and $f_\theta$ a model with parameters $\theta$ . Iterative transfer learning proceeds by alternate rounds:

Model transfer: $\theta_T^{(k)} \leftarrow \operatorname{Update}(\theta_T^{(k-1)}, \mathcal{D}_T, \theta_S^{(k-1)}, \ldots)$
Pseudo-label/corpus update: Creating or refining pseudo-parallel data or labels based on current model predictions.
Feature or representation adaptation: Iteratively aligning feature distributions via additional objectives or domain adaptation techniques.

This process may include iterative training, pseudo-label selection (and filtering), domain or task-specific feature alignment, or ensemble re-weighting, with stopping criteria such as validation error convergence, stabilization of label sets, or a fixed iteration budget.

2. Representative Algorithmic Realizations

Several instantiations of iterative transfer learning have been empirically studied across domains:

a. Iterative Self-transfer Learning for Regression

The Deep Adaptation Network with Three branches for Regression (DAN-TR) exemplifies an iterative pseudo-label self-training mechanism for small data regression. At each iteration $i$ , a model $f^{(i-1)}$ is used for pseudo-labeling a large unlabeled dataset. New pseudo-labeled samples $\{(x_k^u,\hat{y}_k)\}$ are combined with labeled data for re-training DAN-TR, which incorporates shared LSTM layers, source/target branches, and a multi-kernel MMD loss for hidden activation alignment. This process iterates until validation MSE no longer improves, typically for 2–3 cycles. Empirically, this strategy achieves order-of-magnitude reductions in test MSE over direct training when labeled sets are extremely small (Xu et al., 2023).

b. Iterative Matching and Translation (IMaT) for Text Attribute Transfer

Given two corpora with contrasting attributes and no parallel data, IMaT constructs an initial pseudo-parallel corpus by semantic matching using sentence embeddings. A sequence-to-sequence model is then trained and the produced translations are iteratively matched, refined, and selected to minimize content shift (measured by Word Mover’s Distance), further improving both content preservation and attribute accuracy with each cycle. Iteration confers "denoising" and EM-style refinement effects, with BLEU scores and human evaluations improving strictly over 3–5 cycles (Jin et al., 2019).

c. Iterative Ensemble for Short-Answer Grading

An ensemble of a text classifier and a numeric similarity-based classifier is constructed. CCA projects features into a shared latent space. The numeric classifier, transferred from a labeled source question to a target, seeds pseudo-labels for the target question; the text classifier is then bootstrapped on these pseudo-labels, and both are refined iteratively, with ensemble weights adaptively reweighted based on error reductions. The process stops when no new pseudo-labels are added or performance plateaus, typically after 5–6 rounds. This yields strong cross-task generalization even in the absence of direct labeled data (Roy et al., 2016).

d. Iterative Policy Transfer for Reinforcement Learning (IDAPT)

IDAPT alternates between (i) learning visual and dynamical domain alignment functions (grounding the source simulator visually via CycleGANs and dynamically with adversarial feature imitation), and (ii) retraining a policy in the newly "grounded" environment. Each cycle uses online, task-specific data rollouts to refine the alignment networks $G$ (visual) and $\mathcal{D}_S$ 0 (dynamics). Empirically, this approach achieves strict monotonic increases in target environment reward over 1–5 iterative cycles, with final performance matching or exceeding direct domain randomization or adaptive RL baselines, especially when both visual and dynamics gaps are large (Zhang et al., 2021).

e. Iterative Transfer for Safety Validation in RL

In a sequence of related MDPs, Q-functions from solved source tasks are transferred to a new task using a "combiner" with learned attention weights and task-specific state/action-value transformations (A2T+SAVT). The transfer and adaptation of prior Q-functions is performed iteratively across task sequences, and empirical results show improvements in jumpstart performance, sample efficiency, and final performance in sequential gridworld and autonomous driving adversarial validation tasks (Corso et al., 2020).

f. Online Iterative Bagging for Anytime Transfer Learning

In the streaming setting, OTBag creates an ensemble of online weak learners, each iteratively updated via Poisson(1) sampling as new data arrives from the blended source and target streams. The ensemble’s performance on target instances is monitored, and two iterative pruning schemes (SDMV, JDSMV) filter or recombine only weak learners beneficial to the target, mitigating negative transfer throughout the online sequence (Chi et al., 2019).

3. Mathematical Formulation and Analysis

The iterative transfer learning paradigm leverages explicit iterates indexing model, data, or ensemble states; convergence and performance are determined by stopping criteria governed by model selection (e.g., validation error, WMD), ensemble accuracy improvement, or reaching statistical plateau. General forms include:

Self-training loss: $\mathcal{D}_S$ 1 with $\mathcal{D}_S$ 2 scheduled per iteration.
Pseudo-label or pseudo-pair update: Accept a new pseudo-label/pair if and only if a content-alignment metric (e.g., WMD, MSE) is improved.
Iterative Q-function mixture: $\mathcal{D}_S$ 3, with $\mathcal{D}_S$ 4 and $\mathcal{D}_S$ 5 adapted per task (Corso et al., 2020).

Typical theoretical analysis employs monotonicity (validation error, MMD, or WMD decrease per iteration), boundedness of self-training error, or regret/minimax learning curves for adversarial or online variants.

4. Empirical Findings and Application Domains

Empirical studies repeatedly demonstrate that iterative transfer learning outperforms non-iterative transfer or direct training, particularly when labeled data is extremely scarce, domain/task gaps are large, or the target exhibits distribution shift. Results include:

Significant reductions in MSE for regression with minimal labels (e.g., 70–85% lower than direct training) (Xu et al., 2023).
Strictly increasing BLEU/content retention and attribute accuracy in unsupervised text transfer (Jin et al., 2019).
Improved initial and final policy performance, and reduced sample complexity in RL transfer (Zhang et al., 2021, Corso et al., 2020).
Robust anytime performance and negative transfer mitigation in online, streaming domain adaptation (Chi et al., 2019).
Enhanced attack success rates in side-channel cryptanalysis from scarce data, with average and worst-case sample complexity halved by iterative byte-wise transfer (Lou et al., 2024).
Effective transfer of trajectory-tracking policies across heterogeneous robots and sim-to-real in adaptive control with negligible drop in precision (Pereida et al., 2018).

5. Architectures and Implementation Strategies

Practical instantiations span a variety of architectural designs:

Deep networks with branch architectures (e.g., DAN-TR: shared LSTM plus per-domain heads with multi-kernel MMD) (Xu et al., 2023).
Sequence-to-sequence models with iterative pseudo-parallel corpus updates (Jin et al., 2019).
Classifier ensembles with iterative bootstrapping on pseudo-labels, reweighted as performance improves (Roy et al., 2016).
Iterative CycleGAN-based visual mappings and adversarial dynamics mapping for policy transfer (Zhang et al., 2021).
Attention-weighted mixtures of prior Q-functions in RL setting, with learned state/action transforms (Corso et al., 2020).
Online ensembles with Poisson bagging and target-adaptive weak learner selection, suitable for streaming scenarios (Chi et al., 2019).
Deep MLPs or CNNs updated iteratively across highly correlated byte-wise tasks in side-channel analysis (Lou et al., 2024).

Guidance from empirical work emphasizes cycle counts (typically 2–5 sufficing), batch sizes and reliability criteria for pseudo-labeling, explicit alignment objectives (MMD, adversarial, content preservation), and adaptive loss weighting.

6. Quantitative Benchmarks and Performance Tables

The following table compares key empirical results for selected representative methods:

Domain/Task	Iterative Method	Main Metric	Baseline	Iterative Result	Reference
Small-data regression	DAN-TR self-transfer	Test MSE (D20)	9.34e-3	2.78e-3	(Xu et al., 2023)
Text attribute transfer	IMaT	BLEU (Formality)	21.3 (Del+Ret)	38.2	(Jin et al., 2019)
RL domain adaptation	IDAPT (N=5)	Final reward (Walker)	~5–30%	75% (i=5)	(Zhang et al., 2021)
RL safety validation	A2T+SAVT transfer	Final improvement	–	+60% (GW), +5× (AD)	(Corso et al., 2020)
AES side-channel attack	MLP/CNN+ITL	Avg. MTD (thermal)	54.7 (MLP)	53.8	(Lou et al., 2024)
Online transfer (sentiment)	OTBag-JDSMV	Accuracy (%)	69 (best offl.)	75.7	(Chi et al., 2019)

The values in this table are directly cited from the referenced works.

7. Practical Considerations, Variants, and Limitations

Iterative transfer learning can be flexibly adapted to incorporate contemporary techniques such as mean-teacher consistency regularization, adversarial domain alignment, and iterative meta-learning. It is particularly suitable for:

Small labeled target data situations, with abundant unlabeled or related-task data.
Settings with sequentially emerging tasks or domains (e.g., safety validation, continual RL, cryptanalysis).
Heterogeneous or multi-modal data domains, where strict one-shot transfer is insufficient.

Conversely, performance may plateau if correlation between tasks is weak, source models induce negative transfer that cannot be detected, or iteration introduces propagation of errors in pseudo-labels or alignments. Filtering techniques (e.g., SDMV/JDSMV, attention weights, content-preservation thresholds) are essential for robust convergence.

A plausible implication is that, as models become increasingly compositional and as resource constraints for labeled supervision persist, iterative transfer learning will remain a foundation for small-data modeling, cross-domain adaptation, and efficient life-long learning across ML, control, and security tasks.