Instance-Based Transfer Learning

• Instance-based transfer learning is defined as aligning and weighting individual source instances to bridge domain discrepancies.
• It encompasses methods like importance weighting, retrieval-based matching, and joint pooling to optimize target learning.
• The approach enhances performance in vision, NLP, and control tasks by reducing negative transfer through instance-specific adaptations.

Instance-based transfer learning denotes a family of transfer-learning strategies in which the transferable signal is defined at the granularity of individual samples, object instances, patches, sentences, trials, or source exemplars, rather than being reduced to parameter inheritance alone. In one canonical formulation, transfer learning relaxes the standard supervised assumption by allowing a mismatch between source and target domains or tasks, and instance transfer then proceeds by weighting or aligning source samples so that useful source evidence contributes to target learning (Cheplygina et al., 2018). In later work, the same label is also used for jointly training on pooled auxiliary and target instances, retrieving real source neighbors for each target example, reweighting source subjects or trajectories, transferring instance-level patch representations, or fine-tuning instance-segmentation systems pretrained on labeled source instances such as MS COCO (Arnold et al., 2019, Jaikumar et al., 2022). The resulting literature is technically heterogeneous, but its unifying principle is that transfer is mediated by instance-level structure.

1. Definitions and conceptual scope

In the survey literature, a domain is written as $\mathcal{D}=(\mathcal{X},p(\mathbf{x}))$ and a task as $\mathcal{T}=(\mathcal{Y},f(\cdot))$; transfer learning begins when the standard supervised assumption is relaxed so that $\mathcal{D}_S \neq \mathcal{D}_T$ and/or $\mathcal{T}_S \neq \mathcal{T}_T$ (Cheplygina et al., 2018). Within that taxonomy, instance transfer is the case in which source samples are weighted, selected, or aligned so that the effective source distribution becomes closer to the target, often informally expressed as $p(\mathbf{x}_S)\approx p(\mathbf{x}_T)$ (Cheplygina et al., 2018).

A narrower deep-learning variant treats the pretrained source model as an instrument for target-data curation. In that formulation, a source-pretrained network estimates the influence of each target-domain training sample on target validation loss, and target samples whose summed influence is positive are removed before fine-tuning. The influence score is approximated by

$$I_{loss}(x,x_j)= -\nabla_\theta L(f_\theta(x_j))^\top H_\theta^{-1}\nabla_\theta L(f_\theta(x)),$$

so the transfer does not merely initialize parameters; it also optimizes the target dataset itself (Wang et al., 2018).

These definitions are not exhaustive. In multilingual deep retrieval, instance-based transfer learning is defined instead as joint optimization on the union of target-language and auxiliary-language training instances, explicitly contrasted with sequential fine-tuning (Arnold et al., 2019). In instance segmentation and weakly supervised pathology, the same phrase is used for transferring pretrained instance-level visual representations or pretrained instance-segmentation models to new target instances (Jaikumar et al., 2022, Abdulsadig et al., 2024). This suggests that the expression has become an umbrella term for several instance-granular transfer mechanisms rather than a single algorithmic recipe.

2. Principal methodological families

The main methodological families can be organized by what is transferred and how the target learner uses it.

Family	Transfer unit	Representative mechanism
Reweighting and selection	Source instances	Importance weighting, TrAdaBoost, hybrid domain/task weights
Target-set optimization	Target instances	Influence-based removal of harmful target samples
Retrieval and affinity	Source–target pairs	kNN affinity, LSH querying, real-neighbor surrogate fitting
Joint pooling	Combined training instances	Single shared model trained on pooled source and target data
Instance-level representation reuse	Patches or object instances	Pretrained patch encoders, aggregators, or segmentation backbones

In importance-weighted formulations, the source contribution is scaled by a density-ratio-like term such as

$w_{x_j}=\frac{P_T(x_j)}{P_S(x_j)},$

and the weighted empirical objective combines target and source losses through a tradeoff parameter $\alpha$ (Asgarian et al., 2018). A hybrid variant argues that domain similarity alone is insufficient and decomposes the source-instance weight as

$w_x = w_{\text{domain}_x}+w_{\text{task}_x},$

where $w_{\text{domain}_x}$ estimates target-versus-source similarity and $\mathcal{T}=(\mathcal{Y},f(\cdot))$0 measures predictive usefulness for the target task (Asgarian et al., 2018).

Boosting-based regression transfer makes the same concern explicit. ISTRBoost, or S-TrAdaBoost.R2, starts from the observation that a large source set can skew the learner toward the source distribution and induce negative transfer. It therefore performs Euclidean-distance importance sampling relative to the target mean, keeps only the top $\mathcal{T}=(\mathcal{Y},f(\cdot))$1 most relevant source instances with $\mathcal{T}=(\mathcal{Y},f(\cdot))$2, and then uses a boosting schedule that punishes high-error source samples more aggressively while allowing target weights to increase without strict constraints (Gupta et al., 2022). A related safety-critical classification example arises in transient stability assessment: an augmented dataset containing 10,000 same-distribution samples and 5,000 auxiliary samples is combined with TrAdaBoost so that physically informative but distribution-mismatched auxiliary instances can be exploited without being treated as equivalent to target-distribution samples (Meghdadi et al., 2021).

A second family replaces global distribution matching with explicit instance relations. ILA-DA constructs a source–target affinity matrix $\mathcal{T}=(\mathcal{Y},f(\cdot))$3 using kNN pseudo-labeling in feature space, retains only high-confidence target samples, and optimizes a multi-sample contrastive loss that simultaneously increases intra-class compactness and inter-class separation across domains (Sharma et al., 2021). Cross-dataset NLP transfer follows a related logic but uses retrieval rather than contrastive alignment: a target sentence queries a source dataset through Locality Sensitive Hashing, retrieves $\mathcal{T}=(\mathcal{Y},f(\cdot))$4 approximate nearest source neighbors, computes soft attention over the retrieved source embeddings, and trains on the concatenation of target and retrieved-source summaries, optionally regularized by $\mathcal{T}=(\mathcal{Y},f(\cdot))$5 (Chowdhury et al., 2018). In low-resource explanation, ITL-LIME rejects synthetic perturbations entirely, retrieves real source instances from the nearest $\mathcal{T}=(\mathcal{Y},f(\cdot))$6-medoids source cluster, unions them with real target neighbors, and weights the combined neighborhood with a SCARF-style contrastive encoder before fitting the local surrogate (Raza et al., 19 Aug 2025).

A third family treats instance-based transfer as joint pooled learning. In multilingual retrieval, the model is trained once on the union of auxiliary-language and target-language sentence-pair instances, creating a single shared multilingual embedding space rather than a sequence of fine-tuned language-specific models (Arnold et al., 2019). Here the transfer object is the pooled training population itself.

3. Representation transfer, instance discrimination, and multiple-instance learning

A substantial body of work explains transferability through instance-level representation learning rather than classical sample reweighting. The instance-discrimination line studies pretraining objectives in which each image instance must remain distinguishable from all others. Its central empirical conclusion is that the most transferable content is not high-level category semantics but low-level and mid-level visual structure, and that standard supervised pretraining can reduce transferability by enforcing intra-category invariance that is misaligned with detection, segmentation, landmark prediction, and related tasks (Zhao et al., 2020). The same work proposes exemplar-based supervised pretraining, which uses labels to filter negatives but does not explicitly collapse same-class instances together. Relative to MoCo-v2, the reported gains were from 67.5 to 68.9 in ImageNet linear readout, from 48.5 to 48.8 on VOC AP, from 38.7 to 39.4 on COCO box AP, from 34.0 to 34.4 on COCO mask AP, and from 78.6 to 78.8 on Cityscapes mIoU (Zhao et al., 2020).

In computational pathology, instance-based transfer often means transferring patch embeddings or bag-level aggregation behavior. For HER2 scoring from H&E whole-slide images, a frozen AlexNet pretrained on a source dataset is reused as a patch-embedding model inside an attention-based MIL framework. The comparison among IHC HER2 images, PatchCamelyon H&E images, and ImageNet showed that H&E-pretrained embeddings transferred best, with PCAM pretraining yielding an average AUC-ROC of $\mathcal{T}=(\mathcal{Y},f(\cdot))$7 across the four HER2 scores, compared with $\mathcal{T}=(\mathcal{Y},f(\cdot))$8 for IHC and $\mathcal{T}=(\mathcal{Y},f(\cdot))$9 for ImageNet (Abdulsadig et al., 2024). The transferred object is not a slide classifier but a patch encoder whose features are aggregated by MIL attention.

A larger benchmark generalizes this idea from patch encoders to pretrained MIL models themselves. Across 11 MIL architectures and 21 supervised pretraining tasks, pretrained MIL models consistently outperformed random initialization on 19 downstream pathology tasks, and PC-108 pretraining produced an average gain of 3.3 percentage points over training from scratch (Shao et al., 10 Jun 2025). In frozen-feature evaluation, PC-108 improved average performance by $\mathcal{D}_S \neq \mathcal{D}_T$0, and SVCCA plus layer-reset analyses showed that the main transferable component is the learned aggregation strategy, especially attention. Resetting attention caused an average performance drop of $\mathcal{D}_S \neq \mathcal{D}_T$1, whereas resetting all aggregation weights caused a drop of $\mathcal{D}_S \neq \mathcal{D}_T$2 (Shao et al., 10 Jun 2025). This broadens instance-based transfer from “reusing individual instances” to “reusing the learned treatment of instances inside a bag.”

4. Instance segmentation and dense visual transfer

Instance segmentation offers especially clear examples because the transferred knowledge is explicitly tied to object instances. In waste-bottle segmentation, a Mask R-CNN model pretrained on Microsoft COCO is fine-tuned on a custom dataset of 192 bottle images, split into 177 training images and 15 validation images, with pixel-wise polygon annotations stored in VIA-VGG JSON aligned with COCO-style instance segmentation (Jaikumar et al., 2022). The transfer-learning procedure is incremental: first train head layers only for 30 epochs with frozen remaining layers; then fine-tune selected deeper layers or all layers for another 30 epochs, with some variants extended to 100 epochs and optional additional fine-tuning. Images are resized to $\mathcal{D}_S \neq \mathcal{D}_T$3 with zero-padding, video is processed frame-by-frame with OpenCV, ResNet-101 performed better than ResNet-50, SGD performed better than Adam, and the best model M11 reached validation AP$\mathcal{D}_S \neq \mathcal{D}_T$4, AP$\mathcal{D}_S \neq \mathcal{D}_T$5, AP$\mathcal{D}_S \neq \mathcal{D}_T$6, and COCO-style mAP $\mathcal{D}_S \neq \mathcal{D}_T$7, reported in the abstract and conclusion as about 59.4 mAP (Jaikumar et al., 2022).

Robotic self-supervision shows a different transfer path. SelfDeepMask begins with a COCO-pretrained DeepMask network, where COCO provides 886K labeled object instances, then adapts it to a tabletop domain through physical interaction rather than manual labeling (Eitel et al., 2020). A robot records an RGB frame before a push, executes a push action represented by its start pixel, records the post-push frame, estimates optical flow with FlowNet2, zeroes table-plane motion, clusters the filtered flow with normalized graph cuts, rejects rotating segments whose flow-magnitude standard deviation exceeds 15.0, discards interactions whose mean optical flow magnitude exceeds 7.0, and selects the segment containing the push pixel as the moved-object mask (Eitel et al., 2020). Fine-tuning uses the self-labeled dataset ROBOTPUSH, with 2,300 training scenes, 50 validation scenes, and 190 test scenes containing 16 novel objects; the network is trained for 10 epochs with learning rate 0.001, SGD momentum 0.9, and weight decay 0.0005. The abstract reports that SelfDeepMask outperforms COCO-trained DeepMask by 9.5% in average precision, and the strongest noisy-label variant, using co-teaching with 2.3k interactions, achieved [email protected] $\mathcal{D}_S \neq \mathcal{D}_T$8, [email protected] $\mathcal{D}_S \neq \mathcal{D}_T$9, and [email protected]:0.95 $\mathcal{T}_S \neq \mathcal{T}_T$0 (Eitel et al., 2020).

These systems transfer segmentation capability rather than category semantics. In both cases, the source model contributes generic instance-localization and mask-prediction priors, while the target domain supplies new object instances, scene statistics, and annotation or pseudo-annotation structure.

5. Structured, multilingual, and control-oriented applications

In multilingual deep retrieval, instance-based transfer is realized by large-scale pooled training rather than by explicit sample weighting. The experiments use about 720 million next-sentence pairs across the top 35 Wikipedia languages for next-sentence prediction and about 60 million current-context pairs for inverse cloze, with performance measured by sampled recall@k against 100,000 distractors (Arnold et al., 2019). The central result is that positive transfer occurs for all 35 target languages on both tasks, with no observed negative transfer; the average relative gain is about 96% on inverse cloze versus 31% on next-sentence prediction, and some improvements reach up to 200% (Arnold et al., 2019). The paper also shows that direct vocabulary overlap is only a partial explanation: Telugu improves when Turkish is added after direct Telugu–Turkish overlap has been censored, because transfer proceeds transitively through English, and Catalan similarly benefits from censored Ukrainian through English and Russian pivots (Arnold et al., 2019).

In power-system transient stability assessment, instance-based transfer takes the form of physics-informed auxiliary sampling plus reweighting. The method builds a same-distribution dataset by varying all loads between 60% and 140% of nominal values, an auxiliary dataset by varying only loads in the Fault-Affected Area between 40% and 160%, and then combines 10,000 same-distribution samples with 5,000 auxiliary samples into an augmented training set (Meghdadi et al., 2021). TrAdaBoost is used because the auxiliary distribution differs from the test distribution, and the key operational goal is to reduce unsafe false predictions. For contingency line 21–22, TrAdaBoost with the augmented dataset achieved FP 1.55%, Recall 92.74%, Precision 96.13%, Specificity 97.35%, and Accuracy 95.44% (Meghdadi et al., 2021). Here the transferable object is the reweighted auxiliary instance set selected through domain knowledge of disturbance propagation.

Personalized autonomous driving supplies a control-theoretic variant. PP-TIL first pretrains on large-scale expert data, then fine-tunes on mixed expert and user batches rather than on user data alone, while an inverse-reinforcement-learning regularizer matches user style features such as lateral acceleration, longitudinal jerk, efficiency, road offset, and safety (Lin et al., 2024). The objective is

$\mathcal{T}_S \neq \mathcal{T}_T$1

and fine-tuning explicitly classifies trajectory instances into stationary, straight, and turn categories, excluding stationary trajectories because they provide little style information (Lin et al., 2024). The reported experiments used 50% expert plus 50% user data with $\mathcal{T}_S \neq \mathcal{T}_T$2, while a ratio around 75% expert was favorable in some settings, and the strongest results arose when the neural planner was combined with a differentiable nonlinear optimizer based on Theseus as a safety protection layer (Lin et al., 2024).

Cross-subject SSVEP recognition uses subject instances as the transfer unit. iTRCA extracts a subject-general feature by aligning each source subject’s task-related component with the target subject in a common latent space, and a subject-specific feature by preserving the target subject’s own TRCA structure; SS-iTRCA further performs similarity-based subject selection to avoid harmful source subjects (Wang et al., 12 Jun 2025). Across varying signal lengths, channel counts, and target calibration blocks, iTRCA improved over TRCA by 4.65%, 6.07%, and 7.35% on average, while SS-iTRCA improved by 5.87%, 7.18%, and 8.17%; at $\mathcal{T}_S \neq \mathcal{T}_T$3, SS-iTRCA gained 11.32% on Benchmark, 9.08% on BETA, and 21.19% on the self-collected dataset (Wang et al., 12 Jun 2025).

6. Empirical regularities, boundary cases, and unresolved issues

Several regularities recur across the literature. First, most instance-based methods are motivated by target-data scarcity: small bottle datasets, weakly supervised pathology slides, sparse user driving demonstrations, low-resource target populations in healthcare tabular data, and limited target-subject calibration data in BCIs all motivate transferring information from richer auxiliary sources (Jaikumar et al., 2022, Abdulsadig et al., 2024, Lin et al., 2024, Raza et al., 19 Aug 2025). Second, negative transfer is a central design concern. Source oversupply can skew boosting-based regression transfer, similarity-aware subject selection is introduced specifically to suppress harmful source subjects, and ITL-LIME explicitly assumes a related source domain because an excessively dissimilar source may degrade explanations rather than improve them (Gupta et al., 2022, Wang et al., 12 Jun 2025, Raza et al., 19 Aug 2025).

What determines transferability remains more nuanced than simple semantic relatedness. In H&E HER2 scoring, H&E-pretrained PCAM embeddings outperformed IHC HER2 pretraining despite IHC being biomarker-related, indicating that stain and morphology alignment can matter more than task-semantic proximity (Abdulsadig et al., 2024). The instance-discrimination analysis is consistent with this pattern: low-level and mid-level structure transfer better than high-level semantics, and synthetic datasets with mismatched low-level statistics reduce transfer more than semantically different but visually similar real-image datasets (Zhao et al., 2020). Yet multilingual retrieval shows the opposite limit of purely local similarity explanations: direct vocabulary overlap correlates with gains, but transfer can still arise through transitive overlap paths in the multilingual token graph (Arnold et al., 2019). A plausible implication is that useful instance-level correspondences may be direct, indirect, or representation-mediated, depending on the modality.

The term itself also has unstable boundaries. A denoising system built around adaptive instance normalization for synthetic-to-real noise transfer explicitly characterizes itself as hybrid and primarily feature-based rather than classical instance-based transfer learning, even though it uses per-instance normalization statistics (Kim et al., 2020). In the works considered here, “instance-based transfer learning” therefore spans reweighting, selection, retrieval, pooled training, instance-level representation reuse, and learned treatment of instances inside bags. This terminological breadth is not merely semantic. It marks a substantive split between methods that transfer data points, methods that transfer relations among data points, and methods that transfer encoders or aggregators whose internal organization is itself defined over instances.