Parameter Transferring Strategy

• Parameter transferring strategy is a set of methods for reusing and adapting pre-trained model weights to new tasks, enhancing efficiency and generalization.
• It employs adaptor modules, continuous interpolation, and symmetry alignment to mitigate negative transfer and optimize performance.
• Modern strategies balance resource efficiency and robustness by leveraging shared parameters, fine-grained gating, and architectural innovations.

Parameter transferring strategy refers to the set of algorithmic and architectural techniques that enable the transfer, reuse, or adaptation of parameters between models, tasks, or domains. This paradigm is central to efficient transfer learning, multi-task adaptation, and domain adaptation, as it leverages pre-trained weights, submodules, or control parameters from upstream tasks or models to improve efficiency, generalization, or sample efficiency on downstream targets. Modern parameter transferring strategies incorporate architectural innovations, continuous interpolation methods, symmetry alignments, and explicit control over the sharing or adaptation process.

1. Foundations and Motivation

Parameter transfer is motivated by the observation that representations or parameterizations acquired on large-scale data or related tasks often capture reusable semantic structure. Traditional approaches fine-tune all or some layers on the target domain; however, advances in scalable learning and multi-task optimization have led to more nuanced, parameter-efficient, and theoretically grounded strategies. The key goals are (i) rapid adaptation to new tasks, (ii) resource/memory efficiency by minimizing redundant re-training, and (iii) robustness across heterogeneous domains or architectures (Zhao et al., 17 Apr 2025, Zhang et al., 2018, Zhan et al., 19 Nov 2025, Yuan et al., 26 Sep 2025).

The breadth of parameter transfer encompasses fully shared (hard sharing), selectively adapted (partial or soft sharing), continuously interpolated, and aligned transfer (to respect architectural symmetries or task differences). Successful strategies must balance transferability and task discrimination, mitigate negative transfer, and be computationally scalable.

2. Adaptor Modules and Parameter-Efficient Fine-Tuning

Contemporary parameter transfer strategies in large-scale deep models frequently employ lightweight adaptor modules. For instance, the asymmetric adaptor framework for transferring learned image compression (LIC) models from human to multi-task machine perception (Zhao et al., 17 Apr 2025) introduces:

• Shared Adaptors inserted at encoder/decoder stages to capture task-agnostic semantic priors via dual spatial (DWConv) and frequency (FFT/IFFT) branches.
• Task-Specific Adaptors appended in parallel after decoder stages to inject task-level distinctions.
• Multi-Scale Gated Fusion: Outputs from all stages are fused and added residually to the main reconstruction.

This scheme minimizes overhead (≈2.1% of parameters versus full fine-tuning), maintains a single bitstream, and shows superior performance over prior prompt-tuning and single-task PEFT baselines in multi-vision tasks. The adaptors can be trained jointly using an objective combining rate-distortion and task-specific losses: $$L_\text{total} = L_\text{rd} + \sum_{t=1}^T \gamma_t \mathcal{L}_t(X, Y_t)$$ where $$L_\text{rd} = \mathbb{E}_X[r(\hat{y}) + \lambda D(X, \hat{X})]$$ (Zhao et al., 17 Apr 2025).

Adaptor-based strategies retain the generality of a robust, frozen backbone while facilitating parameter-efficient, multi-task adaptation and modular scalability.

3. Parameter Transfer Units and Fine-Grained Interpolation

Rather than discrete selection between freezing and fine-tuning, advanced parameter transfer modules such as the Parameter Transfer Unit (PTU) (Zhang et al., 2018) enable continuous, learnable combinations of source and target activations at each layer: $$\begin{aligned} & r_\ell = \sigma(W^r_\ell [h^{(s)}_\ell; h^{(t)}_\ell]) \ & z_\ell = \sigma(W^z_\ell [h^{(s)}_\ell; h^{(t)}_\ell]) \ & h_\ell^f = (1 - r_\ell) * h_\ell^{(s)} + r_\ell * \varphi(W^h_\ell h_\ell^{(s)}) \ & \tilde h_\ell^{(t)} = (1 - z_\ell) * h_\ell^{(t)} + z_\ell * h_\ell^f \end{aligned}$$ Here, the gates $r_\ell$ and $z_\ell$ are trained to control nonlinear adaptation and the mix of source/target features. PTU thus learns to interpolate between parameter sharing, fine-tuning, or random initialization on a per-layer, per-feature basis. Empirical results show that PTU outperforms manual fine-tuning across CNNs and RNNs, and automatically adapts to network depth and transferability (Zhang et al., 2018).

Key advantages of this approach include:

• Data-driven, feature-level adaptation rather than heuristic per-layer tuning.
• Preservation of well-optimized upstream representations while preventing catastrophic forgetting.
• Flexibility in heterogenous or low-resource transfer scenarios.

4. Alignment, Projection, and Symmetry-Respecting Strategies

Parameter space alignment is crucial when models possess intrinsic symmetries or when naively combining weights can result in negative interference. For LLMs, parameter transfer via "task arithmetic" is improved by first aligning in parameter space via permutation, rotation, and scaling of blocks (especially for Grouped-Query Attention and SwiGLU layers) (Horoi et al., 13 Nov 2025):

1. Permutation alignment: Hungarian assignment for simultaneous row rearrangement of FFN blocks.
2. Rotation alignment: Orthogonal Procrustes solution to jointly align Q/K and V/O pairs within GQA.
3. Scaling alignment: Analytical scaling to equilibrate norm mismatches between aligned subspaces.

This three-stage symmetry alignment eliminates negative transfer inherent to raw parameter addition. Both weight-based and activation-based alignment variants exist, the former directly matching weights and the latter matching empirical activations on a prompt set. This has enabled the effective transfer of reasoning skills across independently tuned LLMs, outperforming naive vector arithmetic (Horoi et al., 13 Nov 2025).

In ELMs, a projective parameter transfer strategy directly enforces a linear mapping between source and target domain parameters via a learned projection matrix $M$: $\beta_\text{target} = M \beta_\text{source}$, optimized jointly with structured sparsity ($\ell_{2,1}$) penalties (Chen et al., 2018).

5. Parameter Transfer in Structured, Multi-Task, and Sequential Scenarios

Across reinforcement learning, quantum optimization, and structured prediction, diverse parameter transfer strategies are tailored to the specifics of problem structure:

• Parameter compositionality: Policy parameters for each task are expressed as a combination of shared basis vectors and learned task-specific composition weights, as in the TaCo/PaCo frameworks for multi-task RL (Sun et al., 2023). Transfer to new tasks reduces to optimizing new composition weights over a frozen shared basis, enabling rapid adaptation and high sample efficiency.
• Interpolating structures for domain shift: In population-based structural health monitoring, large domain shifts are bridged by continuous parameter morphing between source and target through a chain of intermediate structures, each realized by interpolating geometry/material parameters and propagating statistical adaptations hop-by-hop (Dardeno et al., 23 Mar 2026). This decomposes transfer into locally aligned steps and enables positive transfer even between disparate modalities.
• Adaptive parameter transfer in quantum and control settings: For multi-parameter quantum metrology, an adaptive control strategy recursively transfers parameter estimates from one iteration to the next to achieve near-Heisenberg scaling in estimator variance, via reparameterization and feedback-driven control Hamiltonians (Wei et al., 16 Oct 2025).

Setting	Transfer Mechanism	Key Feature
Multi-task RL	Parameter compositionality (Φw)	Subspace sharing
Quantum optimization	Warm-start, Taylor step, clustering, DL-based	Classical estimation
Structural HM	Interpolating across intermediate morphologies	Stepwise domain hops
Quantum metrology	Control parameter transfer via feedback	Adaptive precision loop

6. Theory and Empirical Conditions of Successful Transfer

Theoretical analysis in parameter transfer now quantifies conditions where partial parameter reuse amplifies or degrades downstream performance. For two-layer ReLU CNNs, transfer is beneficial when the inherited parameters encode strong "universal" features and the upstream domain has sufficient data and signal-to-noise, parameterized by

$$\Gamma = \frac{\alpha^2 N_1 \|\mathbf{u}\|_2^4}{\sigma_{p,1}^2 \sigma_{p,2}^2 d}$$

where $\alpha$ is the transfer fraction, $$L_\text{rd} = \mathbb{E}_X[r(\hat{y}) + \lambda D(X, \hat{X})]$$0 upstream samples, $$L_\text{rd} = \mathbb{E}_X[r(\hat{y}) + \lambda D(X, \hat{X})]$$1 universal feature strength, and $$L_\text{rd} = \mathbb{E}_X[r(\hat{y}) + \lambda D(X, \hat{X})]$$2 input dimension (Yuan et al., 26 Sep 2025). Negative transfer arises when upstream universal features are weak relative to target task-specific noise, or the signal does not generalize.

Empirical studies and ablations across domains show:

• Performance boosts scale with upstream data size and feature universality.
• Structured sparsity and careful adapter/task scaling improve cross-task synergy.
• Overhead from parameter-efficient strategies (0–3% parameter cost) enables strong efficiency gains without accuracy penalty.

7. Open Challenges and Best Practices

Despite the advances, several limitations and challenges persist:

• Negative transfer remains a risk, particularly when domains are not sufficiently aligned, universal representations are absent, or the strategy disregards parameter symmetries.
• Architectural and scaling flexibility: Not all strategies generalize to growing parameter counts, varying architectural motifs (e.g., MoE, advanced attention), or high-dimensional control codes.
• Automated selection and adaptation: Although several methods incorporate adaptive gating, RL-based routing, or joint optimization, automatic determination of transfer fractions, chain lengths, and representational subspaces is an open area (Zhan et al., 19 Nov 2025, Dardeno et al., 23 Mar 2026).
• Implementation efficiency: Memory and compute constraints are important, especially in generator-based approaches (e.g., weight distillation) and multi-task adaptation.
• Quantitative tradeoffs: There are tradeoffs between transferability, invariance to domain or architecture, efficiency, and achievable accuracy.

Best practices involve:

• Freezing large pretrained backbones and using lightweight, task-specific adapters for downstream transfer (Zhao et al., 17 Apr 2025).
• Employing data-driven gating/interpolation (e.g., PTU) instead of layerwise heuristics (Zhang et al., 2018).
• Aligning parameter symmetries explicitly prior to weight merging or skill vector transfer in LLMs (Horoi et al., 13 Nov 2025).
• Adopting geometry-aware or compositional frameworks when transferring across disparate domains, tasks, or structural spaces (Dardeno et al., 23 Mar 2026, Sun et al., 2023).

Parameter transferring strategy thus constitutes a rigorous, theoretically and empirically validated body of techniques that is central to modern efficient transfer learning and multi-domain adaptation.