Surrogate and Transfer-based Methods

• Surrogate and transfer-based methods are algorithmic frameworks that use data-driven approximations and knowledge transfer to efficiently solve computationally expensive problems.
• They combine techniques like polynomial chaos expansions, Gaussian processes, and neural surrogates to emulate high-fidelity simulations and adapt to domain shifts.
• These methods find practical applications in engineering, multiphysics simulation, optimization, and adversarial machine learning, achieving significant speedup and accuracy.

Surrogate and Transfer-based Methods

Surrogate and transfer-based methods span a wide class of algorithmic frameworks in computational science, engineering design, optimization, and machine learning that leverage data-driven, model-reduction, or knowledge-transfer techniques to enable efficient solution of expensive or multiphysics problems. These approaches are central wherever the direct solution of a target problem—such as high-fidelity simulation, optimization, or black-box decision-making—is computationally prohibitive, but there exist sources of auxiliary knowledge: cheap approximations, legacy simulations, lower-fidelity models, or data from related domains. This article surveys the mathematical foundations, methodological variants, key application domains, accuracy and performance metrics, and the range of architectures and algorithms developed for surrogate modeling and transfer learning, with particular emphasis on recent advances in partitioned multiphysics algorithms, multifidelity machine learning, transfer optimization, and adversarial inference.

1. Foundational Principles and Problem Settings

Surrogate Modeling. Surrogate models are data-driven or reduced-order approximations of expensive computer simulators or physical systems. Classical surrogates comprise polynomial chaos expansions (PCE), kernel-based interpolators (Kriging/GP), radial basis functions, and machine-learned regression or neural networks. Their role is to provide rapid emulation—potentially with uncertainty quantification—of mappings $y = f(x)$ when direct queries to $f$ are expensive.

Transfer-based Methods. Transfer learning in the surrogate context refers to leveraging knowledge from related source tasks, lower fidelity models, or previously solved problems to accelerate, regularize, or increase the data efficiency of surrogate construction or deployment in a new target task (Wang et al., 2024, Pan et al., 30 Jan 2025, Pan et al., 23 Jan 2025). Scenarios include domain shifts, multi-fidelity simulation landscapes, and parametric model changes.

Partitioned and Coupled Problems. Many multiphysics and interface problems (e.g., fluid-structure, advection-diffusion, subsurface flow, earthquake engineering) are handled by partitioned methods, in which subdomains or subproblems exchange information (states or fluxes) via interface coupling. Surrogate- and transfer-based techniques enable accurate, low-intrusive, and efficient interface data transfer, closing the performance and modularity gaps between monolithic and minimal-remap approaches (Bochev et al., 2024).

Expensive Optimization and Black-Box Inference. Surrogate- and transfer-augmented evolutionary algorithms (e.g., SAEA, BO) address expensive optimization problems by replacing direct objective evaluations with surrogate predictions and leveraging competitive transfer from source task solutions (Xue et al., 2024, Wang et al., 2021, Tfaily et al., 30 Mar 2026). Similarly, adversarial attacks on neural networks exploit transferability from surrogate to target models for efficient black-box generation of adversarial examples (Yang et al., 19 May 2025, Qin et al., 2021, Lord et al., 2022, Miller et al., 2020).

2. Mathematical and Algorithmic Frameworks

Dynamic Flux Surrogate Construction. In partitioned PDEs with non-overlapping subdomains and interface conditions, Dynamic Mode Decomposition (DMD) is used to construct efficient, low-rank surrogates for interface fluxes. Explicitly, snapshot pairs $\{\mathbf{y}_k\}, \{\mathbf{y}_{k+1}\}$ form matrices from which a best-fit linear map $A$ is built via truncated SVD, yielding interface predictions as $\bm\lambda_k = A_\lambda \mathbf{y}_{k-1}$ (Bochev et al., 2024).

Transfer via Domain Warping and Affine Transformation. Transfer learning for surrogates extends from simple affine transformations (rotation, translation) between input domains (Pan et al., 23 Jan 2025) to more general warping using monotonic diffeomorphisms (e.g., Beta CDF per-dimension) (Pan et al., 30 Jan 2025). The optimal parameters (warps, rotations, output scaling/shifting) are identified by minimizing empirical loss given limited target data, employing Riemannian optimization for differentiable models or CMA-ES for non-differentiable ones.

Probabilistic Transfer in Surrogate Regression. Probabilistic frameworks employ Bayesian updating with tempering to blend prior knowledge from source-tasks' posterior over surrogate coefficients into the target task, with the tempering level (interpolating between no and full transfer) set by maximizing expected data-fit (Bridgman et al., 2023). Surrogate modes include PCE with tempered-source prior for the coefficients.

Co-surrogate Knowledge Transfer in Multiobjective Optimization. Combined surrogate models exploit functional dependencies between fast- and slow-evaluation objectives using Gaussian Processes. A "co-surrogate" GP models the difference $f_\text{slow}(x) - f_\text{fast}(x)$, enabling synthetic label generation for the slow objective, subject to uncertainty-based filtering before augmenting the slow-objective surrogate (Wang et al., 2021).

Local Transfer Learning in GP Surrogates. The LOL-GP framework models the transfer strength from multiple source systems via a latent GP-regularized function $\omega_l(x)$, yielding spatially-local transfer coefficients $\rho_l(x)=\max\{0, \omega_l(x)\}$. The target surrogate aggregates sources and a discrepancy GP, and Gibbs sampling is used for inference, preventing negative transfer in mismatched subregions (Wang et al., 2024).

Transfer in Deep Surrogates and Multifidelity Architectures. Neural surrogates in engineering and UQ are trained on a large corpus of low-fidelity data before selectively fine-tuning on scarce high-fidelity or target data, often by freezing all but the output or final layer(s) (Pai et al., 2022, Ishikawa et al., 16 Dec 2025, Jiang et al., 2022, Propp et al., 2024, Jones et al., 2022), or by exploiting physical or data-driven symmetry for augmentation and transfer (e.g., in 3D-CNNs or graph-based networks) (Whalen et al., 2021, Jones et al., 2022).

3. Key Application Domains

Domain	Surrogate/Transfer Role	Representative Methods/Papers
Partitioned multiphysics PDEs	Interface-flux surrogate, reduced coupling solve	DMD-FS, interface operator learning (Bochev et al., 2024)
Engineering/physical simulation	Multifidelity surrogate modeling, low-to-high fidelity transfer	3D U-Nets, masked neural nets, fine-tuning (Ishikawa et al., 16 Dec 2025, Jiang et al., 2022)
Black-box optimization	Knowledge transfer from sources via surrogate competition	SAS-CKT, co-surrogate, TLBO ensemble (Xue et al., 2024, Wang et al., 2021, Tfaily et al., 30 Mar 2026)
Surrogate-based UQ	PCE with Bayesian/tempered transfer; CNNs with multi-dim. data fusion	Transfer-PCE (Bridgman et al., 2023); CNNs (Propp et al., 2024)
Adversarial ML	Transferable attacks via surrogate ensembles or meta-surrogates	SEA, MSM, undertraining, GFCS (Yang et al., 19 May 2025, Qin et al., 2021, Miller et al., 2020, Lord et al., 2022)
Graph-based surrogates	Domain-adaptation, transfer between topologies/loads	GSMs, message-passing GNNs (Whalen et al., 2021)
Hyperparameter optimization	Surrogate alignment, neural transfer functions	HTS (Ilievski et al., 2016)

4. Accuracy, Performance, and Robustness

Accuracy and computational performance metrics are central in evaluating surrogate and transfer-based methods, often benchmarked against monolithic solutions (in partitioned PDEs), or vanilla, no-transfer surrogates in optimization and UQ.

• In partitioned PDEs, dynamic-flux surrogate methods achieve $L^2$/$H^1$ errors only marginally above Schur-complement or monolithic solvers, while providing 10–40× speedup (Bochev et al., 2024).
• Multifidelity deep surrogates reduce HF simulation requirements by an order of magnitude and maintain prediction fidelity within 5%–10% of direct training on all-HF data (Jiang et al., 2022, Ishikawa et al., 16 Dec 2025).
• Probabilistic transfer in PCE yields 20%–50% RMSE reduction in domain/task-shifted UQ problems, with tempering mechanisms preventing negative transfer (Bridgman et al., 2023).
• In adversarial ML, transfer-based attacks surpass 8–70% transferability gains for fixed per-iteration cost, often matching or exceeding the performance of full large-ensemble or meta-optimization alternatives, with surrogate diversity and cross-iteration coverage key to effectiveness (Yang et al., 19 May 2025, Miller et al., 2020, Lord et al., 2022).
• SAS-CKT and similar frameworks rigorously avoid negative transfer by construction, with knowledge competition maintaining or improving convergence speed and surrogate prediction accuracy over extensive benchmarks (Xue et al., 2024).

5. Architectural and Implementation Variants

Offline/Online Decoupling. Many recent frameworks shift training and model identification (e.g., surrogate parameter, operator learning, transfer mapping) to a computationally unconstrained offline phase. Online prediction or simulation proceeds with minimized overhead—often a matrix-vector multiplication or a shallow forward pass—enabling high-throughput deployment (Bochev et al., 2024).

Fine-tuning and Layer Strategy. In transfer learning for neural surrogates, best practice is “coarse” pretraining on cheap or low-fidelity data with a wide search over architectures, followed by selective freezing or fine-tuning on scarce, high-fidelity or target data—often only the last layer or output head—balancing feature reuse and overfitting (Pai et al., 2022, Ishikawa et al., 16 Dec 2025, Jiang et al., 2022).

Adversarial Ensemble and Transfer. Selective ensemble attacks (SEA) decouple within- and cross-iteration model diversity: fixed per-iteration bandwidth but dynamically switching surrogates across iterations achieves almost full ensemble transfer efficacy at trivial resource cost increase (Yang et al., 19 May 2025). Undertrained surrogates in adversarial settings reveal that earlier training epochs—despite higher validation loss—yield gradients with stronger cross-model universality and lower curvature, providing state-of-the-art transfer rates with minimal complexity (Miller et al., 2020).

Competitive Knowledge Transfer in Optimization. The SAS-CKT algorithm formalizes a competition between source and target surrogates or solutions, with theoretically guaranteed nonnegative expected convergence gain; only source solutions with nonnegative similarity compete, and adaptation is optionally handled via translation, domain alignment, or label adaptation routines (Xue et al., 2024).

Domain Warping and Affine Transformations. Recent frameworks generalize classic affine-only input alignment (rotation/translation) by adding non-linear warping (e.g., Beta CDF per-dimension), robustly adapting between source and target tasks even under moderate nonstationarity. Optimization is handled with gradient-based methods for differentiable surrogates and black-box search otherwise (Pan et al., 30 Jan 2025).

6. Limitations, Misconceptions, and Future Perspectives

• Negative Transfer Risk. Naïve transfer can lead to performance degradation when source and target are misaligned, especially under sharp multimodality or significant distribution drift. Robustness is enhanced via local transfer gating (LOL-GP), uncertainty-based synthetic-label filtering, or information-theoretic tempering of priors (Wang et al., 2024, Bridgman et al., 2023, Wang et al., 2021).
• Data Regime Dependencies. Transfer-based methods deliver the largest gains in low-data or data-scarce circumstances (e.g., target set sizes ≪ model dimension); as target data grows, scratch-trained surrogates close the gap (Pan et al., 30 Jan 2025, Pan et al., 23 Jan 2025, Pai et al., 2022).
• Limitations in Domain Alignment. If the source and target functions cannot be related by the chosen transformation class (e.g., non-affine, high-curvature warping, or topological disconnect), transfer efficacy is reduced or fails, and additional adaptation modules may be required (Pan et al., 23 Jan 2025, Wang et al., 2024).
• Efficiency/Intrusiveness Trade-offs. High-accuracy partitioned multiphysics methods or surrogate transfer algorithms may be limited by the need for intermediate states, offline data, or interface access, though recent advances (DMD-FS, Spline-based transfer) have narrowed this gap with minimal-intrusion architectures (Bochev et al., 2024, Larose et al., 2024).
• Interpretations in Neural/Graph Surrogates. Simpler fine-tuning, without explicit adversarial or hierarchical adaptation losses, suffices to yield positive transfer in non-Euclidean surrogate architectures, but generalization to truly unseen topologies or tasks remains limited (Whalen et al., 2021).

Research continues to advance the theoretical underpinnings, negative transfer avoidance, multi-source/meta-surrogate architectures, and domain-adaptive mechanisms in surrogate and transfer-based methods, driven by application demands in design, simulation, uncertainty quantification, and security-sensitive ML contexts.