Surrogate Model Refinement Approach

• Surrogate model refinement approaches are strategies that enhance predictions by targeting high-error regions with techniques like active learning and sensitivity analysis.
• They incorporate empirical uncertainty quantification, cross-validation, and hybrid multi-fidelity methods to iteratively improve accuracy while reducing computational cost.
• These methods enable adaptive sampling and feature extraction in complex simulations, ensuring efficient and robust refinement of surrogate models.

A surrogate model refinement approach encompasses a set of strategies or algorithms designed to improve the predictive accuracy, robustness, and efficiency of surrogate models—mathematical emulators that replace expensive or impractical-to-run high-fidelity simulations or black-box systems in computational science and engineering. The refinement process addresses deficiencies in initial surrogate constructions by targeting regions of the input space where errors are high, uncertainties are large, predictions are biased, or data coverage is inadequate. Modern refinement frameworks leverage uncertainty quantification, active learning, cross-validation, sensitivity analysis, adaptive sampling, dimensionality reduction, hybridization across data sources, residual-based metrics, and sensitivity-driven error bounds to guide the choice of new data points or model updates.

1. Fundamental Principles of Surrogate Model Refinement

Refinement of surrogate models is governed by the recognition that (i) initial surrogate models trained on limited, prior-based, or evenly-spaced data may not capture complex response surfaces in regions of practical interest, and (ii) computational resources for generating new high-fidelity samples are constrained. The core principle is to iteratively and selectively improve the surrogate’s local or global accuracy by:

• Quantifying local prediction error or uncertainty, often via cross-validation, Bayesian posterior variances, or empirical error metrics (Salem et al., 2015, Zhang et al., 2018, Jaber et al., 15 Jan 2024).
• Leveraging active learning or acquisition functions that balance exploration (space-filling) with exploitation (uncertainty minimization) (Salem et al., 2015, Chakroborty et al., 2022, Bogoclu et al., 2021).
• Concentrating new sampling or model improvement in regions of greatest epistemic uncertainty or highest probability density as determined by the current state of inference (Zeng et al., 2022, Mattis et al., 2018).
• Embedding the refinement process within a larger framework that may include Bayesian inference, stochastic inversion, or robust optimization (Meles et al., 6 May 2025, Mattis et al., 2018, Archbold et al., 23 Apr 2024).

Refinement is thus inherently sequential, with each cycle informed by error estimation, uncertainty analysis, and, increasingly, automatic or goal-oriented criteria.

2. Universal and Empirical Uncertainty Quantification Methods

A substantial advance in model-agnostic surrogate refinement is the concept of universal empirical uncertainty quantification, which does not require a Gaussian or probabilistic prior. The Universal Prediction (UP) distribution (Salem et al., 2015) is archetypal, operationalized by constructing an empirical distribution over leave-one-out (LOO) cross-validation sub-model predictions: $$\mu_{n,x}(dy) = \sum_{i=1}^n w_{i,n}(x) \delta_{\hat{s}_{n,-i}(x)}(dy),$$ where $w_{i,n}(x)$ are locally-smoothed weights, and $\hat{s}_{n,-i}(x)$ are the LOO predictions at $x$. The sample mean and variance of this distribution supply uncertainty estimates agnostic to the underlying surrogate type. Unlike kriging variances, the UP variance captures model- and data-induced local heteroscedasticity and enables universally applicable adaptive refinement algorithms, such as UP-SMART (targeting large UP variance) and UP-EGO (an expected-improvement-style sampling criterion based on the UP distribution). This breaks free from canonical assumptions that have constrained adaptive design to Gaussian process (GP) surrogates.

Empirical cross-conformal and Jackknife+ prediction intervals—further advanced for GP surrogates (Jaber et al., 15 Jan 2024)—weight non-conformity scores by the GP posterior standard deviation, yielding intervals that both adapt to local surrogate error and come with finite-sample frequentist coverage guarantees. This approach not only provides local error-sensitivity but also serves as a calibration and model selection tool when choosing among possible GP priors or kernel hyperparameters.

3. Adaptive and Active Sampling Strategies

Adaptive sampling is the methodological core of surrogate model refinement. Various strategies are deployed across the literature:

• Variance-Based and Uncertainty-Driven Sampling: Sampling points are chosen to maximize estimated local uncertainty (kriging variance, UP variance, prediction interval width, or Bayesian posterior uncertainty) (Salem et al., 2015, Zhang et al., 2018).
• Cross-Validation and Empirical Distributions: The construction of empirical prediction distributions from cross-validation (e.g., leave-one-out) or bootstrapping sub-models guides sampling toward regions where surrogate predictions are less stable (Salem et al., 2015).
• Posterior-Focused Sampling: When embedded in Bayesian or stochastic inversion frameworks, adaptive refinement prioritizes high-posterior-density (HPD) regions, as these dominate posterior integrals or credible intervals (Zeng et al., 2022, Mattis et al., 2018, Zhang et al., 2018, Meles et al., 6 May 2025).
• Residual- or Physics-Informed Refinement: In surrogates emulating parametric PDEs, mesh refinement is driven by estimates of the PDE residual and probability density (e.g., importance-sampling by weights based on the PDF), so as to align sampling with regions controlling output statistics (Halder et al., 2019).

Active learning frameworks extend adaptive refinement through acquisition functions expressing the expected value of information, improvement, or error reduction, often operationalized as

$$\text{Acquisition}(x) = \hat{\sigma}^2_n(x) + \delta d(x, X_n),$$

or analogous expressions, where $\hat{\sigma}^2_n(x)$ is a local error metric, and $d(x, X_n)$ is the distance to the nearest sampled point (Salem et al., 2015, Bogoclu et al., 2021).

4. Sensitivity and Goal-Oriented Strategies

Sensitivity-driven strategies refine the surrogate where errors have the largest impact on simulation outputs or quantities of interest (QoIs). This approach computes derivatives of the simulation output with respect to the surrogate error using analytical adjoint methods or implicit function theorem results (Cangelosi et al., 4 Sep 2025, Mattis et al., 2018). For dynamical system surrogates, the state and control trajectory’s sensitivity to surrogate error is quantified via linearized ODEs and Frechet derivative chains, enabling precise identification of regions in model space for targeted refinement. The associated acquisition function then reflects the worst-case impact of local surrogate error on the final QoI: $$\max_{\delta g: |\delta g| \leq P} \left| q'(g)\, \delta g \right|,$$ where $P$ is a pointwise error bound (from RKHS theory) on the surrogate, and $q'(g)$ is the Frechet derivative of the QoI with respect to the component model.

Goal-oriented refinement in stochastic inversion iteratively builds surrogates (e.g., piecewise polynomials or Taylor models on Voronoi tessellations), combining a posteriori local error estimators—including adjoint-derived corrections—to iteratively refine only those surrogate regions that most affect the global goal, such as an expectation or an integral (Mattis et al., 2018).

5. Hybrid, Multi-Fidelity, and Feature-Aware Refinement

Modern surrogate model refinement frameworks increasingly exploit multi-source data, varying fidelities, and high-dimensional embeddings:

• Multi-Fidelity Correction and Model Blending: GP-based corrections applied to multiple low-fidelity models, with local model selection (weighted by predicted discrepancy and possibly cost) or probabilistic mixture surrogates, facilitate accurate prediction and sharp reduction in expensive high-fidelity model usage (Burnaev et al., 2017, Chakroborty et al., 2022, Wilke, 21 Apr 2024).
• Hybrid Surrogates from Simulation and Real-World Data: Bayesian frameworks now hybridize surrogates trained on simulation and measurement data either by weighted combination of predictive distributions or via likelihood power-scaling in the posterior (using a mixing factor β), allowing for diagnostic analysis and correction of model misspecification, improved predictive coverage, and adaptability to data scarcity and extrapolation challenges (Reiser et al., 16 Dec 2024).
• Dimensionality Reduction and Feature Extraction: Shared low-dimensional representations are constructed alongside the surrogate in a supervised nested optimization (e.g., kernel PCA supervised by surrogate generalization error (Lataniotis et al., 2018); or neural-network-based goal-oriented feature extraction with contrastive losses on output differences (Wang et al., 14 Nov 2024)). This joint strategy ensures that dimensionality reduction preserves predictive accuracy, alleviating the curse of dimensionality and leading to uniform generalization improvement across surrogate model classes.

6. Quantitative and Algorithmic Performance

Empirical evaluations in benchmark and engineering contexts consistently show that adaptive or sensitivity-driven refinement approaches yield:

• Rapid convergence of surrogate errors in critical regions (e.g., Q², RMSE, RRMSE, and KL divergence metrics) compared to traditional, uniform, or prior-only sampling (Salem et al., 2015, Mattis et al., 2018, Zhang et al., 2018, Zeng et al., 2022, Chakroborty et al., 2022, Wang et al., 14 Nov 2024).
• Dramatic reductions (often 1–2 orders of magnitude) in computational cost, number of high-fidelity model calls, or total simulation time required to reach prescribed accuracy in posterior inference, optimization, or design tasks (Burnaev et al., 2017, Meles et al., 6 May 2025, Chakroborty et al., 2022).
• Reduced surrogate model bias, especially regarding the posterior mean or modes, and a more representative quantification of uncertainty, particularly when using adaptive sampling over the posterior or advanced multi-source/hybrid surrogate construction (Zeng et al., 2022, Meles et al., 6 May 2025, Reiser et al., 16 Dec 2024).
• Data-efficient strategies that outperform purely space-filling methods even in high-dimensional, non-linear, and multimodal problem settings (Zeng et al., 2022, Lataniotis et al., 2018, Wang et al., 14 Nov 2024).

A sample of methods, techniques, and relevant metrics is provided in the summary table:

Method/Class	Refinement Mechanism	Quantitative/Coverage Guarantee or Performance
UP Distribution	Weighted LOO cross-validation sub-models	Empirical variance, universal applicability
Bayesian GP	Posterior variance, cross-conformal intervals	Frequentist coverage, local adaptivity
Kernel Interpolation	RKHS error bounds, adjoint/Fréchet sensitivity	Explicit worst-case QoI error reduction
Multi-fidelity GP	GP correction of LF models, local model mixing	RRMS reduction, 1–2 orders cost decrease
Goal-Oriented Neural	Contrastive/distance-constrained feature learning	Uniform error convergence, generalizability
Active Subset Sim	U-function–based sample selection	HF calls reduced by 100x or more
Iterative Bayesian	Sequential posterior-guided retraining	MAP and credible intervals improved

7. Broader Implications and Generalization

Advances in surrogate model refinement represent a convergence between empirical machine learning, classical uncertainty quantification, and computational design of experiments. Sustained progress is characterized by:

• Decoupling of uncertainty quantification from restrictive probabilistic assumptions, making advanced refinement broadly applicable across model classes.
• Emphasis on data efficiency and practical exploitation of hybrid, multi-source, or multi-fidelity data in real-world engineering and science problems.
• Recognition of the criticality of local error and sensitivity information—not just for prediction but to rigorously guide where expensive computation is most impactful.
• Integration with modern data-driven, feature-extraction, and dimensionality-reduction approaches, ultimately extending the reach of surrogate modeling into ever larger and more complex input/output domains.

Open challenges remain in automating parameter and hyperparameter selection in clustering, adaptive sampling, and feature learning; in extending empirical uncertainty quantification to classes of deep learning surrogates; and in further formalizing the diagnostic capabilities provided by hybrid, blended, or locally-adaptive surrogate outputs. However, the methods summarized form a mature and rigorous foundation for efficient and robust surrogate model refinement across computational engineering, scientific modeling, and beyond.