Unified Surrogate Formulation

• Unified surrogate formulation is a framework that integrates diverse data sources and modeling paradigms to approximate expensive objective functions.
• It combines structural and functional metrics for sample selection, model updating, and uncertainty quantification in optimization workflows.
• Empirical results demonstrate reduced computational cost and enhanced prediction accuracy in applications like genetic programming, CFD, and PDE surrogates.

A unified surrogate formulation is a methodological framework in which surrogates—interpolants or regressors trained to approximate expensive or inaccessible objective functions—are constructed, deployed, and optimized using principles that integrate multiple sources of information, modeling paradigms, or application modalities. The unification can be along structural, functional, or procedural dimensions, advancing surrogate modeling in optimization, simulation acceleration, machine learning, and scientific computing.

1. Mathematical Dualities: Structural Unification of Surrogates

Unified surrogate formulations frequently exploit dual perspectives—phenotypic and genotypic, continuous and discrete representations, or deterministic and probabilistic uncertainty estimators. For example, in the pheno-geno unified surrogate genetic programming (PGU-SGP) paradigm, surrogate fitness prediction leverages both phenotypic behavior (action or ranking statistics in sampled decision situations) and explicit genotypic structure (primitives’ normalized frequency in an individual’s program tree). The unified similarity is defined as

$$\mathrm{PGU}(x_1, x_2) = w_p \cdot \tilde{d}_p(x_1, x_2) + w_g \cdot \tilde{d}_g(x_1, x_2),$$

where $\tilde{d}_p$ and $\tilde{d}_g$ are max-normalized phenotypic and genotypic distances, and $w_p, w_g$ are convex weights (Tan et al., 15 Apr 2025). This balances behavioral and structural diversity in surrogate selection and response modeling.

In other domains, unification takes the form of embedding discrete symbolic structures into continuous spaces, such as mapping symbolic regression discoveries into low-dimensional continuous descriptors for Gaussian process surrogates (Fang et al., 22 Dec 2025), or unifying black-box and preference-based optimization under a single surrogate acquisition regime (Previtali et al., 2022).

2. Unified Surrogate Algorithms and Workflow

A unified surrogate algorithm typically consists of:

• Encoding and Metric Definition: Selection and computation of joint representations (e.g., concatenated phenotypic and genotypic distances; continuous mappings of symbolic programs).
• Sample Selection and Diversity Management: Clustering or acquisition based on unified metrics, often hierarchical or via hybrid exploration-exploitation rules.
• Fitness Evaluation and Surrogate Update: Partitioning the population (or candidate models) into evaluation and estimation groups, performing high-fidelity evaluations on representatives, and updating the surrogate sample bank using diversity-preserving rules.
• Prediction and Selection: K-nearest neighbor, GP, or alternative surrogate predictions over the unified space, with mechanisms to disallow elite carryover of models only estimated (and not truly evaluated).
• Evolution, Refinement, or Optimization: Parent selection, variation (GP), or global search (other domains), closing the unified surrogate-assisted loop (Tan et al., 15 Apr 2025, Fang et al., 22 Dec 2025, Previtali et al., 2022).

3. Exemplary Unified Surrogate Models

The following table summarizes key strategies across representative unified formulations:

Approach	Unification Principle	Surrogate Type	Application Context
PGU-SGP (Tan et al., 15 Apr 2025)	Pheno-genotypic metric fusion	KNN (simulation fitness)	Genetic programming optimization
Multi-output GP (Fang et al., 22 Dec 2025)	Symbolic-to-continuous mapping	Multi-output GP	Symbolic regression + CFD
gMRS (Previtali et al., 2022)	Black-box & preference integration	RBF/GP, QP	Black-box & preference optimization
USM-Net (Regazzoni et al., 2022)	Physical/geometric/coordinate blending	ANN	PDE surrogates, variable domains
FM+Ising (Wang et al., 2 Jul 2025)	FM+slack/latent binary variables	Factorization Machine	Combinatorial, quantum-enhanced surrogate

Each formulation explicitly combines information from heterogeneous model or data sources to inform surrogate response, candidate diversity, and uncertainty quantification.

4. Unification in Surrogate Loss Functions

Beyond structural model unification, recent developments provide unified frameworks for generating surrogate loss functions for end-to-end supervised learning across metrics. The UniLoss framework refactors batch-wise performance metrics into modular steps: real-valued scores, pairwise comparisons, thresholding to binaries, and aggregation for final metric computation. Differentiable relaxations for each nondifferentiable step—such as sigmoid approximations for indicator functions, and continuous (e.g., IDW) interpolants for Boolean logic—yield a universal, task-agnostic surrogate loss that is fully differentiable and can be used in gradient descent-based training pipelines (Liu et al., 2020). This formulation unifies the optimization of disparate evaluation metrics under a single computational and theoretical regime.

5. Unification of Surrogate-Based Optimization Strategies

Unified surrogate optimization schemes treat disparate feedback types—numerical function evaluations (black-box settings) and pairwise preferences (human-in-the-loop, subjective criteria)—using common surrogate modeling and sampling instruments. In the generalized Metric Response Surface (gMRS) approach, the same RBF or GP surrogate and exploration metrics drive sample selection regardless of the feedback modality, ensuring theoretical convergence properties in both modes (Previtali et al., 2022).

Similarly, universal uncertainty quantification (UP distribution) uses cross-validated predictions from any surrogate—probabilistic or deterministic—to synthesize empirical predictive distributions, facilitating exploration/ exploitation tradeoffs across model classes (Salem et al., 2015).

6. Applications, Impact, and Empirical Results

Unified surrogate formulations have demonstrated efficiency and accuracy improvements across several domains:

• Combinatorial Scheduling: PGU-SGP reduced training time by ≈76% vs. vanilla GP, improved convergence under fixed computational budget, and enhanced surrogate quality (Pearson correlation 0.73–0.85) compared to conventional PC-only surrogates (Tan et al., 15 Apr 2025).
• Physics-Based Symbolic Model Training: Surrogate-augmented symbolic CFD-driven training maintained predictive accuracy while substantially reducing training cost, enabling multi-objective optimization with matrix-valued GP surrogates (Fang et al., 22 Dec 2025).
• Surrogate Loss Design: The UniLoss approach delivered performance on par with specialized, hand-tuned surrogate losses for classification accuracy and AUC, applying a single formulation to a variety of tasks (Liu et al., 2020).
• Variable-Domain PDE Surrogates: USM-Nets produced accurate, mesh-free predictions for CFD problems across parametric and geometric variations; using “universal coordinates” improved generalization error relative to physically anchored networks (Regazzoni et al., 2022).

These approaches enable surrogate models to generalize better across task variations, support multi-objective optimization, and reduce sample complexity and wall-clock cost.

7. Limitations and Future Directions

Unified surrogate formulations face limitations including sensitivity to the quality of encoded representations (e.g., choice of phenotypic/ genotypic features, landmark vectors), potential for increased model complexity, and the computational cost of initial data (e.g., FOM snapshots for USM-Nets). The construction of sufficiently rich, bijective “universal coordinates” for complex domains remains technically challenging (Regazzoni et al., 2022). Hybrid surrogates involving quantum-enhanced sampling (Wang et al., 2 Jul 2025) or preference-integration (Previtali et al., 2022) require further research on scalability and robustness.

Future directions include learning latent or task-adaptive embeddings for model input spaces, leveraging multi-fidelity or active sampling for uncertainty minimization across broader parametric/structural grids, and extending unification strategies to time-dependent, multi-physics, or very high-dimensional surrogate regimes.

---

Unified surrogate formulation thus provides a rigorous and extensible framework, integrating heterogeneous information and optimization paradigms to address increasing model complexity and computational constraints in modern data-driven science and engineering. Principal advances leverage structural fusion, universal uncertainty, and generalized loss design to support flexible and theoretically principled surrogate-based workflows.