Evolutionary Multi-Objective Operator Learning Networks

• The paper presents a novel architecture that employs evolutionary multi-objective optimization and replica-exchange SGLD to effectively balance data, physics, and boundary losses for robust PDE solutions.
• The methodology uses a refined Pareto sampling strategy to automatically determine optimal parameter trade-offs, ensuring high model generalizability and accurate prediction of nonlinear PDE dynamics.
• Bayesian uncertainty quantification is integrated to deliver credible predictive intervals, making the framework valuable for real-world applications such as fluid dynamics and parameter inversion.

Evolutionary multi-objective optimization for replica-exchange-based physics-informed operator learning networks is a computational framework designed to efficiently solve parametric partial differential equations (PDEs), integrating principles from evolutionary algorithms, multi-objective optimization, physics-informed learning, and advanced sampling techniques. This approach addresses key limitations of conventional operator learning architectures by providing adaptive balancing of multiple loss terms, improved robustness under noisy or sparse data, and intrinsic quantification of predictive uncertainty.

1. Conceptual Framework and Definition

The operator learning network—referred to as Morephy-Net (Lu et al., 31 Aug 2025)—extends physics-informed Deep Operator Networks (PI-DON) and related neural operator models by embedding an evolutionary multi-objective optimization scheme and a replica-exchange stochastic gradient Langevin dynamics (reSGLD) sampler. The architecture splits the network into “branch” and “trunk” modules, generating predictions for PDE solution fields as

$$G_\theta(\kappa)(\mathbf{x}) = \sum_{k=1}^p b_k(\kappa)\,t_k(\mathbf{x}) + b_0,$$

where $\kappa$ denotes the problem input (such as initial conditions or PDE parameters).

Training involves simultaneous minimization of multiple loss components:

• Data loss ($\mathcal{L}_{\text{data}}$): measuring fit to observed or surrogate data
• Physics loss ($\mathcal{L}_{\text{PDE}}$): enforcing the PDE residual and physical laws
• Boundary/initial loss ($\mathcal{L}_{\text{bc}}$): satisfying boundary and initial conditions

Instead of aggregating these terms via static weights, the framework formulates the optimization as a multi-objective problem and employs evolutionary algorithms (e.g., NSGA-III with refined Pareto sampling) to achieve adaptive Pareto-optimal trade-offs.

2. Evolutionary Multi-objective Optimization in Operator Learning

Evolutionary multi-objective optimization (EMO) replaces traditional single-weighted minimization with simultaneous search for solutions minimizing all objectives. Candidate solutions are evolved via non-dominated sorting genetic algorithms (NSGA-III), generating populations ($\{\theta_j\}$) evaluated for each objective: $$\mathbf{F}(\theta) = \left(\mathcal{L}_{\text{data}}(\theta),\,\mathcal{L}_{\text{PDE}}(\theta),\,\mathcal{L}_{\text{bc}}(\theta)\right).$$ The algorithm identifies a Pareto front of parameter configurations $\theta^*$ which are not dominated with respect to all loss terms: $$\theta^* \in \{\theta : \nexists\, \tilde\theta \ \text{such that} \ \mathbf{F}(\tilde\theta) \leq \mathbf{F}(\theta) \ \text{and} \ \mathbf{F}(\tilde\theta) \neq \mathbf{F}(\theta)\}.$$ Refined Pareto sampling (RPS) is applied to sample diverse and representative trade-off solutions from the non-dominated set.

This methodology adaptively balances the operator (data) and physics losses without sensitivity to manually selected weights, yielding more generalizable models in settings with conflicting objectives, as in operator learning for noisy or limited data regimes.

3. Replica-Exchange Stochastic Gradient Langevin Dynamics

Replica-exchange stochastic gradient Langevin dynamics (reSGLD) is integrated to improve exploration of the high-dimensional, non-convex parameter space. Multiple SGLD chains run in parallel at different “temperatures” ($\tau$), i.e., levels of stochastic noise: $$\theta^{(k+1)} = \theta^{(k)} - \eta^{(k)} \nabla \tilde{L}(\theta^{(k)}) + \sqrt{2\eta^{(k)}\tau}\,\xi^{(k)}, \quad \xi^{(k)} \sim \mathcal{N}(0, I).$$ Chains at higher $\tau$ explore broadly, escaping local minima, while lower $\tau$ chains refine solutions. Periodic replica swaps are attempted using an acceptance criterion

$$\tilde{S} = \exp\left[\left(\frac{1}{\tau_1} - \frac{1}{\tau_2}\right)\left(\tilde{L}(\theta^{(1)}) - \tilde{L}(\theta^{(2)}) - \Delta\right)\right],$$

where $\Delta$ corrects for mini-batch stochasticity. This strategy enhances global convergence and mitigates trapping by facilitating information exchange across diverse solution states.

4. Bayesian Uncertainty Quantification

A salient property of this architecture is built-in Bayesian uncertainty quantification. The ensemble of sampled network parameters from reSGLD forms a (pseudo-)posterior distribution over model predictions. Predictive mean and variance at space-time coordinates $(x, t)$ are estimated as

$$\bar{u}(x, t) = \frac{1}{N_s}\sum_{i=1}^{N_s} G_{\theta_i}(\kappa)(x, t),$$

$$\sigma^2(x, t) = \frac{1}{N_s - 1}\sum_{i=1}^{N_s} \left(G_{\theta_i}(\kappa)(x, t) - \bar{u}(x, t)\right)^2,$$

where $\{\theta_i\}$ are SGLD samples. This approach yields credible intervals and uncertainty diagnostics directly from the optimization process, contributing to model reliability in scientific applications.

5. Numerical Results and Comparative Performance

The framework’s efficacy has been validated on canonical problems:

• Burgers’ equations (forward and inverse): Morephy-Net captures nonlinear shock dynamics and accurately reconstructs solution fields, outperforming DeepONet, PI-DON, and PI-FDON architectures, especially under noisy data scenarios.
• Time-fractional mixed diffusion-wave equations (TFMDWE): Lower $L_1$ and $L_2$ errors and uncertainty intervals reliably encompassing the ground truth demonstrate robustness in both forward solution and inverse parameter estimation tasks.

Notable findings include:

• Superior accuracy and robustness to noise compared to baseline operator learning approaches
• Stable prediction of harder cases (e.g., sharp discontinuities, parameter inversion)
• Reliable uncertainty quantification derived from the model ensemble

6. Mechanistic Implications and Extensions

The evolutionary multi-objective optimization strategy allows for adaptive and self-organized balancing of conflicting criteria within physics-informed operator learning frameworks. Replica-exchange methods further enhance exploration and convergence, particularly in multimodal or biased landscapes associated with PDE parameterization. Integrated Bayesian uncertainty quantification supports rigorous assessment of model reliability.

These features render the framework applicable to:

• Data assimilation in real-time physical systems (fluid dynamics, turbulence modeling)
• Inverse problem settings with limited or noisy observed data
• Design optimization and control, where uncertainty propagation is crucial

A plausible implication is that similar evolutionary multi-objective and replica-exchange strategies can be generalized to alternative physics-informed architectures and high-dimensional multi-objective optimization settings, potentially incorporating advanced techniques such as enhanced ideal objective vector estimation (Zheng et al., 28 May 2025), ML2O/GML2O learning-based optimizers (Yang et al., 2023), and surrogate-assisted approaches for accelerating multiphysics applications (Botache et al., 2023).

7. Outlook and Research Directions

Ongoing developments focus on further improving the multi-objective optimization (e.g., extending reference point assignment (Prieto et al., 2020), adaptive operator selection, bias-corrected ideal vector estimation (Zheng et al., 28 May 2025)), integrating explainable AI modules and surrogate models for efficiency (Botache et al., 2023), and formalizing convergence guarantees of hybrid evolutionary-physics-informed architectures. Extensions to many-objective settings, more sophisticated replica management, and real-world robustness studies—particularly addressing highly biased or sparse data domains—constitute active research avenues. This methodological convergence is likely to drive advances in scientific machine learning for complex physics-based models and operator learning.