---
title: Residual DeepONet Scheme
url: https://www.emergentmind.com/topics/residual-deeponet-scheme
type: topic
---

# Residual DeepONet Scheme

The Residual DeepONet scheme encompasses a family of operator-learning neural network architectures and methodologies for enhancing surrogate modeling of high-dimensional parametric systems, particularly those governed by partial differential equations (PDEs), by explicitly learning and correcting the residual error between a fast surrogate (e.g., a reduced order model, ROM) and a high-fidelity reference. This paradigm is implemented in several forms, notably for stationary reduced-order modeling via multi-fidelity DeepONet, for time-dependent PDE surrogates via physics-informed time-integrated DeepONet, and for encoding geometric and parametric complexity in elastoplastic systems using residual U-Net–DeepONet hybrids. The unifying principle is to combine classical projection-based or data-driven surrogates with a DeepONet architecture designed to predict, correct, or monitor model residuals, thereby achieving dramatic improvements in predictive accuracy, stability, and generalization—often at negligible additional online computational cost [2302.12682, 2508.05190, 2306.03645].

## 1. Mathematical Formulation of Residual Learning in DeepONet

Let $\Omega\subset\mathbb{R}^d$ denote the spatial domain and $\mathcal{P}\subset\mathbb{R}^p$ the parameter domain. For a parametric PDE, denote by $u_h(\mu,x)$ the high-fidelity (“truth”) solution at parameter $\mu$ and spatial location $x$. A reduced-order model (ROM) $u_R(\mu,x)$ provides a fast, low-dimensional approximation. The pointwise residual is defined as
$$
r(\mu,x) = u_h(\mu,x) - u_R(\mu,x).
$$
Residual DeepONet seeks to learn a neural approximation $\mathcal{R}_\theta(\mu,x)\approx r(\mu,x)$, so that the corrected output
$$
u_{\text{corr}}(\mu, x) = u_R(\mu, x) + \mathcal{R}_\theta(\mu, x)
$$
approaches the true solution $u_h$ with much reduced error. For time-dependent models, the Physics-Informed Time-Integrated (PITI) DeepONet learns to approximate the time-derivative $u_t = \mathcal{L}[u]$ and integrates it forward via explicit or implicit numerical schemes, while monitoring residuals as a proxy for out-of-distribution detection [2302.12682, 2508.05190].

## 2. DeepONet Architectures for Residual Learning

### 2.1 Standard and Multi-Fidelity DeepONet

DeepONet decomposes the operator learning task into two neural sub-networks:
- **Branch Network** $b(\cdot;\theta_b)$: encodes parametric input (e.g., $\mu$, or sensor projections $\tilde{u}$ in gappy-POD) to output $p$ coefficients.
- **Trunk Network** $t(\cdot;\theta_t)$: encodes the location $x$ into $p$ basis functions.

The residual is expressed as an inner product:
$$
\mathcal{R}_\theta(\mu, x) = \sum_{k=1}^p b_k\bigl(\mu; \theta_b\bigr) \, t_k\bigl(x; \theta_t\bigr).
$$

### 2.2 Physics-Informed Dual-Output DeepONet (Time Integration)

PITI-DeepONet employs a dual-output architecture:
- **Branch**: takes the discretized current field $\mathbf{u}^n$.
- **Two Trunks**: output the state $\hat{u}^n(x, t)$ and its time-derivative $\widetilde{u_t}^n(x, t)$, each assembled as a branch–trunk inner product.
- **Time-Stepping**: Network-predicted $\widetilde{u_t}^n$ is advanced using schemes such as Euler, RK4, or Adams–Bashforth–Moulton [2508.05190].

### 2.3 ResUNet-DeepONet for Complex Geometries

For prediction in elastoplasticity under complex geometries, a residual U-Net (ResUNet) trunk encodes 2D spatial patterns, while a fully-connected branch network encodes scalar loads. Element-wise multiplication fuses the branch and trunk in the latent space, with final output by channel-wise reduction.

## 3. Multi-Fidelity Training and Loss Functions

### 3.1 Training Objectives

Residual DeepONet is trained to minimize the mean squared residual:
$$
\mathcal{L}(\theta) = \frac{1}{N\,n}\sum_{i=1}^N\sum_{j=1}^n \left[\mathcal{R}_\theta(\mu^{(i)}, x_j) - r^{(i,j)}\right]^2
$$
where $N$ and $n$ index the parameter and spatial samples. In gappy-POD, the branch takes $\tilde{u}$ from sensor placements as input.

PITI-DeepONet is trained with hybrid losses including PDE residual, initial condition matching, boundary consistency, and optional data-driven losses on the state and time-derivative:
$$
\mathcal{L} = \lambda_{\text{PDE}}\,\mathcal{L}_{\text{PDE}} + \lambda_R\,\mathcal{L}_R + \lambda_{\text{BC}}\,\mathcal{L}_{\text{BC}} + \lambda_C\,\mathcal{L}_C + \lambda_u\,\mathcal{L}_u + \lambda_{u_t}\,\mathcal{L}_{u_t}
$$
[2302.12682, 2508.05190].

### 3.2 Implementation and Hyperparameters

Empirically, effective configurations include:
- Shallow branch/trunk (2–3 layers, 20–50 neurons, softplus or PReLU), $p=20$–$30$ outputs for standard DeepONet [2302.12682].
- ResUNet trunk with four-level encoder–decoder, $\sim$3.5 million parameters, with small MLP branch (32 output channels) for geometry–load fusion in elastoplasticity [2306.03645].
- Training with Adam optimizer, learning rates $3$–$5\times 10^{-3}$, $10^4$–$5\times 10^4$ epochs depending on the problem.


## 4. Validation, Performance, and Benchmark Results

Numerical results consistently show Residual DeepONet significantly outperforms both standalone ROMs and vanilla operator networks. Representative findings:

| Problem              | Baseline ROM     | DeepONet    | MFDeepONet/PITI |
|----------------------|------------------|-------------|-----------------|
| 1D toy (POD $\varepsilon=0.99$)   | 0.324           | 0.270      | **0.247**         |
| 2D Navier–Stokes (POD $\varepsilon=0.99$) | 0.105           | 0.072      | **0.022**         |
| 1D Heat Eq. ($t=5$) (FR)           | 1.5             | –           | **0.24**          |
| 1D Burgers’ ($t=1$) (FR)           | 0.13            | –           | **0.018**         |
| 2D Allen-Cahn ($t=1$) (FR)         | 0.20            | –           | **0.12**          |
| Elastoplastic ResUNet-DeepONet     | 8.2%            | 27.5% (FC) | **8.5%**          |

Mean errors for MFDeepONet are reduced by up to an order of magnitude (e.g., $0.135\to0.010$ in gappy-POD Navier–Stokes). Computational speed-up is substantial: online costs are the sum of the ROM and DeepONet forward evaluations, both $O(10^{-3})$s per case, contrasting with finite-element simulations at $O(10^1)$s [2302.12682, 2508.05190, 2306.03645].

## 5. Critical Analysis and Scope of Application

### Strengths

- **Data Efficiency**: No extra high-fidelity queries are required beyond initial ROM construction.
- **General-Purpose Compatibility**: Can be applied to any non-intrusive ROM (POD-RBF, gappy-POD, DEIM, manifold methods).
- **Projection-Focused Learning**: Targets the easier sub-task of learning projection error rather than the entire solution field.
- **Substantial Error Correction**: Error reductions are most pronounced when baseline surrogates are only moderately accurate.
- **Physics-Informed Generalization**: In time-dependent settings, explicit residual monitoring serves as a zero-cost out-of-distribution detector and quality estimator with high correlation ($\rho\gtrsim 0.99$) to true error.

### Limitations

- Marginal utility if the baseline ROM is already highly accurate (e.g., high POD energy threshold $\epsilon\to 1$).
- Offline costs for network training and hyperparameter optimization can be nontrivial.
- Extension to unsteady or history-dependent systems requires additional modeling (e.g., recurrent or convolutional trunk/branch).
- No guaranteed a priori error bounds—entirely data-driven.

### Applicability

Residual DeepONet frameworks have demonstrated efficacy in:
- Parametric PDE surrogates for real-time inference.
- Outer-loop tasks such as sensitivity analysis, optimization, and uncertainty quantification, especially with variable and complex geometric domains [2302.12682, 2306.03645].
- Time-dependent multi-physics models where traditional rollout methods (full rollout, autoregressive) fail due to error accumulation or instability; PITI-DeepONet offers significant improvements for long-horizon prediction [2508.05190].


## 6. Architectures and Mechanisms: Extensions and Future Directions

### Intrusive Variants

Learning residual source-terms for projection-based Galerkin ROMs offers a route to intrusive versions of the method, with the potential for enforcing hard constraints (e.g., conservation).

### Hybrid Fusion and Advanced Networks

Elementwise fusion in latent space (ResUNet-DeepONet) enables effective encoding of rich geometric context and parameter coupling. This approach is extendable to recurrent, attention-based, or physics-constrained trunk/branch networks, and prospective work includes:
- Time-sequence and path-dependent loading via RNN/CNN branch extensions.
- Plastic-strain or joint field prediction for elastoplastic modeling.
- Physics-informed regularization within the residual network to enforce conservation laws [2302.12682, 2508.05190, 2306.03645].

### Adaptive Sampling and Quality Estimation

Residual-based error indicators from DeepONet enable active learning strategies that systematically enrich the training set where model confidence is low—a plausible direction for more robust outer-loop deployments.

## 7. Significance in Scientific Machine Learning

Residual DeepONet constitutes a key advance in operator learning for scientific machine learning, enabling the correction of model-form errors in real-time surrogates while leveraging deep neural networks’ flexibility for generalizing across high-dimensional, parametrically rich inputs and outputs. Its multi-fidelity, physics-informed, or hybrid instantiations systematically reduce surrogate error and offer practical reliability benefits (e.g., fail-safe detection of out-of-sample states), making it a foundational tool in the numerical simulation and design of complex physical systems [2302.12682, 2508.05190, 2306.03645].

Source: https://www.emergentmind.com/topics/residual-deeponet-scheme