Residual Operator Learning in Neural PDEs

• Residual Operator Learning is a modeling principle that learns the discrepancy between a baseline prediction and the true solution to enhance prediction accuracy.
• It is applied in neural PDE solvers, iterative methods, and probabilistic frameworks, leading to significant error reductions and improved generalization.
• Empirical studies show that additive reconstructions and residual-to-update mappings effectively refine coarse predictions in simulation and control tasks.

Searching arXiv for the cited papers and closely related work on residual operator learning. arXiv search query: residual operator learning neural operator PDE residual correction

Residual operator learning denotes a family of formulations in which the learned map is not the full target operator in one shot, but a correction, defect, closure, or refinement relative to a baseline object such as an auxiliary trajectory, a coarse surrogate, a linearized solve, a symbolic prefix, or a bag-derived estimator. In the recent literature, this pattern appears in neural PDE solvers, probabilistic operator surrogates, learned preconditioners, weakly supervised uncertainty estimation, robot manipulation, teleoperation, and residual policy refinement. The common reconstruction pattern is additive or compositional: a baseline prediction is first produced or retrieved, and a learned residual operator supplies the unresolved part needed to obtain the final output (Yue et al., 2024, Jha, 2023, Wang et al., 19 Mar 2026, Bhola et al., 14 Dec 2025, Yildirim et al., 3 Jun 2026).

1. Conceptual scope and terminology

The expression “residual operator” has been used in several technically distinct but structurally related senses. In PDE solving, DeltaPhi reformulates direct operator learning

$\mathcal{G}:\mathcal{A}\to\mathcal{U}$

as residual operator learning

$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$

so that the model predicts $u_i-u_j$ between a target solution and a similar auxiliary solution rather than $u_i$ directly (Yue et al., 2024). In the residual-based corrector literature, the residual operator is the linearized variational correction

$$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$

applied after a neural-operator surrogate has produced $\tilde u$ (Jha, 2023). In learned iterative solvers, the learned object is a residual-to-update map embedded inside Born-series, Newton–Kantorovich, or preconditioned iterations rather than an end-to-end solution operator (Wang et al., 19 Mar 2026, Bacho et al., 25 Nov 2025).

Outside PDEs, the same phrase is used for structured correction operators anchored to a pre-existing decision map. In weakly supervised multi-instance uncertainty estimation, the residual operator is

$$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$

a residual-corrected instance estimator derived from a bag-level symmetric-function classifier (Liu et al., 2024). In inverse manipulation, a symbolic planner first restores what it can, and a residual policy learns the continuous control needed to satisfy unresolved inverse predicates, yielding

$$\pi_{\text{inv}}=\pi_{\text{sym}}\circ \pi_{\text{res}}.$$

(Yildirim et al., 3 Jun 2026)

A useful unifying description is that residual operator learning shifts the learning target from the full response manifold to a smaller discrepancy space. In some papers that discrepancy is deterministic and additive; in others it is a solver increment, a metric-weighted defect, a probabilistic correction flow, or a constrained residual control law. This suggests that the field is less a single architecture than a modeling principle about what should be learned.

2. Core mathematical patterns

A first recurring pattern is baseline-plus-correction reconstruction. DeltaPhi trains on residual labels

$$\mathcal{T}^{res}=\{(a_i,a_{k_i}),u_i-u_{k_i}\}_{i=1}^N,$$

with objective

$$\ell=\sum_{i=1}^{N}\mathcal{L}\big(\mathcal{G}_{\theta}^{\Delta}(a_i,a_{k_i}),u_i-u_{k_i}\big),$$

and reconstructs the final prediction as

$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$0

The auxiliary sample is selected by a retrieval function based on similarity and top-$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$1 random sampling during training (Yue et al., 2024). Closely related additive reconstructions appear in residual video diffusion for PDEs,

$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$2

after S-DeepONet supplies a coarse prior (Park et al., 8 Jul 2025), in low-to-high fidelity probabilistic transport,

$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$3

where $$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$4 is learned probabilistically by flow matching in function space (Bhola et al., 14 Dec 2025), and in field-space closure models for PIC simulation,

$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$5

with an additional latent residual closure on the source representation (Lyu et al., 16 Jun 2026).

A second pattern is residual-to-update learning inside an iterative solver. The Neural Preconditioned Born Series replaces the scalar CBS relaxation by a neural operator: $$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$6 so the network learns a correction operator in preconditioned residual coordinates rather than the solution field directly (Wang et al., 19 Mar 2026). CHONKNORIS adopts the same principle in a Newton–Kantorovich setting by learning Cholesky factors of a regularized inverse Hessian-like operator and updating

$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$7

(Bacho et al., 25 Nov 2025)

A third pattern is residual weighting or residual geometry. The preconditioner-learning literature argues that matching residuals in the Euclidean norm can be mismatched to the actual operator geometry, and instead defines

$$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$8

so that residual learning is carried out in the metric induced by the shifted-Laplacian/Born reference operator (Wang et al., 19 Mar 2026). The adaptive-training literature makes a complementary point: residual-based weighting rules can be derived variationally, with exponential weights targeting $$\mathcal{G}^{\Delta}:\mathcal{A}^2\to \Delta\mathcal{U},$$9-like objectives and linear residual weights recovering a variance/$u_i-u_j$0-type regime (Toscano et al., 17 Sep 2025).

These patterns are mathematically distinct, but all replace “learn the whole operator” by “learn the unresolved part of an operator action.”

3. PDE-centered residual operator learning

The most fully developed use of the term is in neural PDE solving. DeltaPhi is explicitly motivated by limited-data, low-resolution, and biased-data regimes in which direct neural operators may overfit the training resolution or training distribution. It keeps the neural-operator backbone

$u_i-u_j$1

unchanged, but changes the inputs and labels to residual form by concatenating $u_i-u_j$2 and predicting $u_i-u_j$3. The paper also introduces customized auxiliary inputs such as partial auxiliary history, the auxiliary output $u_i-u_j$4, and similarity scores $u_i-u_j$5, and reports that these improve optimization (Yue et al., 2024).

Residual formulations in PDE surrogates also appear as hierarchical refinement. ResFNO learns the map from cure cycle $u_i-u_j$6 to temperature history $u_i-u_j$7, but replaces direct fitting of the raw time-domain residual with a Fourier residual mapping based on a low-mode reconstruction of the cure cycle. The purpose is to reduce the singular error near non-differentiable turning points of piecewise cure cycles and to preserve time-resolution independence of the operator parameterization (Chen et al., 2021). The two-stage S-DeepONet plus video-diffusion framework similarly decomposes the spatio-temporal solution into a coarse, physics-consistent prior and a residual containing high-frequency structures. On lid-driven cavity flow, the reported mean relative $u_i-u_j$8 error drops from $u_i-u_j$9 for S-DeepONet to $u_i$0 for prior-conditioned residual diffusion, while on dogbone plasticity it drops from $u_i$1 to $u_i$2 (Park et al., 8 Jul 2025).

Residual closure is also used for multi-field kinetic plasma simulation. LRC-FNO introduces two residual levels: a latent closure refiner corrects information lost by source compression, and a residual-closure FNO restores the high-resolution field correction missed by a coarse field solver. In the 2D scrape-off layer benchmark, the paper reports relative $u_i$3 errors of $u_i$4 for the self-consistent potential and $u_i$5 for the magnetic vector potential in single-step prediction, while also using the network as an initialization for iterative corrections in closed-loop PIC integration (Lyu et al., 16 Jun 2026).

Residual operator learning can also be coupled to geometry normalization. NDNO maps varying 3D component geometries to a common reference domain by a diffeomorphic neural network and then learns the operator from residual stress fields to deformation fields on that reference domain. The learned deformation is finally mapped back to the original geometry. This converts variable-domain operator learning into reference-domain residual-stress-to-deformation learning with smoothness, invertibility, and Sinkhorn similarity constraints on the geometry map (Liu et al., 9 Sep 2025).

A probabilistic variant is given by residual-augmented flow matching in function space. There the model learns a stochastic correction transport from a low-fidelity PDE approximation to the high-fidelity solution manifold rather than learning the full conditional solution distribution from scratch. The operator backbone is FiLM-conditioned and resolution invariant, and the learned vector field is decomposed into a linear operator plus a nonlinear operator for stability and expressiveness (Bhola et al., 14 Dec 2025).

4. Correctors, variational correctness, and operator-theoretic analysis

A major branch of the literature treats residual operator learning not as a surrogate for the solution operator itself, but as a post-processing or solver mechanism with explicit error-control semantics. The residual-based corrector operator for nonlinear variational boundary-value problems linearizes the PDE residual around a neural-operator prediction and solves

$u_i$6

after which $u_i$7. Under boundedness and invertibility assumptions, the corrected error is quadratic in the original prediction error. In numerical experiments on a nonlinear reaction–diffusion model, the paper reports almost two orders of increase in accuracy, and in topology optimization the error of surrogate-based optimizers can be as high as $u_i$8 without correction but below $u_i$9 with correction (Jha, 2023).

The variationally correct operator-learning literature makes a different but related point: standard PDE-residual losses are often not equivalent to the true solution error because of non-compliant norms and ad hoc boundary penalties. RBNO therefore constructs first-order system least-squares objectives whose residual values are provably equivalent to the solution error in PDE-induced norms, and constrains the network output to a conforming reduced basis. In this setting the residual loss serves as a reliable, computable a posteriori error estimator (Qiu et al., 24 Dec 2025).

Learned preconditioners provide a third interpretation. The Born-series-inspired metric paper argues that the relevant residual geometry is not Euclidean for indefinite operators such as high-frequency Helmholtz. Using the identity

$$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$0

it defines the metric $$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$1 and the loss

$$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$2

which matches the geometry used at inference time (Wang et al., 19 Mar 2026). The adaptive variational framework generalizes this logic by linking residual transformations, sampling distributions, and target norms; exponential weighting corresponds to a softmax tilt and targets uniform-error reduction, while quadratic potentials recover linear residual weighting and a variance/$$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$3-type regime (Toscano et al., 17 Sep 2025).

At a more abstract level, operator defect theory treats iterative approximation schemes as products of contractions $$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$4 with defect operator $$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$5. The exact telescoping identity

$$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$6

interprets residuals as operator-theoretically defined defect energies rather than heuristic errors. This framework is then applied to Kaczmarz-type methods, RKHS interpolation, kernel compression, and greedy KPCA (Jorgensen et al., 26 Jan 2026).

5. Extensions beyond PDE surrogates

Residual operator learning has also been adopted in settings where the “baseline object” is not a PDE approximation. In multi-instance uncertainty estimation, MIREL derives an instance estimator $$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$7 from the permutation-invariant bag classifier

$$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$8

then corrects it by a residual branch

$$\mathcal F^C(m,\tilde u)=\tilde u-\delta_u R(m,\tilde u)^{-1}R(m,\tilde u),$$9

The output is mapped to a Dirichlet evidential predictor by $\tilde u$0. On MNIST-bags, the ablation reports bag-level $\tilde u$1 and instance-level $\tilde u$2 for the residual formulation; on CAMELYON16, bag-level $\tilde u$3 and instance-level $\tilde u$4 are reported (Liu et al., 2024).

In inverse manipulation, the forward skill is first abstracted as a STRIPS-like operator $\tilde u$5, and the inverse target is defined by

$\tilde u$6

A BFS-based symbolic planner executes scripted primitives, yielding a handoff state $\tilde u$7; satisfied predicates become fences, unresolved predicates become the active residual objective, and a Soft Actor-Critic policy learns the continuous correction. On ManiSkill3 PushCube, the symbolic prefix alone yields $\tilde u$8 mm mean distance and $\tilde u$9 success @ 1 cm, whereas symbolic + RL yields $$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$0 mm and $$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$1 success (Yildirim et al., 3 Jun 2026).

In dexterous teleoperation, ResPilot begins with an optimization-based retargeter, HKVM, then learns fingerwise Gaussian-process residuals in an angle-aware 2D vector representation. The final command is

$$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$2

so the GP corrects rather than replaces the baseline hand-mapping policy. The method uses 24 calibration poses, about 4.5 minutes total calibration and GP fitting time, GP inference of about 9 ms across all fingers, and reports joint workspace $$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$3 versus $$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$4 for HKVM and fingertip workspace $$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$5 versus $$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$6 (Naughton et al., 2024).

Residual refinement is also used in long-horizon control. KORR keeps the standard executed action

$$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$7

but conditions $$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$8 on a Koopman-predicted next latent state

$$R(\mathbf{x})=g_{\phi}(f_{\psi}(\mathbf{x})) + r_{\pi}(f_{\psi}(\mathbf{x})),$$9

On FurnitureBench tasks, the reported gains are largest under stronger perturbations, for example One_Leg High with disturbance improves from $$\pi_{\text{inv}}=\pi_{\text{sym}}\circ \pi_{\text{res}}.$$0 for ResiP to $$\pi_{\text{inv}}=\pi_{\text{sym}}\circ \pi_{\text{res}}.$$1 for KORR, and Lamp Med with disturbance improves from $$\pi_{\text{inv}}=\pi_{\text{sym}}\circ \pi_{\text{res}}.$$2 to $$\pi_{\text{inv}}=\pi_{\text{sym}}\circ \pi_{\text{res}}.$$3 (Gong et al., 16 Sep 2025).

A different use appears in evolutionary multitasking, where MFEA-RL uses a VDSR model to generate a high-dimensional residual representation of an individual, a ResNet-based mechanism for dynamic skill-factor assignment, and a random mapping mechanism to project crossover back into the original decision space. The benchmarks are CEC2017-MTSO, WCCI2020-MTSO, and a Sensor Coverage Problem (Wang et al., 27 Mar 2025).

6. Empirical tendencies, misconceptions, and limitations

A recurrent empirical finding is that residual formulations are especially effective when the baseline captures large-scale structure and the learned residual is smaller, more regular, or statistically more diverse than the full target. DeltaPhi explicitly attributes its gains to auxiliary trajectories acting as a nonparametric physical prior and to reshaping the residual label distribution through retrieval range $$\pi_{\text{inv}}=\pi_{\text{sym}}\circ \pi_{\text{res}}.$$4; the paper reports consistent improvements over direct learning on Darcy Flow, Navier–Stokes, and five irregular-domain tasks, with gains often larger when training data are scarce and under zero-shot resolution generalization (Yue et al., 2024). The S-DeepONet-plus-diffusion paper makes the same point in generative form, arguing that diffusion can focus on sharpening high-frequency structures once the global low-frequency scaffold is supplied by the prior (Park et al., 8 Jul 2025).

A common misconception is to equate residual operator learning with ordinary architectural skip connections. The literature is more specific. In several papers the residual object is an operator on discrepancy space, a variational corrector, or a solver increment with its own objective and inference semantics, not merely a residual block in a neural network (Jha, 2023, Wang et al., 19 Mar 2026, Bacho et al., 25 Nov 2025). Another recurrent clarification is that a small residual loss is not automatically physically meaningful; the variationally correct literature emphasizes that residual norms must be compatible with PDE-induced norms and boundary conditions if they are to function as error estimators (Qiu et al., 24 Dec 2025).

The literature also records clear limitations. DeltaPhi states that benefits may be limited when input–output correlation is weak, as in chaotic systems, and that residual learning can be harder to optimize than direct learning, requiring sufficient backbone expressivity; for long-horizon time-series PDEs, gains may shrink as the relation between initial input and future output weakens (Yue et al., 2024). The probabilistic residual-flow framework assumes access to a useful low-fidelity model (Bhola et al., 14 Dec 2025). KORR reports that non-linear dynamics predictors can hurt performance relative to Koopman linearity in future-state-guided residual refinement (Gong et al., 16 Sep 2025). ResPilot notes that contact constraints are task-dependent and may hinder some dynamic rotations or pivoting tasks (Naughton et al., 2024).

These results suggest a general, but not universal, principle. Residual operator learning is most effective when a baseline operator, prior, or planner already captures a substantial invariant component of the target behavior, leaving a correction problem that is simpler than the original one. When that decomposition is poorly aligned with the underlying dynamics, the residual formulation can lose its advantage.