Adjoint Operator Learning

• Adjoint Operator Learning is a set of techniques that leverage adjoint computations to efficiently compute gradients and recover operators in high-dimensional settings.
• Methods include fast exact adjoint applications for structured operators like wavelet reconstruction and convolution, as well as adjoint-free recovery using projection-based estimation.
• Extensions such as stochastic operator networks and continuous adjoint training in reduced-order models demonstrate improved training stability and effective uncertainty management in noisy data.

Searching arXiv for the papers on arXiv and closely related operator-learning work on adjoints. arXiv search query: "(Folberth et al., 2017) Efficient Adjoint Computation for Wavelet and Convolution Operators" Adjoint operator learning can be understood as a cluster of methods in which adjoint operators, adjoint equations, or adjoint-free surrogates are central to the approximation, identification, or training of operators. In the literature considered here, this includes efficient application of exact adjoints for structured linear maps such as multistage wavelet reconstruction and finite convolution, projection-based recovery of compact non-self-adjoint operators without querying the adjoint, stochastic operator networks trained through an adjoint backward stochastic differential equation, and data-driven reduced-order models trained by a continuous adjoint-state method (Folberth et al., 2017, Boullé et al., 2024, Bausback et al., 10 Jul 2025, Liu et al., 12 Jan 2026).

1. Conceptual roles of the adjoint

Adjoints appear in these works in several technically distinct roles. In large-scale first-order optimization, gradients of differentiable objective terms often involve a linear operator and its adjoint, so the practical problem is to apply both rapidly. This is the setting for the efficient adjoint constructions for wavelet reconstruction and convolution (Folberth et al., 2017).

In operator recovery, the adjoint is traditionally tied to learning both range and co-range information. Classical low-rank recovery or randomized SVD forms sketches $Y=K\Phi$ and also sketches $K^*$, since without querying $K^*$ it appears difficult to learn left singular directions or the dual action of a non-self-adjoint operator. The adjoint-free framework addresses precisely this issue by replacing adjoint queries with projection onto a known eigenbasis of a self-adjoint prior operator $L$ (Boullé et al., 2024).

In stochastic operator learning, the adjoint is elevated from a linear-algebraic primitive to a stochastic control object. The Stochastic Operator Network formulates the branch net as an Itô SDE, introduces adjoint processes $(B_t,C_t)$ through a backward SDE, and replaces the gradient of the loss function by the gradient of the Hamiltonian in the SGD update (Bausback et al., 10 Jul 2025).

In reduced-order modeling, the adjoint takes the form of a continuous-time dual ODE associated with a trajectory-based loss. The adjoint-state method yields exact gradients with respect to reduced operator parameters while avoiding numerical differentiation of noisy snapshot data (Liu et al., 12 Jan 2026).

These roles are complementary rather than interchangeable. One line of work asks how to compute $K^*$ efficiently; another asks when $K^*$ can be avoided; the remaining two use adjoint dynamics as the mechanism for end-to-end training.

2. Exact adjoint application for structured operators

A concrete adjoint-computation problem arises when a fast software routine implements a structured operator in a form that obscures its true adjoint. For multistage discrete wavelet reconstruction, the synthesis operator $W:\mathbb R^p\to\mathbb R^N$ is written as

$W = W_{zpd}\circ E,$

where $E$ is a boundary-extension operator and $K^*$0 is a zero-padded multilevel inverse wavelet transform. In matrix notation,

$K^*$1

By standard rules,

$K^*$2

For orthogonal wavelets, $K^*$3, and for biorthogonal wavelets, $K^*$4 with analysis using the dual wavelet filters. Accordingly, in common cases,

$K^*$5

The technical obstacle is the extension factor. The derivation covers zero padding, symmetric extension, and periodic extension, and shows that in every case one obtains an $K^*$6-cost routine for extension and its adjoint by simple indexing and scaling. For zero padding, $K^*$7 is the simple extract-the-middle-$K^*$8-entries operator. For symmetric extension, assuming $K^*$9,

$K^*$0

For periodic extension, $K^*$1 is expressed through $K^*$2 times the appropriate block overlaps. The fast implementation of $K^*$3 is then: extend $K^*$4 to length $K^*$5 by $K^*$6, and call the library’s forward wavelet-analysis transform $K^*$7. Step 1 costs $K^*$8; Step 2 costs $K^*$9. The paper explicitly notes that MATLAB Wavelet Toolbox, PyWavelets, and Wavelab can be leveraged by re-implementing only the extension adjoint around existing analysis or synthesis routines (Folberth et al., 2017).

The second example is finite convolution. For $L$0 and $L$1, the full linear convolution operator $L$2 is

$L$3

with $L$4 interpreted as zero outside $L$5. Here $L$6 is a Toeplitz matrix, and the adjoint satisfies

$L$7

Equivalently,

$L$8

The direct implementation has cost $L$9, while FFT-based convolution yields $(B_t,C_t)$0. In a first-order method, the canonical pattern is

$(B_t,C_t)$1

The significance of these constructions is narrow but foundational: exact adjoints can be made as cheap, asymptotically, as the forward operator itself. This removes a common implementation barrier in gradient-based inverse problems and related operator-learning pipelines.

3. Adjoint-free recovery of compact operators

A different line of work begins from an apparent paradox: practical operator-learning methods often recover a non-self-adjoint operator using only forward data, even though classical intuition suggests that access to $(B_t,C_t)$2 should be essential. The adjoint-free framework addresses this for compact operators $(B_t,C_t)$3 on separable Hilbert spaces by introducing a known self-adjoint prior operator

$(B_t,C_t)$4

with known eigenpairs

$(B_t,C_t)$5

Let

$(B_t,C_t)$6

be the orthogonal projection onto the leading $(B_t,C_t)$7 eigenfunctions. Using only forward evaluations $(B_t,C_t)$8, one defines

$(B_t,C_t)$9

The core theorem states that, provided $K^*$0,

$K^*$1

The paper further states that equality occurs in the worst case when $K^*$2 (Boullé et al., 2024).

In concrete settings, $K^*$3 is chosen as a smoothing self-adjoint operator, often a power of the Laplace–Beltrami operator such as $K^*$4 with Dirichlet or Neumann boundary conditions. The eigenfunctions then act as a Fourier basis, and the operator is approximated by sampling the forward action $K^*$5 for $K^*$6. Under the spectral growth estimate $K^*$7, the error bound implies that achieving $K^*$8 requires

$K^*$9

that is, $K^*$0. On a $K^*$1-dimensional domain with $K^*$2, $K^*$3, so $K^*$4.

The principal PDE example is the recovery of Green’s functions of elliptic solution operators. If $K^*$5 is the solution operator for a uniformly elliptic PDE and elliptic regularity gives $K^*$6, then choosing $K^*$7 yields an adjoint-free operator-norm estimate and the sample complexity $K^*$8, where each sample $K^*$9 is a forward PDE solve.

The framework is not universal. Compactness and regularity relative to $W:\mathbb R^p\to\mathbb R^N$0 are essential, and the paper states that absent such prior information, the finite-dimensional lower/upper-bound analysis shows that one cannot hope to recover a general non-normal operator without adjoint queries. This directly qualifies the common misconception that adjoint-free learning is always possible.

4. Adjoint BSDEs in stochastic operator networks

The Stochastic Operator Network extends deterministic operator learning to settings where the operator output itself is random. The starting point is the DeepONet factorization

$W:\mathbb R^p\to\mathbb R^N$1

with a branch net $W:\mathbb R^p\to\mathbb R^N$2 encoding the input function $W:\mathbb R^p\to\mathbb R^N$3 and a trunk net $W:\mathbb R^p\to\mathbb R^N$4 encoding the evaluation location $W:\mathbb R^p\to\mathbb R^N$5. SON replaces the deterministic branch net by the terminal state of an SDE driven by $W:\mathbb R^p\to\mathbb R^N$6, so that each forward pass injects sample-wise Brownian noise whose magnitude is learned (Bausback et al., 10 Jul 2025).

With pseudo-time $W:\mathbb R^p\to\mathbb R^N$7, usually $W:\mathbb R^p\to\mathbb R^N$8, the branch features satisfy

$W:\mathbb R^p\to\mathbb R^N$9

In Euler–Maruyama form,

$W = W_{zpd}\circ E,$0

The terminal features $W = W_{zpd}\circ E,$1 replace the branch representation, and the operator output is formed by the usual branch–trunk inner product.

Training is posed as a stochastic optimal-control problem with terminal loss

$W = W_{zpd}\circ E,$2

Under the Stochastic Maximum Principle, the adjoint processes $W = W_{zpd}\circ E,$3 solve the backward SDE

$W = W_{zpd}\circ E,$4

with terminal condition

$W = W_{zpd}\circ E,$5

The point-wise Hamiltonian is

$W = W_{zpd}\circ E,$6

and for a terminal-only loss $W = W_{zpd}\circ E,$7,

$W = W_{zpd}\circ E,$8

The gradient formula becomes

$W = W_{zpd}\circ E,$9

with

$E$0

Algorithmically, SON performs a forward SDE simulation for the branch net, a standard forward pass for the trunk net, a terminal loss evaluation, a backward BSDE sweep, and parameter updates via Hamiltonian gradients. The backward step uses a sample-wise one-step update,

$E$1

The diffusion parameters are untied trainable weights, often one scalar per layer or a small MLP, so SON learns the pointwise standard deviation of the operator output. The reported complexity per iteration is approximately $E$2 for the SDE plus BSDE and $E$3 for trunk backprop, with wall-time described as comparable to deterministic DeepONet because the adjoint BSDE is solved sample-wise without storing the full Brownian path or Jacobians.

The main conceptual contribution is the reinterpretation of operator-learning backpropagation as stochastic adjoint dynamics. Rather than merely propagating deterministic sensitivities, the method couples mean behavior and uncertainty through drift and diffusion.

5. Continuous adjoint training for reduced-order operators

In data-driven reduced-order modeling, the adjoint method is used to train a reduced operator from trajectory data without estimating time derivatives from noisy measurements. The full-order dynamics are written

$E$4

often with quadratic structure

$E$5

Using a POD basis $E$6, $E$7, one writes $E$8, and the reduced model becomes

$E$9

The parameter vector is

$K^*$00

with $K^*$01 (Liu et al., 12 Jan 2026).

The training objective is a trajectory-based loss,

$K^*$02

Equivalently, with $K^*$03,

$K^*$04

The stated advantages are that this avoids noisy finite-difference estimates of $K^*$05 and provides intrinsic temporal smoothing or regularization through time integration.

The adjoint is obtained from the Lagrangian

$K^*$06

Stationarity with respect to $K^*$07 gives

$K^*$08

Stationarity with respect to $K^*$09 gives the exact gradient

$K^*$10

For the quadratic ansatz,

$K^*$11

The implementation requires one forward reduced-order solve, one backward adjoint solve, and gradient assembly. The paper states the cost per iteration as one forward solve plus one backward adjoint solve plus $K^*$12 assembly. It also describes a warm start from standard operator inference with Tikhonov or TSVD regularization, per-mode weighting for noisy snapshots, and a short-horizon multiple-shooting strategy to avoid exploding gradients over long integration.

The empirical study covers viscous Burgers’ equation, the two-dimensional Fisher–KPP equation, and an advection–diffusion equation. Metrics are reported as test-window relative state error in $K^*$13 scale. The stated findings are that on clean data, adjoint-trained ROM and OpInf deliver similar accuracy, whereas under sparse sampling or additive Gaussian noise the adjoint approach yields lower RSE, more stable roll-outs, and less overfitting than OpInf variants based on second-order or sixth-order finite differences.

6. Limitations, misconceptions, and open directions

Several misconceptions are corrected by these works. One is that the adjoint is always either indispensable or dispensable. The adjoint-free theory shows that a family of non-self-adjoint infinite-dimensional compact operators can be approximated without querying $K^*$14, but only under a prior structure encoded by a self-adjoint operator $K^*$15 and the regularity condition $K^*$16 (Boullé et al., 2024). The same source also states that absent such prior, one cannot hope to recover a general non-normal operator without adjoint queries. This makes the dispute conditional rather than absolute.

A second misconception is that adjoint-based training must be numerically fragile in the presence of noisy data. The reduced-order modeling framework argues the opposite for its setting: by minimizing a trajectory-based loss, it removes the need to estimate time derivatives from noisy measurements and introduces intrinsic temporal regularization through time integration. Its numerical comparisons then report better accuracy and enhanced roll-out stability under reduced temporal snapshot density and additive Gaussian noise (Liu et al., 12 Jan 2026).

A third misconception is that explicit adjoint computation is necessarily more burdensome than the forward operator. For wavelet reconstruction and convolution, exact adjoints are derived with the same asymptotic order as the forward application, and can reuse existing libraries with only small boundary-handling or kernel-flipping modifications (Folberth et al., 2017).

The stochastic setting introduces a different limitation profile. SON learns uncertainty through diffusion parameters and trains through an adjoint BSDE, but its formulation presupposes an SDE-based branch representation, pseudo-time discretization, and a stochastic-control interpretation of training (Bausback et al., 10 Jul 2025). This suggests that the meaning of “adjoint” in operator learning is broad: it may denote a matrix adjoint, a functional-analytic dual action, a stochastic co-state, or a continuous-time adjoint state, depending on the model class.

Open questions are stated explicitly in the adjoint-free analysis. These include the optimal choice of the prior operator $K^*$17 to minimize $K^*$18, and how the adjoint-free scheme integrates with data-driven architectures such as neural operators. The sample-complexity rate $K^*$19 also deteriorates with dimension, reflecting the usual curse of dimensionality. A plausible implication is that future work will continue to alternate between two strategies: designing better priors that reduce dependence on adjoint access, and designing more efficient adjoint solvers when exact dual information remains the most effective route.