AdjointNet Framework

• AdjointNet is a machine learning framework that integrates legacy physics solvers with discrete adjoint methods to enforce conservation and numerical properties up to machine precision.
• It couples neural network surrogates with established simulation codes, enabling fast, high-fidelity parameter estimation and inversion without modifying mature codes.
• The framework supports scalable workflows for uncertainty quantification, active learning, and experimental design in large-scale physical simulations such as fluid dynamics and porous-media flow.

AdjointNet is a machine learning framework that enables strict physics-constrained parameter estimation, inversion, and experimental design by embedding legacy physics-based simulation codes—including their discrete-adjoint solvers—directly into the neural network training loop. This approach is distinct from physics-informed neural networks (PINNs), which softly enforce partial differential equation (PDE) constraints via losses constructed from automatic differentiation. AdjointNet preserves mathematical properties such as consistency, stability, and convergence guarantees of the host physics code throughout the learning process, and accommodates scalable high-fidelity workflows using existing scientific simulation software without code modification (Karra et al., 2021).

1. Motivation and Scope

Conventional PINNs impose balance laws as soft constraints in the loss, e.g., via $R_{\rm PDE} = \|F(u_\theta(x),\theta)\|_{L^2}^2$, requiring auto-differentiation through re-discretized PDEs. This methodology is hampered by the need to rewrite mature scientific codes, the approximate satisfaction of conservation properties (errors stagnate at $10^{-5}$ for many problems), and performance bottlenecks on stiff or large-scale problems. The requirement for high volumes of collocation points makes these approaches infeasible for large domains commonly tackled with advanced codes such as PFLOTRAN, OpenFOAM, or WRF-Hydro.

AdjointNet addresses these gaps by integrating domain-specific (e.g., Fortran, MPI) physics solvers—preserving their numerical rigor—into machine learning–driven parameter estimation tasks. It reuses discrete adjoint solvers already present in many such codes, enforces physics constraints up to machine precision, and avoids the overhead of reformulating complex PDE systems in machine learning frameworks (Karra et al., 2021).

2. Framework Architecture and Mathematical Formulation

AdjointNet is structured to couple a neural network surrogate with a legacy PDE solver and its adjoint. The methodological pipeline comprises:

1. A neural network with trainable parameters $\theta$ that maps coordinates $x$ to proposed physical properties, e.g., $p = \mathrm{NN}_\theta(x)$ (for instance, a permeability field).
2. The forward pass calls the legacy simulation code to solve the system $F(u, p) = 0$ over the domain $\Omega$ for the state variable $u$ under the specified parameters $p$ and boundary/initial conditions, returning $u(x)$.
3. The objective function is $$L(u, \theta) = L_\text{data}(u; u_\text{obs}) + R(\theta)$$, combining data misfit (e.g., $L^2$ norm between observation and model output) and optional regularization.
4. Gradients for network parameter updates are obtained through a combination of autodiff (for $L$ with respect to $u$) and the discrete adjoint from the physics solver (for $u$ with respect to $p$, then back to $\theta$).

The Lagrangian is: $$J(u, \lambda, \theta) = L(u, \theta) + \lambda^\top F(u, p(\theta))$$ with the adjoint (Lagrange multiplier) $\lambda$. The adjoint equation is given by: $$\left(\frac{\partial F}{\partial u}\right)^\top \lambda + \frac{\partial L}{\partial u} = 0$$ and the reduced gradient for the loss is: $$\nabla_\theta \hat{L} = \frac{\partial L}{\partial \theta} + \left(\frac{\partial p}{\partial \theta}\right)^\top \left(\frac{\partial F}{\partial p}\right)^\top \lambda$$ where the term $\frac{\partial p}{\partial \theta}$ is efficiently back-propagated through the neural surrogate ($\mathrm{NN}_\theta$), and $$\left(\frac{\partial F}{\partial p}\right)^\top \lambda$$ is supplied by the embedded adjoint solver.

Algorithmic structure:

1. Propose $p = \mathrm{NN}_\theta(x)$
2. Solve forward model $F(u, p) = 0$ for $u$
3. Compute $L(u, \theta)$ and $\frac{\partial L}{\partial u}$
4. Solve adjoint equation $$(\frac{\partial F}{\partial u})^\top \lambda = -\frac{\partial L}{\partial u}$$
5. Compute full gradient $\nabla_\theta \hat{L}$ as above
6. Update $\theta$

This procedure, which leverages the efficiency of discrete adjoints (roughly the cost of one forward model run, independent of parameter dimension), is central to the scalability and rigor of AdjointNet (Karra et al., 2021).

3. Enforcement of Physics Constraints and Numerical Properties

Every forward evaluation within AdjointNet calls the physics solver, ensuring $F(u, p) = 0$ is enforced pointwise, up to solver tolerance. As a result:

• All conservation laws, e.g., mass, momentum, and energy balances, are honored to machine precision, as dictated by the original numerical discretization.
• The approach inherits stability, convergence, and well-posedness properties of the underlying solver, contingent on the original code satisfying relevant coercivity or inf-sup conditions.
• There is no additional re-discretization or loss of numerical order; global errors remain $O(h^r)$ for a solver of order $r$.
• This contrasts with PINNs, where conservation is only approximate and may degrade with problem size or complexity.

4. Parameter Estimation, Uncertainty Quantification, and Active Learning

AdjointNet is applicable to inverse parameter estimation under observational data subject to physics constraints: $$\min_\theta \|u(\theta) - u_\text{obs}\|^2\quad \text{subject to} \quad F(u(\theta), p(\theta)) = 0$$ providing point estimates through gradient-based optimization. Bayesian and variational extensions are accommodated, e.g., using Laplace approximations (requiring Hessian estimates via second adjoint solves) or MCMC accelerated by adjoint-based gradients. The approach enables active learning and experimental design through efficient calculation of parameter sensitivities $\partial u/\partial\theta$ for expected information gain analyses, guiding optimal sensor placement and sequential data assimilation (Karra et al., 2021).

5. Representative Applications and Performance

AdjointNet has been validated on multiple process models, leveraging direct wrapping of Fortran/C solvers (e.g., PFLOTRAN, finite-difference Navier–Stokes solver):

• Homogeneous porous-media flow: Estimation of permeability $k$, achieving $<1\%$ relative error in $\sim30$ epochs. Data-fit loss reached $1.5 \times 10^{-3}$, with state prediction errors within $0.05$ MPa.
• Sequential assimilation: Addition of observation points produced rapid convergence toward the true parameter value, with diminishing error phases.
• Heterogeneous flow: Two-network architecture produced accurate interface representation and final loss $<10^{-8}$ in $100$ epochs.
• 2D Navier–Stokes (lid-driven cavity): Neural network estimation of viscosity $\nu$ with $2.1\%$ prediction error; rapid loss reduction below $10^{-8}$ in $\sim200$ epochs.

No modification of solver source code was required; training times ranged from tens to a few hundred epochs and total wall times were tractable for desktop or single-node cluster use (Karra et al., 2021).

6. Implementation and Scalability

Key components of implementation:

• Legacy physics codes are accessed via bindings in Python or C (for example, using MPI-enabled PFLOTRAN).
• Neural networks are constructed and optimized with major ML frameworks (TensorFlow, PyTorch), with custom autograd functionality for adjoint hooks.
• Standard optimizers such as Adam or SGD are employed; initialization uses Glorot-Uniform for weights, zero vector for biases.
• For scalability, parallel adjoint solvers ensure that the approach extends naturally to 3D domains and large-scale HPC settings (Karra et al., 2021).

AdjointNet thus represents a methodologically rigorous pathway to integrate established physics simulation software with differentiable machine learning, guaranteeing fidelity to both experimental data and conservation laws at scale. It enables physics-constrained learning, inversion, uncertainty quantification, and experimental design, all while retaining the numerical guarantees of mature simulation codes.