Adjoint-Based Optimization Techniques

• Adjoint-based optimization techniques are methods that compute gradients for high-dimensional control variables in PDE/ODE-constrained systems using a dual formulation.
• They leverage forward and adjoint solves to decouple gradient cost from parameter dimension, significantly reducing computational expense in complex simulations.
• These techniques underpin optimal control, shape and topology optimization, and have broad applications in fluid mechanics, electromagnetics, and machine learning-based models.

Adjoint-based optimization techniques provide a mathematically rigorous and computationally efficient framework for optimizing functionals that depend on solutions to PDEs, ODEs, or other large-scale physical simulation models. By leveraging the adjoint (dual) problem, sensitivities with respect to high-dimensional parameter spaces can be computed at a cost that is independent of the number of control variables. These methods are foundational in optimal control, shape and topology optimization, and design under constraints arising from complex physical systems in fluid mechanics, electromagnetics, photonics, structural mechanics, climate modeling, fusion energy, and beyond.

1. Mathematical Foundations and Problem Formulation

Adjoint-based optimization is anchored in PDE-constrained or system-constrained optimization, where the goal is to minimize an objective functional $J(p)$ with respect to parameters $p$ subject to governing equations $F(u,p)=0$ for the state variable $u$. Typical formulations include:

• Lagrangian approach: Introduce Lagrange multipliers (adjoint states, $\lambda$), forming

$$\mathcal{L}(u, p, \lambda) = J(u, p) + \langle \lambda, F(u, p) \rangle,$$

and require stationarity with respect to $u$ and $\lambda$.

• Continuous adjoint equations: The adjoint equation results from the stationarity condition $\delta\mathcal{L}/\delta u=0$. For differentiable $F$, the adjoint PDE is

$$(\partial F / \partial u)^T \lambda + \partial J / \partial u = 0,$$

with appropriate terminal or boundary conditions, returning sensitivities in a backward-in-time or dual variable manner (Paul, 2020).

• Gradient computation: The reduced gradient

$$\nabla_p J = \partial J/\partial p + \lambda^T (\partial F/\partial p)$$

can be evaluated at the cost of a forward and one adjoint solve—regardless of the dimension of $p$ (Zahr et al., 2015, Paul et al., 2019).

• Discrete adjoint: For fully discrete time-stepping or monolithic algebraic systems, the adjoint is obtained by transposing the Jacobian of the update mapping (often termed the "discrete adjoint" approach) (Huang et al., 2018, Zahr et al., 2015).

2. Core Algorithms and Computational Schemes

Efficient adjoint-based optimization algorithms share several fundamental principles:

• Duality and backward propagation: The adjoint state propagates sensitivities from objectives to controls via backward integration in time or analogous algebraic operations for steady-state or statically discretized systems (Zahr et al., 2015, Huang et al., 2018).
• Solver-consistent discretization: Quantities of interest (QoIs) and their gradients must be discretized using the same numerical scheme as the forward problem to ensure consistency, especially for high-order time and space schemes (Zahr et al., 2015, Huang et al., 2018).
• Parameter update loop: At each optimization iteration, a (quasi-)Newton, BFGS, or similar gradient-based update is performed using the adjoint-computed gradient, with constraints handled via Lagrange multipliers or interior-point strategies (Zahr et al., 2015, Huang et al., 2018).
• High-dimensionality scalability: The cost of gradient evaluation via the adjoint is independent of the parameter dimension, yielding order-of-magnitude speed-ups in high-dimensional shape, topology, or control problems (Paul et al., 2019, Paul, 2020, Xu et al., 2020).

Step	Forward Solve	Adjoint Solve	Gradient Assembly
PDE/ODE Integration	$F(u,p)=0$	$\lambda$ via adjoint PDE	Combine $\lambda$ and $\partial F/\partial p$, $\partial J/\partial p$

3. Extensions: State Constraints, Structured Manifolds, and Reduced Order Models

Adjoint techniques have been generalized to address additional complexities:

• State constraints: Projected adjoint-based methods enforce general state constraints (e.g., energy conservation, bounded outputs) by projecting the unconstrained gradient onto the tangent space of the constraint manifold via solution of a secondary adjoint PDE with constraint-derived right-hand side (Matharu et al., 2023).
• Manifold constraints via generative models: Recent approaches constrain design parameters to a learned manifold, such as a diffusion-model-generated set of admissible shapes. The adjoint gradient is propagated through the generative network by chain rule and automatic differentiation (Chen et al., 31 Jul 2025):

$\nabla_z J = (\partial x/\partial z)^T \nabla_x J,$

where $x=G_\theta(z)$ and $G_\theta$ is the generative map. This constrains optimization to physically meaningful or manufacturable subspaces.

• Reduced-order modeling (ROM): Adjoint-based optimization within reduced-order frameworks employs projection to low-dimensional bases for both primal and adjoint fields, with custom snapshot strategies (e.g., modified gradient descent adjoint basis collection) to maintain gradient accuracy (Hawkins et al., 2024).

4. Applications and Domain-Specific Methodologies

4.1 Electromagnetics and Photonics

Adjoint-based inverse design is central in nanophotonics, enabling the synthesis of complex dielectric permittivity distributions for custom device functions. The adjoint electromagnetic fields provide gradients of figures of merit (transmission, scattering, etc.) with remarkable efficiency (Yeung et al., 2021). The equation for the gradient of a figure-of-merit (FOM) $F[\epsilon]$ with respect to spatially varying permittivity $\epsilon(r)$ is typically

$$\frac{\partial FOM}{\partial \epsilon(r)} = -\operatorname{Re}\{E_{adj}(r) \cdot E_{dir}(r)\},$$

requiring only two simulations per gradient.

Integrating automated machine learning and explainability (XAI), adjoint outputs are further interpreted using surrogate CNNs and Shapley-feature attributions, enabling explanation-based re-optimization to escape local minima (Yeung et al., 2021).

4.2 Fluid and Plasma Physics

In high-Reynolds and multiphysics or multiphase flows, adjoint-based optimization is widely used for shape, topology, and control parameter design. Techniques must account for moving or deforming geometries (using ALE mappings), high-order schemes (DG, RK), and may employ "dual consistency"—ensuring adjoint equations reflect model regularizations such as Cahn-Hilliard in multiphase CFD (Zahr et al., 2015, Kühl et al., 2022).

For stellarator optimization in fusion engineering, adjoint methods compute the gradient of quantities depending on linear PDEs (e.g., drift-kinetic equations for neoclassical transport, coil sensitivity) with 2–4 solves rather than hundreds (Paul et al., 2019, Paul, 2020).

Domain	Governing Equation	Key Feature of Adjoint Technique
Photonics EM Design	Maxwell's Equations	Field-based shape gradient in 2 solves
Fluid/FSI	Compressible NS, ALE-DG	High-order, partitioned adjoint (IMEX)
Neoclassical Fusion	Drift-kinetic eq.	Linear system adjoint; shape sensitivity

4.3 Large-Scale Dynamic Systems

Memory limitations in adjoint-based sensitivity analysis for dynamic PDEs are addressed by algorithms that combine checkpointing, data compression, or superposition principles (for self-adjoint problems), reducing storage from $O(N \times M)$ to $O(M)$ (number of gridpoints) (Herrmann et al., 19 Sep 2025, Kukreja et al., 2018). These approaches enable billion-parameter optimization on GPU architectures.

4.4 Machine-Learned and Gray-Box Models

Where analytical forms are partially or wholly unknown ("gray-box" settings), adjoint gradients can be recovered by first inferring a "twin model" matching the space-time solution of the target system, then applying adjoint analysis to the surrogate (Chen et al., 2016, Chen et al., 2015). This approach restores the $\mathcal{O}(1)$ scaling of gradient cost in the number of controls, even for proprietary or black-box simulators.

In forward models such as aerodynamic shape optimization, machine learning can also efficiently predict adjoint variables based on local flow features, allowing gradient-based optimizers to operate at half the classical runtime cost with negligible degradation in final design (Xu et al., 2020).

5. Algorithmic Advances and Best Practices

Recent developments emphasize:

• Fully discrete adjoint consistency: Deriving and discretizing adjoint equations using exactly the same discretization (mesh, time stepping, basis functions) as the forward problem ensures "discrete consistency" and rapid, robust optimization convergence (Zahr et al., 2015, Huang et al., 2018).
• Automatic differentiation (AD): Adjoint solvers in modern codes are increasingly implemented using reverse-mode AD, both for classical PDEs and when backpropagating through machine-learning components, as in manifold-constrained shape optimization (Chen et al., 31 Jul 2025).
• Constraint enforcement: State constraints, geometry constraints, or manifold enforcement should be addressed at the adjoint level, often via projected gradients or secondary adjoint solves, to maintain optimizer stability and feasibility (Matharu et al., 2023, Chen et al., 31 Jul 2025).

6. Limitations and Challenges

Known challenges include:

• Non-convexity and local minima: Adjoint-based optimizers are susceptible to local minima, particularly in highly nonconvex or structurally complex design spaces. Hybridization with surrogate modeling (AutoML, XAI), multi-start, or explanation-based re-initialization are effective in overcoming these situations (Yeung et al., 2021).
• Memory and computational bottlenecks: For large dynamic problems, storing forward solutions for adjoint evaluation can be prohibitive; superposition-based adjoints and combined checkpointing/compression are necessary for petascale problems (Kukreja et al., 2018, Herrmann et al., 19 Sep 2025).
• Non-holomorphic and complex-valued PDEs: Adjoint approaches for non-holomorphic cost or constraints require CR-calculus and generalized Lagrangian/adjoint systems accounting for Wirtinger derivatives. The generalized adjoint system becomes a block system in both $z$ and $\overline{z}$ (Zheng et al., 19 Jan 2026):

$$\begin{bmatrix} C_z^\dagger & C_{\overline{z}}^\dagger \ C_{\overline{z}}^T & C_z^T \end{bmatrix} \begin{pmatrix} \lambda \ \overline{\lambda} \end{pmatrix} = \begin{pmatrix} \nabla_z J \ \nabla_{\overline{z}} J \end{pmatrix}$$

enabling optimization in complex settings.

• Accuracy of surrogate adjoints: Data-driven adjoint surrogates (e.g., DNN-based) can accelerate optimization but introduce a trade-off in gradient accuracy; careful validation is necessary to prevent optimizer degradation, especially in highly sensitive design scenarios (Xu et al., 2020).

7. Impact and Future Directions

Adjoint-based optimization techniques have fundamentally shifted the landscape of high-dimensional, PDE-constrained, and physics-driven design. Their ability to decouple gradient cost from the number of parameters enables practical optimization and control in domains previously dominated by brute-force or finite-difference approaches. Current trends point towards:

• Integration with machine learning for both surrogate modeling and to encode prior knowledge or manifold constraints, further accelerating convergence and robustness (Yeung et al., 2021, Chen et al., 31 Jul 2025).
• Scalable implementations on exascale and GPU architectures via memory-efficient adjoint algorithms and parallel partitioned solvers (Herrmann et al., 19 Sep 2025, Huang et al., 2018).
• Generalization to complex variables and non-holomorphic functionals, accommodating advance needs in electromagnetics and signal processing (Zheng et al., 19 Jan 2026).
• Reduced-order and model reduction techniques ensuring adjoint efficiency and gradient accuracy in real-time and embedded control settings (Hawkins et al., 2024).

These advances collectively extend adjoint-based optimization as a versatile, foundational tool for scientific computing, engineering design, and data-driven inverse problems.