PDE-Aware Optimizer for Control & Inverse Problems

• PDE-aware optimizers are specialized frameworks that embed the structure and constraints of PDEs into the optimization process to tackle control and inverse problems.
• They leverage automatic differentiation and symbolic assembly, as seen in platforms like Firedrake, to derive accurate gradients and Hessians efficiently.
• Advanced block preconditioning techniques ensure robust convergence across varying mesh resolutions and parameters in challenging scientific applications.

A PDE-aware optimizer is a numerical or computational optimization framework that directly integrates the structure, numerical solution, and constraint satisfaction of partial differential equations (PDEs) into its update rules, parameterizations, and solvers. In contrast to black-box or generic optimizers, PDE-aware optimizers exploit the mathematical, variational, and operator-theoretic properties of PDE-constrained problems, enabling efficient and robust solution of both forward and inverse problems in science and engineering. Such optimizers are foundational for optimal control, parameter estimation, design, and scientific machine learning involving PDEs. This entry provides a comprehensive overview of PDE-aware optimizers, focusing on modern algorithmic realizations, automatic differentiation-based assembly, saddle-point system solvers, and advanced preconditioning as exemplified by recent frameworks (Leveque et al., 2024).

1. Mathematical Formulation of PDE-Constrained Optimization

A PDE-aware optimizer targets problems of the form:

• Minimize a cost functional $J(v, u)$, where $v$ is the state and $u$ is the control variable,
• Subject to the PDE constraint $r(v, u) = 0$ over a domain $\Omega$ (possibly with time interval $[0, t_f]$ for instationary problems).

For canonical quadratic tracking with distributed control:

• Stationary case:

$$J(v,u) = \frac12\|v-v_d\|_{L^2(\Omega)}^2 + \frac{\beta}{2}\|u\|_{L^2(\Omega)}^2$$

Subject to $r(v, u) = D v - (u + f) = 0$ in $\Omega$.

• Instationary case:

$$J_t(v,u) = \int_0^{t_f} \left[\frac12\|v(t)-v_d(t)\|^2 + \frac{\beta}{2}\|u(t)\|^2 \right]\,dt$$

Subject to $v$0 in $v$1, with initial and boundary conditions.

First-order optimality conditions are encoded by the Lagrangian,

$v$2

with adjoint variable $v$3 and corresponding Karush–Kuhn–Tucker (KKT) system. In finite-element discretization, the perturbed optimality conditions yield a block-structured system involving mass and stiffness matrices, control regularization, and PDE operators. This methodology extends to non-linear PDEs and time-dependent systems by stacking block structures for all time steps.

2. Automatic Differentiation and Assembly in Firedrake

Contemporary PDE-aware optimizers leverage high-level DSLs and robust AD tools. In the Firedrake platform, problems are posed in the Unified Form Language (UFL), enabling symbolic definition of variational forms for forward PDEs, controls, objective functionals, and constraints. Automatic differentiation is then used to derive:

• The gradient of $v$4 w.r.t. $v$5 and $v$6,
• The adjoint operator $v$7,
• Directional and second derivatives (Hessian blocks) of the Lagrangian.

AD operates at the symbolic layer: given the weak form, the system automatically generates Jacobians and Hessians, which are then compiled and assembled into PETSc matrix objects for high-performance linear algebra. This encapsulates the full Newton linearization and enables "all-at-once" KKT solves for stationary and time-dependent control problems (Leveque et al., 2024).

3. Block-Structured Solvers and Preconditioning Strategies

Each Newton-KKT iteration requires the solution of large, indefinite saddle-point systems whose structure is dictated by the PDE-constrained optimization formulation. The canonical block system for the induced linearized KKT conditions is: $v$8 with $v$9, $u$0, $u$1 assembled from mass, stiffness, and control terms.

Efficient solution relies on tailored block preconditioners:

• The negative-one-sided (block-triangular) preconditioner,
• The "matching strategy": construct approximations $u$2 and $u$3 (Schur complement) so that

$u$4

with $u$5 (e.g., Jacobi or Chebyshev smoothing) and $u$6, where $u$7 is chosen so that $u$8. For $u$9-control, $r(v, u) = 0$0 suggests $r(v, u) = 0$1.

• For incompressible flows (e.g., Stokes, Navier–Stokes), more complex block-preconditioners are needed, such as the block pressure–convection–diffusion preconditioner. These leverage further PDE structure such as divergence and pressure coupling.

Preconditioning is crucial for mesh and parameter-independent convergence of Krylov subspace methods (e.g., GMRES, FGMRES), as verified by nearly constant iteration counts in large-scale tests (Leveque et al., 2024).

4. Software Design and User Workflow

PDE-aware optimizers can be specified with a compact, high-level user interface. In the Firedrake-based framework, typical user code for a heat control problem is limited to defining:

• The mesh and function space,
• The forward operator (e.g., diffusion form),
• Desired state, forcing, and boundary conditions as functions,
• Control configuration (space, regularization parameter, time discretization),
• Solver parameters (e.g., GMRES, custom Python preconditioner).

The backend automatically constructs the weak form, performs all symbolic differentiation, assembles the entire KKT system, and invokes PETSc field-split or custom preconditioners, with all linear algebra and Newton iteration details abstracted away. The infrastructure supports both stationary and instationary (time-dependent) PDEs with high efficiency and minimal user effort (Leveque et al., 2024).

5. Representative Numerical Results and Performance

Quantitative evaluations demonstrate robust mesh and parameter scalability:

• Stationary Poisson control: For mesh refinement level $r(v, u) = 0$2 (e.g., $r(v, u) = 0$3 degrees of freedom), number of GMRES iterations remains $r(v, u) = 0$4 for all regularization $r(v, u) = 0$5, with linear CPU time scaling and, e.g., $r(v, u) = 0$6, $r(v, u) = 0$7 yielding $r(v, u) = 0$8 GMRES iterations in 47 s.
• Instationary Navier–Stokes control: For $r(v, u) = 0$9 and viscosity $\Omega$0, Picard-outer/FGMRES-inner iterations are $\Omega$1 (Picard), $\Omega$2 (FGMRES), with convergence robust to mesh resolution, regularization, and Reynolds number.

These properties are attributed to the integration of all-at-once variational KKT structure, PDE-exact AD, and structure-aware preconditioning, yielding efficient, scalable, and parameter-robust solution even for high-dimensional, time-dependent flow control problems (Leveque et al., 2024).

6. Broader Context and Significance

The PDE-aware optimizer, as instantiated in the all-at-once, AD-augmented, block-preconditioned framework above, represents a significant advance over generic optimizers. Rigorous exploitation of PDE variational structure, high-level symbolic differentiation, and block-aware solvers enable:

• Drastically reduced user code for complex control and optimization problems,
• Automated and error-free derivative calculations,
• Robust handling of large and ill-conditioned KKT systems,
• Nearly optimal, parameter-independent Krylov convergence.

By bridging the gap between numeric simulation, variational modeling, and large-scale optimization, PDE-aware optimizers are central to modern computational optimal control, scientific machine learning, and inverse problem domains where PDE constraints are intrinsic (Leveque et al., 2024).