Iterative Pontryagin Maximum Principle (IPMP)

• IPMP is a control methodology that iteratively enforces Pontryagin optimality through forward state propagation, backward costate calculation, and Hamiltonian-driven control updates.
• It employs a variational framework applicable across diverse fields such as quantum control, PDE control, and CNN training, each adapting the fundamental IPMP structure.
• The technique refines control laws using switching functions or gradient steps, addressing both smooth and nonsmooth optimization problems with practical convergence considerations.

Iterative Pontryagin Maximum Principle (IPMP) denotes a class of indirect optimal-control procedures that repeatedly enforce Pontryagin optimality conditions by alternating a forward propagation of the state, a backward propagation of the costate, and a control improvement step obtained from Hamiltonian maximization or minimization. In recent work, this template appears explicitly under the name IPMP for filtered radical-pair quantum control and in closely related forms for quantum state transfer, nonsmooth PDE control, learned-dynamics control, convolutional neural network training, and policy improvement in rough environments (Abdulla et al., 3 Aug 2025, Dehaghani et al., 2022, Wachsmuth, 2024, Gu et al., 2022, Hofmann et al., 15 Apr 2025, Ashkarian et al., 8 Jan 2026).

1. Variational framework and optimality system

At its most abstract, IPMP operates on an optimal-control problem with dynamics

$\dot{x}(t)=f(x(t),u(t),t), \qquad x(0)=x_0,$

running and terminal costs $L(x,u,t)$ and $\phi(x(T))$, and Hamiltonian

$$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$

The corresponding costate satisfies

$$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$

while optimal controls satisfy a pointwise Hamiltonian extremality condition, written either as $\arg\min$ or $\arg\max$ depending on sign convention (Gu et al., 2022).

The same structure persists in discrete time. For

$$x_{k+1}=f(x_k,u_k), \qquad J=\Phi(x_T)+\sum_{k=0}^{T-1}L(x_k,u_k),$$

the discrete Hamiltonian is

$H_k=L(x_k,u_k)+\lambda_{k+1}^\top f(x_k,u_k),$

the backward recursion is

$$\lambda_T=\frac{\partial \Phi}{\partial x}(x_T), \qquad \lambda_k=\frac{\partial L}{\partial x}(x_k,u_k)+\left(\frac{\partial f}{\partial x}(x_k,u_k)\right)^\top \lambda_{k+1},$$

and stationarity is expressed by $L(x,u,t)$0 or an equivalent constrained extremality condition (Gu et al., 2022).

Problem-specific realizations of this framework vary substantially. In radical-pair coherent spin control, the state is a family of Schrödinger wavefunctions $L(x,u,t)$1, the costate $L(x,u,t)$2 solves an adjoint Schrödinger equation backward in time, and the control acts through a filtered electromagnetic field $L(x,u,t)$3 satisfying $L(x,u,t)$4 (Abdulla et al., 3 Aug 2025). In quantum state transfer, the state is a density operator $L(x,u,t)$5, the dynamics follow the Liouville–von Neumann equation, and the costate evolves under the adjoint commutator dynamics (Dehaghani et al., 2022). In elliptic PDE control, the state and adjoint belong to $L(x,u,t)$6 and solve Poisson equations (Wachsmuth, 2024). In CNN training, the layer index becomes the time variable of a discrete-time control system, with trainable parameters as controls and backpropagated adjoints as costates (Hofmann et al., 15 Apr 2025). In rough environments, both state and adjoint are rough differential equations, and the controls are relaxed, measure-valued policies (Ashkarian et al., 8 Jan 2026).

2. Canonical iterative architecture

Across these settings, the recurring IPMP pattern is a forward–backward sweep with an explicit control-improvement step.

1. Initialization: start from a feasible control sequence or policy. Typical choices include a constant-in-time control within the admissible set, a random initialization, or a warm start near a constraint vertex (Abdulla et al., 3 Aug 2025, Gu et al., 2022).
2. Forward solve: propagate the state under the current control. Examples include Schrödinger propagation for each triplet-born initial state, density-matrix propagation on a uniform grid, Poisson solves for elliptic PDEs, rollout under a learned neural surrogate, or rough-path integration for relaxed controls (Abdulla et al., 3 Aug 2025, Dehaghani et al., 2022, Wachsmuth, 2024, Gu et al., 2022, Ashkarian et al., 8 Jan 2026).
3. Backward solve: propagate the costate from terminal or transversality data. In quantum problems this is an adjoint Schrödinger or commutator equation; in PDEs it is an adjoint Poisson problem; in deep learning it is a layerwise adjoint recursion; in rough environments it is a backward rough differential equation (Abdulla et al., 3 Aug 2025, Dehaghani et al., 2022, Wachsmuth, 2024, Hofmann et al., 15 Apr 2025, Ashkarian et al., 8 Jan 2026).
4. Hamiltonian evaluation: compute switching functions, Hamiltonian gradients, or pointwise minimizers/maximizers. This may use explicit commutators, measurable selections of $L(x,u,t)$7, automatic differentiation through a neural surrogate, or an infinitesimal $L(x,u,t)$8-function identified with a Hamiltonian difference (Dehaghani et al., 2022, Wachsmuth, 2024, Gu et al., 2022, Ashkarian et al., 8 Jan 2026).
5. Control improvement: update the control by enforcing the PMP condition. Depending on the model, this step can be bang-bang, projected-gradient, measurable-patch replacement with Armijo globalization, augmented-Hamiltonian maximization, hard-thresholding, or Gibbs/softmax policy improvement (Abdulla et al., 3 Aug 2025, Wachsmuth, 2024, Hofmann et al., 15 Apr 2025, Ashkarian et al., 8 Jan 2026).
6. Stopping or globalization: terminate when successive controls coincide up to tolerance, when a PMP residual is sufficiently small, or when changes in parameters or objective values fall below prescribed thresholds. Some variants embed line search or sufficient-decrease conditions to stabilize nonconvex or nonsmooth updates (Dehaghani et al., 2022, Wachsmuth, 2024, Hofmann et al., 15 Apr 2025).

This suggests that IPMP is best understood as a structural template rather than a single fixed algorithm.

3. Control laws, switching functions, and structural regimes

A central feature of IPMP is that the control update is dictated by the local structure of the Hamiltonian.

In box-constrained problems with Hamiltonian linear in the control and no running control penalty, the update is typically bang-bang. For the filtered radical-pair model, the switching components are

$L(x,u,t)$9

and the optimal control is

$\phi(x(T))$0

The paper proves the bang-bang structure from linearity of the Hamiltonian in $\phi(x(T))$1, convexity and compactness of the admissible box, and non-degeneracy of state and costate (Abdulla et al., 3 Aug 2025). The same pattern appears in finite-dimensional quantum state transfer, where box-constrained controls satisfy

$\phi(x(T))$2

with switching functions

$\phi(x(T))$3

(Dehaghani et al., 2022).

When quadratic regularization is introduced, the update becomes an interior projected law rather than a pure switching law. In the density-operator setting, adding

$\phi(x(T))$4

yields

$\phi(x(T))$5

projected onto box constraints if present (Dehaghani et al., 2022). In the learned-dynamics setting of Neural-PMP, the control update is explicitly a gradient step on the Hamiltonian stationarity residual,

$\phi(x(T))$6

followed by projection onto admissible bounds (Gu et al., 2022).

Nonsmooth and nonconvex problems require different update mechanisms. For the elliptic PDE problem with discrete-valued control cost, the pointwise PMP reads

$\phi(x(T))$7

and for

$\phi(x(T))$8

the resulting rule is a rounding-and-clipping law (Wachsmuth, 2024). In sparse CNN training, the regularizer

$\phi(x(T))$9

is handled by layerwise approximate maximization of an augmented Hamiltonian, producing hard-thresholding updates in the $$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$0 case (Hofmann et al., 15 Apr 2025).

In relaxed rough-control problems with entropy regularization, the update is neither bang-bang nor purely deterministic. The Hamiltonian includes an entropy term, and the optimal policy takes Gibbs form,

$$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$1

or, in the policy-improvement iteration,

$$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$2

normalized over $$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$3 (Ashkarian et al., 8 Jan 2026).

4. Major instantiations

The modern literature uses IPMP across qualitatively different state spaces and admissible sets.

Setting	State/costate carrier	Control-improvement mechanism
Radical-pair quantum control	Schrödinger state and adjoint in Hilbert space	Switching-function sign under filtered field (Abdulla et al., 3 Aug 2025)
Quantum state transfer	Density operator and adjoint commutator dynamics	Bang-bang or projected regularized update (Dehaghani et al., 2022)
Nonsmooth elliptic PDE control	$$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$4 state and adjoint	Pointwise $$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$5 plus Armijo patch update (Wachsmuth, 2024)
Unknown-dynamics control	Learned discrete surrogate and backward costates	Hamiltonian-gradient step with projection (Gu et al., 2022)
CNN training	Layerwise forward/backward sweeps	Augmented-Hamiltonian maximization with adaptive $$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$6 (Hofmann et al., 15 Apr 2025)
Rough-environment control	Forward and backward rough differential equations	Gibbs update driven by $$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$7-function (Ashkarian et al., 8 Jan 2026)

These examples show that IPMP is not confined to smooth finite-dimensional ODE control. It extends to Hilbert-space quantum systems with filtered actuation, matrix-valued dynamics, linear elliptic PDEs with discontinuous or discrete control costs, discrete-time systems with unknown dynamics learned by neural networks, deep network training viewed as an optimal-control problem over layers, and relaxed-control rough differential equations (Abdulla et al., 3 Aug 2025, Wachsmuth, 2024, Gu et al., 2022, Hofmann et al., 15 Apr 2025, Ashkarian et al., 8 Jan 2026).

They also show that the “state” propagated forward need not be a physical trajectory in the classical sense. It can be a density matrix, a PDE solution, a batch of neural activations, or a rough-path state process. A plausible implication is that IPMP is better classified by its variational mechanism—forward state, backward adjoint, Hamiltonian-based policy update—than by any particular application domain.

5. Descent, convergence, and optimality guarantees

Theoretical guarantees differ markedly across IPMP variants.

In the radical-pair Hilbert-space formulation, Fréchet differentiability of the objective in $$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$8 is proved, the Pontryagin Maximum Principle in Hilbert space is established, and the bang-bang structure of the optimal control is derived. The paper further reports numerical simulations based on IPMP and the gradient projection method in Hilbert spaces, for which “the convergence, stability and the regularization effect are demonstrated” (Abdulla et al., 3 Aug 2025).

For nonsmooth elliptic PDE control, the analysis is stronger. The method defines a pointwise residual

$$H(x,u,\lambda,t)=L(x,u,t)+\lambda(t)^\top f(x,u,t).$$9

and aggregate discrepancy

$$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$0

Under compact $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$1, either the algorithm stops finitely with $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$2, or $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$3. Moreover, the resulting sequence is a minimizing sequence for $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$4, and if the PMP inequality holds pointwise, it is also sufficient for global optimality in that setting (Wachsmuth, 2024).

The bSQH algorithm for CNN training adds a quadratic augmentation to the Hamiltonian and proves a sufficient-decrease inequality. In full-batch mode, if the sufficient decrease condition holds and the empirical loss is bounded below, then $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$5 is monotonically decreasing and $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$6. Under additional assumptions, accumulation points are Clarke-stationary, including in the non-smooth $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$7 case (Hofmann et al., 15 Apr 2025).

Neural-PMP provides a different form of guarantee. The paper states that

$$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$8

so the Hamiltonian-gradient update coincides with gradient descent on the full objective. If $$\dot{\lambda}(t)=-\frac{\partial H}{\partial x}(x,u,\lambda,t), \qquad \lambda(T)=\frac{\partial \phi}{\partial x}(x(T)),$$9 is differentiable, strictly convex in controls, and $\arg\min$0 is $\arg\min$1-Lipschitz, then gradient descent with $\arg\min$2 converges globally; in nonconvex cases, convergence is to local minima (Gu et al., 2022).

By contrast, the density-operator quantum-state-transfer paper explicitly states that it “does not derive monotonicity or convergence guarantees”; the proposed scheme is a multiple-shooting variant of a forward–backward sweep, and regularization is suggested in practice for singular behavior (Dehaghani et al., 2022). In rough environments, the entropic policy-improvement theorem gives monotonic value improvement,

$\arg\min$3

for the Gibbs-updated policy, but full convergence results are left open (Ashkarian et al., 8 Jan 2026).

A common misconception is therefore that IPMP inherently guarantees monotone descent or convergence. The literature shows instead that such properties depend on additional devices: line search on measurable patches, quadratic augmentation, convexity and Lipschitz assumptions, or entropy-based policy-improvement arguments.

6. Relation to adjacent methods and principal limitations

IPMP overlaps with, but is not identical to, several neighboring algorithmic families. In quantum control, GRAPE uses discrete-time gradients and line search, while Krotov uses implicit updates that can guarantee monotonic objective increase for suitable functionals; PMP/IPMP instead derives switching functions and pointwise Hamiltonian maximization rules, often yielding bang-bang structure under hard bounds (Dehaghani et al., 2022). In nonsmooth PDE control, the paper contrasts its maximum-principle-based descent algorithm with successive approximations, the min-$\arg\min$4 strategy, proximal or semi-smooth Newton methods, and trust-region steepest descent for binary controls (Wachsmuth, 2024). In learned-dynamics control, Neural-PMP is compared with PPO, RS-MPC, and linearized MPC, emphasizing that the algorithm is an indirect forward–backward sweep rather than a model-free RL method (Gu et al., 2022). In CNN training, bSQH is positioned against gradient-based training such as SGD/Adam and against MSA-style PMP training, with the augmented Hamiltonian introduced to stabilize non-smooth $\arg\min$5-regularized updates (Hofmann et al., 15 Apr 2025). In rough environments, the closest analogue is policy iteration with entropic regularization, but the update is derived from a rough-path PMP and an infinitesimal $\arg\min$6-function rather than from classical Markovian HJB theory (Ashkarian et al., 8 Jan 2026).

Several limitations recur. Indirect methods remain sensitive to initialization and to local optima, particularly in high-dimensional or long-horizon problems (Gu et al., 2022). Singular arcs may arise when switching functions vanish, and regularization or second-order conditions may be required for robust numerics (Dehaghani et al., 2022, Abdulla et al., 3 Aug 2025). Learned-dynamics variants inherit model-error sensitivity because the Hamiltonian gradient depends on $\arg\min$7 and $\arg\min$8 (Gu et al., 2022). Rough-environment formulations require compactness, regularity, and pathwise anticipative analysis, and extensions below roughness threshold $\arg\min$9 are left for future work (Ashkarian et al., 8 Jan 2026). CNN-oriented IPMP requires control-affine reformulations or sufficiently large augmentation parameters to make layerwise Hamiltonian maximization tractable (Hofmann et al., 15 Apr 2025).

Taken together, these results indicate that IPMP is not a single solver but a variational family whose concrete realization depends on state geometry, admissible-control structure, regularization, and the availability of globalization mechanisms. Its unifying principle is the iterative enforcement of Pontryagin extremality through repeated state propagation, adjoint propagation, and Hamiltonian-based control improvement.