Discrete Pontryagin Maximum Principle

• Discrete PMP is a foundational method that establishes necessary optimality conditions for discrete-time control systems by introducing adjoint variables and Hamiltonian maximization.
• It extends classical control techniques to systems evolving on manifolds and Lie groups while accommodating both pointwise and global constraints.
• It enables robust, geometric, and numerical solutions, influencing applications in quantum control, robust optimization, and neural network training.

The discrete Pontryagin Maximum Principle (PMP) is a cornerstone in finite-horizon optimal control theory for systems evolving in discrete time. It delivers first-order necessary conditions for optimality, generalizing the variational extremality of an optimal trajectory by introducing adjoint (costate) variables and enforcing a maximization (or saddle-point, in minimax settings) condition on an appropriate Hamiltonian function. The discrete PMP extends to systems on smooth manifolds, matrix Lie groups, and under complex constraints, including pointwise state, control, and global (e.g., frequency or rate) restrictions. This article surveys the mathematical structure, derivations, and representative applications of the discrete PMP, consolidating contemporary research spanning geometric, robust, and structure-preserving perspectives.

1. General Formulation and Problem Classes

The discrete PMP addresses optimal control problems over a finite horizon, $k=0,1,\dots,N$, for systems with state evolution

$$x_{k+1} = f_k(x_k, u_k), \quad x_0 \textrm{ given},$$

and cost functionals of the form

$$J(\{x_k, u_k\}) = \Phi(x_N) + \sum_{k=0}^{N-1} L_k(x_k, u_k),$$

subject to

• pointwise state constraints: $x_k \in S_k \subset M$, often expressed via smooth inequalities $G_k(x_k) \le 0$,
• pointwise control constraints: $u_k \in U_k \subset \mathbb{R}^m$,
• and global constraints, such as spectral (frequency) or rate limitations: typically linear, e.g. $\sum_{k=0}^{N-1} F_k u_k = 0$ for frequency support constraints (K et al., 2018, Paruchuri et al., 2017, Kotpalliwar et al., 2018, Ganguly et al., 2023).

The system may evolve on a finite-dimensional manifold $M$ or on a matrix Lie group $G$ equipped with a Lie algebra $\mathfrak{g}$, with dynamics expressed accordingly. The data $(f_k, L_k, \Phi, G_k)$ are typically assumed $C^1$, though various relaxations (e.g., Gâteaux differentiability) are admissible under weaker regularity assumptions (Blot et al., 2016).

2. Discrete Hamiltonian, Adjoint Equations, and PMP Conditions

Stage Hamiltonian: For state $x_k \in M$, costate $p_{k+1} \in T^*_{x_{k+1}}M$, and control $u_k \in U_k$,

$$H_k(x_k, p_{k+1}, u_k) = \langle p_{k+1}, f_k(x_k, u_k) \rangle - L_k(x_k, u_k)$$

or, on a Lie group,

$$H_k(q_k, x_k, \lambda_{k+1}, p_{k+1}, u_k) = \langle \lambda_{k+1}, \exp^{-1}(\cdot) \rangle_{\mathfrak{g}^*} + \langle p_{k+1}, f(q_k, x_k, u_k) \rangle + \ell(q_k, x_k, u_k)$$

(Phogat et al., 2016, Kotpalliwar et al., 2018).

Costate Recursions: The optimal trajectory $\{ (\bar{x}_k, \bar{u}_k) \}$ admits (possibly non-vanishing) multipliers:

• abnormality multiplier $\lambda \ge 0$,
• costates $\{p_k \}$ (backward recursions),
• constraint multipliers for state ($\mu_k \ge 0$), global constraints ($\nu$ or $\gamma$).

The adjoint equations are: $$p_k = \partial_x H_k(\bar{x}_k, p_{k+1}, \bar{u}_k) + (DG_k(\bar{x}_k))^* \mu_k,$$ with the backward boundary at $k=N$ involving $-\lambda d\Phi(\bar{x}_N)$ and possible additions from active constraints (K et al., 2018, Kipka et al., 2017).

Stationarity/Maximization Condition: At each $k$,

$$\left\langle \partial_u H_k(\bar{x}_k, p_{k+1}, \bar{u}_k) + F_k^T \nu,~ u - \bar{u}_k \right\rangle \le 0$$

for all $u$ in a local tent of $U_k$ at $\bar{u}_k$ (the tangent cone; for convex $U_k$, equivalent to a global maximum),

$$\bar{u}_k = \arg\max_{u \in U_k} H_k(\bar{x}_k, p_{k+1}, u) - \langle \nu, F_k u \rangle$$

(K et al., 2018, Paruchuri et al., 2017, Kotpalliwar et al., 2018).

Complementary Slackness: For $G_k(x_k) \le 0$,

$\mu_k^i\, G_k^i(\bar{x}_k) = 0,~ \mu_k^i \ge 0$

for all $i$, enforcing activity only at binding constraints.

Nontriviality: $(\lambda, \nu, \{\mu_k\})$ do not vanish simultaneously.

3. Frequency and Rate Constraints

Frequency Constraints

Global frequency constraints restrict the DFT spectrum of $u_k$: $$\widehat{u}^{(j)}_\ell = \sum_{k=0}^{N-1} u_k^{(j)} e^{-2\pi i \ell k / N},~ \operatorname{supp}~\widehat{u}^{(j)} \subset W^{(j)}$$ For linear enforcement, this takes the form $\sum_{k=0}^{N-1} F_k u_k = 0$, leading to a Lagrange multiplier $\nu$ in Hamiltonian maximization (K et al., 2018, Paruchuri et al., 2017, Kotpalliwar et al., 2018).

Rate Constraints

A rate constraint imposes bounds on increments, $u_{k+1} - u_k \in V$ (compact/convex $V$): $$\lambda_k \in N_V(u_{k+1} - u_k),~ \langle \lambda_k, u_{k+1} - u_k \rangle = 0$$ and modifies the stagewise Hamiltonian by rate-multiplier terms $\langle \lambda_{k-1} - \lambda_k, u_k \rangle$. The stationarity and costate recursions must accommodate these, yielding coupled backward equations for both $p_k$ and $\lambda_k$ (Ganguly et al., 2023).

4. Geometric and Structure-Preserving Extensions

The geometric discrete-time PMP extends to nonlinear controlled systems on smooth manifolds or matrix Lie groups, with tangent/cotangent bundle and Lie-algebraic structures for states and costates. The Hamiltonian is constructed using the left-trivialized cotangent lift, and the adjoint recursion involves the coadjoint action: $\lambda_k = \operatorname{Ad}^*_{\exp(-\xi_k)} \lambda_{k+1} + (T_{q_k}\Exp_{q_k, x_k})^*[ \partial_q \ell + (\partial_q \psi)^T \nu_k ]$ with appropriate state and transversality conditions (Phogat et al., 2016, Kipka et al., 2017, Kotpalliwar et al., 2018). Frequency constraints, state-action constraints, and even robust (min-max) optimal control all admit a geometric formulation (Joshi et al., 2020).

For open quantum systems, the discrete contact PMP pairs the costate propagation with a geometric integrator (contact Lie-group variational integrator) ensuring preservation of the CPTP property (completely positive trace-preserving) for Lindblad flow, and the contact geometry necessary for consistent optimal control on quantum state spaces (Colombo, 21 Dec 2025).

5. Abnormal Extremals, Regularity, and Constraint Qualifications

The emergence of abnormal multipliers ($\lambda=0$) is linked to the degeneracy or redundancy of constraints or insufficient coercivity in the cost. Constraint qualifications (weak regularity) are imposed to rule out purely abnormal extremals, such as the requirement that the only solution to

$$(DG_k(x))^* \mu = 0,~ \mu \ge 0,~ \mu^i G_k^i(x) = 0~\forall i$$

is $\mu = 0$. This guarantees positivity of $\lambda$ and "normal" extremality. In the presence of numerous constraints (as in highly band-limited control), abnormal extremals become unavoidable (K et al., 2018, Paruchuri et al., 2017).

Under weakened assumptions (e.g., only Gâteaux differentiability at the optimum, nonconvex constraint sets), the PMP framework remains valid, accommodating nonconvex or rough problems (e.g., economics or population dynamics), provided suitable local regularity at the optimal trajectory (Blot et al., 2016).

6. Algorithmic Realizations and Numerical Methods

The indirect optimal control approach leverages the PMP to define forward-backward shooting algorithms, integrating state and costate equations with a Hamiltonian maximization (or saddle-point, in minimax settings) at each stage. For CNN training with discrete PMP, the batch Sequential Quadratic Hamiltonian (bSQH) algorithm alternates forward and backward propagation (analogous to backpropagation) with layerwise maximization of an augmented Hamiltonian, employing adaptive penalty scaling for stability and convergence (Hofmann et al., 15 Apr 2025).

For open quantum systems, structure-preserving integrators (e.g., contact LGVI) ensure that both geometric and physical invariants are maintained throughout the iteration, circumventing the drift (e.g., trace loss, positivity violation) observed in explicit RK2 or other non-geometric schemes (Colombo, 21 Dec 2025).

7. Representative Applications and Illustrative Examples

Linear-Quadratic Problems: Setting $M = \mathbb{R}^n$, $f_k(x, u) = A_k x + B_k u$, $L_k(x, u) = \frac12x^T Q_k x + \frac12u^T R_k u$, and frequency or rate constraints leads to a coupled system of Riccati and side-constraint equations, solvable by multiple-shooting or QP (K et al., 2018, Ganguly et al., 2023, Paruchuri et al., 2017).

Quantum Control: For unitary or Lindblad evolution, the discrete PMP guides forward-backward iteration for finite-dimensional density operators, maximizing fidelity and minimizing control effort (Dehaghani et al., 2023, Colombo, 21 Dec 2025).

Robust Control: In the min-max setting, adjoint recursions and a Hamiltonian saddle-point condition involving both the control $u_k$ and the disturbance $d_k$ define a game-theoretic discrete PMP (Joshi et al., 2020).

Neural Network Training: Layerwise PMP optimization, with layer Hamiltonians and adjoints, provides a rigorous justification for a family of "indirect" training algorithms, especially effective for sparsity-promoting objectives (Hofmann et al., 15 Apr 2025).

---

The discrete Pontryagin Maximum Principle in its modern geometric, constrained, and structure-preserving forms encompasses a broad class of finite-horizon optimal control problems, bridging classical and quantum, Euclidean and manifold, deterministic and robust optimization. Rigorous adjoint recursions, Hamiltonian maximization (or saddle-point) criteria, and flexible handling of global and local constraints make PMP indispensable in dynamic optimization and algorithmic control theory across engineering, applied mathematics, and data-driven domains (K et al., 2018, Kipka et al., 2017, Paruchuri et al., 2017, Phogat et al., 2016, Colombo, 21 Dec 2025, Ganguly et al., 2023, Joshi et al., 2020, Dehaghani et al., 2023, Blot et al., 2016, Hofmann et al., 15 Apr 2025).