---
title: Discrete Pontryagin Maximum Principle
url: https://www.emergentmind.com/topics/discrete-pontryagin-maximum-principle-pmp
type: topic
---

# Discrete Pontryagin Maximum Principle

The discrete Pontryagin Maximum Principle (PMP) is a cornerstone in finite-horizon optimal control theory for systems evolving in discrete time. It delivers first-order necessary conditions for optimality, generalizing the variational extremality of an optimal trajectory by introducing adjoint (costate) variables and enforcing a maximization (or saddle-point, in minimax settings) condition on an appropriate Hamiltonian function. The discrete PMP extends to systems on smooth manifolds, matrix Lie groups, and under complex constraints, including pointwise state, control, and global (e.g., frequency or rate) restrictions. This article surveys the mathematical structure, derivations, and representative applications of the discrete PMP, consolidating contemporary research spanning geometric, robust, and structure-preserving perspectives.

## 1. General Formulation and Problem Classes

The discrete PMP addresses optimal control problems over a finite horizon, $k=0,1,\dots,N$, for systems with state evolution
\[
x_{k+1} = f_k(x_k, u_k), \quad x_0 \textrm{ given},
\]
and cost functionals of the form
\[
J(\{x_k, u_k\}) = \Phi(x_N) + \sum_{k=0}^{N-1} L_k(x_k, u_k),
\]
subject to
- pointwise state constraints: $x_k \in S_k \subset M$, often expressed via smooth inequalities $G_k(x_k) \le 0$,
- pointwise control constraints: $u_k \in U_k \subset \mathbb{R}^m$,
- and global constraints, such as spectral (frequency) or rate limitations: typically linear, e.g. $\sum_{k=0}^{N-1} F_k u_k = 0$ for frequency support constraints [1807.00698, 1708.04419, 1803.03052, 2305.14940].

The system may evolve on a finite-dimensional manifold $M$ or on a matrix Lie group $G$ equipped with a Lie algebra $\mathfrak{g}$, with dynamics expressed accordingly. The data $(f_k, L_k, \Phi, G_k)$ are typically assumed $C^1$, though various relaxations (e.g., Gâteaux differentiability) are admissible under weaker regularity assumptions [1601.08147].

## 2. Discrete Hamiltonian, Adjoint Equations, and PMP Conditions

**Stage Hamiltonian:** For state $x_k \in M$, costate $p_{k+1} \in T^*_{x_{k+1}}M$, and control $u_k \in U_k$,
\[
H_k(x_k, p_{k+1}, u_k) = \langle p_{k+1}, f_k(x_k, u_k) \rangle - L_k(x_k, u_k)
\]
or, on a Lie group,
\[
H_k(q_k, x_k, \lambda_{k+1}, p_{k+1}, u_k) = \langle \lambda_{k+1}, \exp^{-1}(\cdot) \rangle_{\mathfrak{g}^*} + \langle p_{k+1}, f(q_k, x_k, u_k) \rangle + \ell(q_k, x_k, u_k)
\]
[1612.08022, 1803.03052].

**Costate Recursions:** The optimal trajectory $\{ (\bar{x}_k, \bar{u}_k) \}$ admits (possibly non-vanishing) multipliers:
- abnormality multiplier $\lambda \ge 0$,
- costates $\{p_k \}$ (backward recursions),
- constraint multipliers for state ($\mu_k \ge 0$), global constraints ($\nu$ or $\gamma$).

The adjoint equations are:
\[
p_k = \partial_x H_k(\bar{x}_k, p_{k+1}, \bar{u}_k) + (DG_k(\bar{x}_k))^* \mu_k,
\]
with the backward boundary at $k=N$ involving $-\lambda d\Phi(\bar{x}_N)$ and possible additions from active constraints [1807.00698, 1707.03873].

**Stationarity/Maximization Condition:** At each $k$,
\[
\left\langle \partial_u H_k(\bar{x}_k, p_{k+1}, \bar{u}_k) + F_k^T \nu,~ u - \bar{u}_k \right\rangle \le 0
\]
for all $u$ in a local tent of $U_k$ at $\bar{u}_k$ (the tangent cone; for convex $U_k$, equivalent to a global maximum),
\[
\bar{u}_k = \arg\max_{u \in U_k} H_k(\bar{x}_k, p_{k+1}, u) - \langle \nu, F_k u \rangle
\]
[1807.00698, 1708.04419, 1803.03052].

**Complementary Slackness:** For $G_k(x_k) \le 0$,
\[
\mu_k^i\, G_k^i(\bar{x}_k) = 0,~ \mu_k^i \ge 0
\]
for all $i$, enforcing activity only at binding constraints.

**Nontriviality:** $(\lambda, \nu, \{\mu_k\})$ do not vanish simultaneously.

## 3. Frequency and Rate Constraints

### Frequency Constraints

Global frequency constraints restrict the DFT spectrum of $u_k$:
\[
\widehat{u}^{(j)}_\ell = \sum_{k=0}^{N-1} u_k^{(j)} e^{-2\pi i \ell k / N},~
\operatorname{supp}~\widehat{u}^{(j)} \subset W^{(j)}
\]
For linear enforcement, this takes the form $\sum_{k=0}^{N-1} F_k u_k = 0$, leading to a Lagrange multiplier $\nu$ in Hamiltonian maximization [1807.00698, 1708.04419, 1803.03052].

### Rate Constraints

A rate constraint imposes bounds on increments, $u_{k+1} - u_k \in V$ (compact/convex $V$):
\[
\lambda_k \in N_V(u_{k+1} - u_k),~ \langle \lambda_k, u_{k+1} - u_k \rangle = 0
\]
and modifies the stagewise Hamiltonian by rate-multiplier terms $\langle \lambda_{k-1} - \lambda_k, u_k \rangle$. The stationarity and costate recursions must accommodate these, yielding coupled backward equations for both $p_k$ and $\lambda_k$ [2305.14940].

## 4. Geometric and Structure-Preserving Extensions

The geometric discrete-time PMP extends to nonlinear controlled systems on smooth manifolds or matrix Lie groups, with tangent/cotangent bundle and Lie-algebraic structures for states and costates. The Hamiltonian is constructed using the left-trivialized cotangent lift, and the adjoint recursion involves the coadjoint action:
\[
\lambda_k = \operatorname{Ad}^*_{\exp(-\xi_k)} \lambda_{k+1} + (T_{q_k}\Exp_{q_k, x_k})^*[ \partial_q \ell + (\partial_q \psi)^T \nu_k ]
\]
with appropriate state and transversality conditions [1612.08022, 1707.03873, 1803.03052]. Frequency constraints, state-action constraints, and even robust (min-max) optimal control all admit a geometric formulation [2007.13459].

For open quantum systems, the discrete contact PMP pairs the costate propagation with a geometric integrator (contact Lie-group variational integrator) ensuring preservation of the CPTP property (completely positive trace-preserving) for Lindblad flow, and the contact geometry necessary for consistent optimal control on quantum state spaces [2512.18879].

## 5. Abnormal Extremals, Regularity, and Constraint Qualifications

The emergence of abnormal multipliers ($\lambda=0$) is linked to the degeneracy or redundancy of constraints or insufficient coercivity in the cost. Constraint qualifications (weak regularity) are imposed to rule out purely abnormal extremals, such as the requirement that the only solution to
\[
(DG_k(x))^* \mu = 0,~ \mu \ge 0,~ \mu^i G_k^i(x) = 0~\forall i
\]
is $\mu = 0$. This guarantees positivity of $\lambda$ and "normal" extremality. In the presence of numerous constraints (as in highly band-limited control), abnormal extremals become unavoidable [1807.00698, 1708.04419].

Under weakened assumptions (e.g., only Gâteaux differentiability at the optimum, nonconvex constraint sets), the PMP framework remains valid, accommodating nonconvex or rough problems (e.g., economics or population dynamics), provided suitable local regularity at the optimal trajectory [1601.08147].

## 6. Algorithmic Realizations and Numerical Methods

The indirect optimal control approach leverages the PMP to define forward-backward shooting algorithms, integrating state and costate equations with a Hamiltonian maximization (or saddle-point, in minimax settings) at each stage. For CNN training with discrete PMP, the batch Sequential Quadratic Hamiltonian (bSQH) algorithm alternates forward and backward propagation (analogous to backpropagation) with layerwise maximization of an augmented Hamiltonian, employing adaptive penalty scaling for stability and convergence [2504.11647].

For open quantum systems, structure-preserving integrators (e.g., contact LGVI) ensure that both geometric and physical invariants are maintained throughout the iteration, circumventing the drift (e.g., trace loss, positivity violation) observed in explicit RK2 or other non-geometric schemes [2512.18879].

## 7. Representative Applications and Illustrative Examples

**Linear-Quadratic Problems:** Setting $M = \mathbb{R}^n$, $f_k(x, u) = A_k x + B_k u$, $L_k(x, u) = \frac12x^T Q_k x + \frac12u^T R_k u$, and frequency or rate constraints leads to a coupled system of Riccati and side-constraint equations, solvable by multiple-shooting or QP [1807.00698, 2305.14940, 1708.04419].

**Quantum Control:** For unitary or Lindblad evolution, the discrete PMP guides forward-backward iteration for finite-dimensional density operators, maximizing fidelity and minimizing control effort [2302.09142, 2512.18879].

**Robust Control:** In the min-max setting, adjoint recursions and a Hamiltonian saddle-point condition involving both the control $u_k$ and the disturbance $d_k$ define a game-theoretic discrete PMP [2007.13459].

**Neural Network Training:** Layerwise PMP optimization, with layer Hamiltonians and adjoints, provides a rigorous justification for a family of "indirect" training algorithms, especially effective for sparsity-promoting objectives [2504.11647].

---

The discrete Pontryagin Maximum Principle in its modern geometric, constrained, and structure-preserving forms encompasses a broad class of finite-horizon optimal control problems, bridging classical and quantum, Euclidean and manifold, deterministic and robust optimization. Rigorous adjoint recursions, Hamiltonian maximization (or saddle-point) criteria, and flexible handling of global and local constraints make PMP indispensable in dynamic optimization and algorithmic control theory across engineering, applied mathematics, and data-driven domains [1807.00698, 1707.03873, 1708.04419, 1612.08022, 2512.18879, 2305.14940, 2007.13459, 2302.09142, 1601.08147, 2504.11647].

Source: https://www.emergentmind.com/topics/discrete-pontryagin-maximum-principle-pmp