Sub-optimal control by primal-dual gradient dynamics

Abstract: This note generalizes the port-Hamiltonian formulation of the continuous time primal-dual gradient algorithm for static constrained convex optimization to the convex optimal control problem.The resulting dynamics is shown to be a port-Hamiltonian system of partial differential equations, involving ordinary physical time as well 'algorithmic' time. Convergence to the optimal control solution is indicated, and it is argued that sub-optimal control strategies could be derived starting from the partial differential equation formulation.

• The paper extends continuous-time primal-dual gradient dynamics to infinite-dimensional convex optimal control via a port-Hamiltonian PDE framework.
• It establishes stability and convergence through Lyapunov and LaSalle invariance principles using a shifted Hamiltonian approach.
• The methodology enables practical sub-optimal control strategies for real-time applications by discretizing the port-Hamiltonian PDE system.

Sub-optimal Control via Primal-Dual Gradient Dynamics: Port-Hamiltonian PDE Formulation

Overview and Motivation

The paper "Sub-optimal control by primal-dual gradient dynamics" (2606.18818) systematically generalizes the port-Hamiltonian formulation of continuous-time primal-dual gradient algorithms (PDGCT) from static constrained convex optimization to convex optimal control problems. Central to this work is the explicit realization of infinite-dimensional PDGCT as a port-Hamiltonian system of partial differential equations (PDEs), inherently coupling both physical time (of the system dynamics) and algorithmic time (of the optimizer dynamics). The analysis foregrounds interconnection properties, stability guarantees via Lyapunov arguments, and demonstrates that such a port-Hamiltonian PDE perspective enables both theoretical convergence indications and practical construction of sub-optimal control strategies.

Static Case: Port-Hamiltonian Structure of PDGCT

The paper situates the classical primal-dual gradient algorithm within the port-Hamiltonian system framework for static constrained convex optimization. Given a strictly convex objective $V(q)$ with affine constraints $Cq = b$, the associated primal-dual gradient dynamics are interpreted as incremental port-Hamiltonian systems operating on energy variables. The Hamiltonian encapsulates the optimizer state, ensuring that the dissipation inequality:

$$(q - q_*)^\top \left[\frac{\partial V}{\partial q}(q) - \frac{\partial V}{\partial q}(q_*)\right] \leq 0$$

yields guaranteed convergence to the unique optimal solution $(q_*,\lambda_*)$ under strict convexity. The incremental port-Hamiltonian structure supports compositionality and permits augmented systems (e.g., distributed optimization and interconnections), leveraging Lyapunov arguments based on shifted Hamiltonians.

Infinite-Dimensional Extension: Optimal Control via PDGCT

Transitioning to the infinite-dimensional setting, the optimization target becomes a convex cost functional over a linear system:

$$\min_{u(\cdot)} \int_0^T K(x(t), u(t)) dt \quad \text{s.t.} \quad \dot{x} = Ax + Bu, ~ x(0) = x_0.$$

The dynamics-driven constraints are encoded as $\dot{x}(t) - Ax(t) - Bu(t) = 0$, with the Lagrangian dual variables now function-valued over $[0, T]$. The corresponding PDGCT yields a system of PDEs incorporating both physical time $t$ and algorithmic time $\tau$:

$$\frac{\partial}{\partial \tau} \begin{bmatrix} Xx \ Uu \ Pp \end{bmatrix} = J \begin{bmatrix} x \ u \ p \end{bmatrix} - \begin{bmatrix} \frac{\partial K}{\partial x} \ \frac{\partial K}{\partial u} \ 0 \end{bmatrix}$$

where $Cq = b$0 is a skew-adjoint operator intertwining differential and algebraic terms derived from system dynamics. The equilibrium, characterized by vanishing right-hand side, coincides with the solution given by Pontryagin's Minimum Principle for linear dynamics and convex cost criteria.

Port-Hamiltonian PDE Structure and Stability Analysis

The port-Hamiltonian system formulation is realized through suitable energy variables and functionals:

$Cq = b$1

and its shifted version with respect to the optimal solution. Skew-adjointness ensures dissipation properties analogous to the static case, substantiating stability. Under imposed boundary conditions consistent with optimality ($Cq = b$2), the invariant set defined by the Lyapunov functional is restricted to the equilibrium solution, with controllability of $Cq = b$3 ensuring uniqueness. The analysis leverages infinite-dimensional LaSalle invariance principles for convergence guarantees.

Practical Implications: Sub-optimal Control and Computational Schemes

This framework directly motivates the derivation of sub-optimal control strategies via numerical discretizations of the port-Hamiltonian PDE system. By operating the PDGCT iteratively in algorithmic time, practitioners can obtain feasible (but potentially sub-optimal) control policies well-suited for computationally-constrained settings, such as real-time model predictive control (MPC)—circumventing requirements for full dissipativity checks due to inherent passivity guarantees. Moreover, the port-Hamiltonian structure facilitates interconnections with physical systems for distributed optimal control, lending itself to extensibility for state/input constraints, incorporation of general positive operators, and hybrid scenarios.

Theoretical Implications and Future Perspectives

The explicit PDE-based port-Hamiltonian representation consolidates the bridge between optimizer and system dynamics, laying the groundwork for rigorous stability and interconnection analysis in infinite-dimensional systems. This perspective invites further exploration into generalized operator choices, advanced constraint integration, and input-output augmentation for distributed control architectures. Future investigations may focus on scalable numerical schemes derived from the port-Hamiltonian PDEs, deeper links to passivity-based control in both finite and infinite-dimensional instances, and formalization of weak convergence properties under relaxed technical conditions.

Conclusion

The paper establishes that continuous-time primal-dual gradient dynamics can be systematically extended to infinite-dimensional convex optimal control problems through a port-Hamiltonian PDE framework. This methodology yields robust convergence results, theoretical stability guarantees, and practical avenues for sub-optimal control implementation, notably in computationally challenging settings. The approach strengthens the synergy between optimization and system-theoretic paradigms in control, with clear potential for future innovation in distributed, constrained, and energy-based control designs.