Mirror Descent for Deterministic Optimal Control

Abstract: We study an explicit mirror-descent method for finite-horizon deterministic optimal control problems. The method is motivated by Pontryagin's maximum principle: at each iteration, one solves the state and adjoint equations and updates the control by maximizing a first-order approximation of the regularized Hamiltonian penalized by a Bregman divergence. In the Euclidean case, the update reduces to a projected gradient step in the control variable. Under global smoothness assumptions and uniform convexity of the mirror map, we prove a relative smoothness estimate for the cost functional and derive an energy dissipation inequality for sufficiently small step sizes. Under an additional concavity assumption on the unregularized Hamiltonian and convexity of the terminal cost, we establish relative convexity of the regularized objective. These estimates yield an $O(1/n)$ convergence rate in the unregularized convex case and a geometric rate when the control regularization parameter is positive. Numerical examples illustrate the behavior of the method in linear-quadratic, degenerate convex, and nonlinear high-dimensional settings.