Direct Optimal Control Methods

• Direct optimal control methods are discretize-then-optimize strategies that convert infinite-dimensional control problems into finite nonlinear programming problems.
• They employ techniques like single shooting, multiple shooting, collocation, and finite element transcription to enforce system dynamics and constraints.
• These methods offer robustness to nonsmoothness and flexible constraint handling, making them suitable for complex, high-dimensional trajectory optimization tasks.

Direct optimal control methods are discretize-then-optimize strategies for solving systems and trajectory optimization problems governed by differential (or difference) equations. Rather than first deriving the necessary optimality conditions via calculus of variations (as in indirect methods), direct methods transcribe the original infinite-dimensional control problem into a finite-dimensional nonlinear programming (NLP) problem. This transcribed problem, which encodes state and control trajectories as decision variables along with the imposed dynamics and path/boundary constraints, is then solved using established numerical optimization algorithms. Direct optimal control methods are notable for their robustness to nonsmoothness, amenability to constraint handling, and broad applicability to nonlinear and high-dimensional problems.

1. Fundamental Principles and Classification

The core principle of direct optimal control is the discretization of both the control (and often the state) variables, replacing the original continuous dynamics and objective functions with a high-dimensional but finite parameter optimization. The discretization can be performed using:

• Single Shooting (or Sequential Shooting): The control is parameterized (e.g., as piecewise constant over intervals), the state is computed by integrating the dynamical system forward in time from the initial condition, and constraints are imposed at the final state or at specific waypoints. Only control parameters (and possibly time intervals) are decision variables. This approach results in low-dimensional NLPs but can be sensitive to instability and local minima for long-time horizons or stiff dynamics (Shahab et al., 2013).
• Multiple Shooting: The time horizon is divided into intervals. Control and initial state values at interval starts are decision variables. Piecewise integration across intervals generates the state trajectory, with additional continuity (defect) constraints enforcing continuity between intervals. This improves numerical conditioning and convergence properties, especially for unstable systems (Kiessling et al., 15 Mar 2024).
• Direct Collocation: Both state and control are discretized (e.g., at mesh or quadrature points). The system dynamics are enforced at collocation nodes, typically using polynomial or spline interpolants. This approach yields large sparse NLPs with advantageous Jacobian structures for interior-point solvers. It is widely used in trajectory optimization, robotics, aerospace, and quantum control (Martinsen et al., 2019, Trowbridge et al., 2023).
• Direct Transcription with Finite Elements: The entire state and control trajectories are represented as continuous finite element functions, and their coefficients become NLP decision variables. Dynamics and path constraints are enforced via collocation or weighted residuals (Neuenhofen et al., 2020). This includes integral penalty-barrier methods that penalize equality constraint violations in the objective.
• Direct Policy or Parameterized Policy Optimization: In reinforcement learning and closed-loop optimal control, direct methods can optimize over parameterized policy classes (linear, nonlinear, or neural network controllers) directly in the space of feedback control laws rather than over open-loop controls (Lu et al., 2019, Howell et al., 2020).

This discretization step fundamentally transforms an infinite-dimensional dynamical system problem into one of finite (though often high) dimension, which can then be solved with off-the-shelf NLP solvers such as IPOPT, SNOPT, or sequential quadratic programming variants.

2. Mathematical Formulation and Discretization

A prototypical continuous-time optimal control problem is typically written in Bolza form:

$$\begin{align*} \min_{u(\cdot),\; t_f} & \quad J = \phi(x(t_f), t_f) + \int_{t_0}^{t_f} L(x(t), u(t), t) \,dt \ \text{subject to:} & \quad \dot{x}(t) = f(x(t), u(t), t),\quad x(t_0) = x_0 \ & \quad g(x(t), u(t), t) \leq 0, \quad h(x(t), u(t), t) = 0 \ \end{align*}$$

After discretization using a selected scheme (single/multiple shooting, collocation, etc.), this is transcribed into an NLP:

$$\begin{align*} \min_{w} & \quad J(w) := \Phi(X_N, w) + \sum_{k=0}^{N-1} L(X_k, U_k, T_k) T_k \ \text{s.t.} & \quad X_{k+1} = f_p(X_k, U_k, T_k), \quad k=0,\dotsc,N-1 \ & \quad g_k(X_k, U_k, T_k) \leq 0, \quad h_k(X_k, U_k, T_k) = 0 \ & \quad \text{(path constraints, boundary constraints, etc.)} \end{align*}$$

where $X_k$ and $U_k$ are the discretized states and controls, $T_k$ may represent optimized time intervals, and $f_p$ approximates the time evolution operator (e.g., via a Taylor expansion or integration scheme) (Shahab et al., 2013).

For direct collocation, using Lagrange polynomial basis and collocating at specific points (e.g., Legendre–Gauss–Radau) yields defect constraints:

$$\sum_{j=1}^{N+1} D_{ij} X_j = (t_f - t_0)/2\, a(X_i, U_i, t(\tau_i)), \quad i=1,\ldots,N$$

where $D$ is the differentiation matrix evaluated at the collocation points, and $a$ is the vector field corresponding to the ODE right-hand side (Agamawi et al., 2019, III et al., 9 Oct 2024).

For second-order systems, position-based or order-tailored collocation (e.g. semi-Hermite bases) uses the intrinsic second-order structure to reduce variable count and enhance computational efficiency (Simpson et al., 2022).

3. Treatment of Constraints and Problem Structure

Direct methods support general path, boundary, and state-control constraints:

• Physical constraints: Can be enforced directly as algebraic inequalities (e.g., torque and speed bounds, actuator saturation, position box constraints) (Shahab et al., 2013, Martinsen et al., 2019).
• Terminal constraints: E.g., fixed or windowed final state, orientation, or timing constraints, framed as equality or inequality constraints on the terminal decision variables.
• State-dependent and mixed constraints: Managed in transcription by introducing additional defect constraints, slack variables, or by using penalty/barrier approaches if strict feasibility is not required (Neuenhofen et al., 2020).
• Bang-bang and hybrid switching structure: Structure-exploiting adaptive mesh refinement methods with switching time estimation are used to accurately capture control discontinuities by introducing domain partitioning and explicit switching time parameterization (Agamawi et al., 2019).

Regularization and constraint handling in direct optimal control can also include:

• Integral penalty-barrier methods: Adding terms penalizing the integral of squared constraint residuals in the cost, controlling the trade-off between feasibility and optimality (Neuenhofen et al., 2020).
• Integration error regularization: Augmenting the NLP objective with terms that penalize large local integration errors (estimated via embedded Runge-Kutta pairs), thus discouraging spurious solutions arising from low-accuracy discretizations especially in stiff or embedded applications (Harzer et al., 16 Mar 2025).

4. Numerical Implementation and Algorithmic Aspects

The large-scale NLPs resulting from direct methods necessitate:

• Efficient derivative computation: Vectorized and sparse second-order forward automatic differentiation is essential for exploiting sparsity across mesh points. Frameworks such as those described in (Zou et al., 13 Jun 2025) enable computation of both gradients and Hessians in sparse coordinate/list formats suitable for parallel processing and memory optimization.
• Adaptive mesh refinement: Modern direct collocation methods utilize dual-refinement (both h- [interval splitting] and p- [polynomial degree increase]) and reduction strategies with validated error estimation (e.g., by comparing the collocation solution with an explicit ODE simulation on the final mesh), thereby producing minimum-size meshes that achieve the desired tolerance (III et al., 9 Oct 2024).
• Advanced integration schemes: For high-order or stiff systems, specialized transcription approaches—such as modified Euler or RK4 for second-order systems (Tang et al., 10 Mar 2024), or integral formulations of Legendre–Gauss–Lobatto collocation for improved costate accuracy (Abadia-Doyle et al., 16 Jun 2025)—yield advances in accuracy and stability.
• Solution of the resulting NLP: Solvers such as IPOPT, SNOPT, or Sequential Quadratic Programming variants are commonly used. For problems with embedded real-time requirements or extremely large-scale discretizations, specialized Riccati-based algorithms or feasibility-projection variants (e.g., FP-DDP) can provide rapid convergence and dynamic feasibility at each iteration (Kiessling et al., 15 Mar 2024).

5. Typical Applications and Scenarios

Direct optimal control methods have been effectively deployed in a diverse range of domains:

• Mobile and autonomous robotics: Time-energy optimal navigation with direct transcription, including multi-objective costs and optimization over both control variables and discretization time intervals for energy-efficient path planning (Shahab et al., 2013).
• Mechanical systems and aerospace: Trajectory optimization for underactuated mechanisms, spacecraft maneuvering (e.g., orbit raising using integral-form LGL collocation), implementation in docking maneuvers with real-time constraints, and hybrid systems with fast changing dynamics (Martinsen et al., 2019, Abadia-Doyle et al., 16 Jun 2025).
• Quantum control: Direct collocation methods (e.g., PICO) applied to high-fidelity, free-time, and minimum-time control in quantum computing, including applications to single-qubit, two-qubit, and multi-qubit gates as well as state transfer in cavity QED systems. Advantages include flexible handling of pulse amplitude/smoothness constraints and the ability to optimize over Hamiltonian parameters such as cavity coupling strength (Trowbridge et al., 2023, Ramos et al., 2 Jan 2024).
• Chemistry and molecular control: Direct optimization for laser-driven manipulation of molecular reactions, e.g., transferring hydrogen in the presence of vibrational strong coupling, where system parameters such as cavity coupling and final time are included as co-designed variables in the optimization (Ramos et al., 2 Jan 2024).
• Reinforcement learning and direct policy optimization: Combination of direct trajectory optimization with parameterized feedback policies and deterministic sampling for robust motion planning in nonlinear stochastic systems, with convergence guarantees and demonstrated equivalence to optimal LQR policies in the linear-quadratic-Gaussian setting (Howell et al., 2020, Lu et al., 2019).

6. Recent Advances and Methodological Developments

The past decade has produced several methodological advances in direct optimal control:

Theme	Key Advances	References
High-order/collocation-adapted formulations	Second-order tailored collocation, integral LGL forms	(Simpson et al., 2022, Abadia-Doyle et al., 16 Jun 2025)
Adaptive mesh/error control	Dual h/p mesh adaptation with validated error bounds	(III et al., 9 Oct 2024)
Efficient derivative computation	Vectorized sparse second-order AD, parallelization	(Zou et al., 13 Jun 2025)
Spurious solution mitigation	Integration error regularization using embedded RK	(Harzer et al., 16 Mar 2025)
Costate and adjoint accuracy	Full-rank adjoint systems, novel costate estimates	(Abadia-Doyle et al., 16 Jun 2025)
Feasibility-preserving DDP	FP-DDP algorithms with dynamic feasibility at every step	(Kiessling et al., 15 Mar 2024)

Direct methods have also integrated sampling-based approaches for Pontryagin-optimal solution synthesis, relaxed measure-valued control formulations, and specialized methods for handling bang-bang structure and switching function estimation (He et al., 2017, Agamawi et al., 2019).

7. Limitations, Trade-offs, and Future Directions

Key issues in direct optimal control include:

• Trade-off between discretization accuracy and computational cost: Higher discretization order or mesh density increases accuracy but also problem size and solver time. Integral penalty/barrier or integration error regularization can mediate this trade-off in resource-limited applications (Neuenhofen et al., 2020, Harzer et al., 16 Mar 2025).
• Costate recovery and sensitivity estimates: Some collocation and low-order methods may yield inaccurate costates or require post-processing/filtering. Recent full-rank adjoint formulations and alternative costate estimates directly improve this aspect (Abadia-Doyle et al., 16 Jun 2025).
• Nonconvexity and local minima: Like all nonlinear programming problems, direct transcription of optimal control problems can be susceptible to local minima, especially in high-dimensional or hybrid/integer-constrained scenarios. Careful initialization, problem structure exploitation, and global optimization strategies remain areas of ongoing research.
• Parallelization and scalability: For large-scale and multi-phase systems, efficient exploitation of sparsity, parallel computation, and vectorization—as emphasized in vectorized sparse AD frameworks—are essential.

Looking forward, research continues to focus on mesh adaptation strategies, error estimation, open-source solver infrastructure, integration with ML-based policy learning, and the extension of direct optimal control principles to new domains, including embedded systems, quantum devices, and large-scale cyber-physical systems.