Interior Point Differential Dynamic Programming

• Interior Point Differential Dynamic Programming (IPDDP) is an algorithm family combining DDP's recursive structure with interior point techniques for efficient nonlinear optimal control under constraints.
• It integrates barrier and primal–dual methods to directly manage nonlinear equality and inequality constraints, enhancing robustness and achieving local quadratic convergence.
• Applied in robotics and real-time control, IPDDP produces smooth trajectories for obstacle avoidance, contact-implicit planning, and hybrid model predictive control.

Interior Point Differential Dynamic Programming (IPDDP) refers to a family of algorithms that integrate the structure-exploiting backward–forward recursion of Differential Dynamic Programming (DDP) with the primal-dual and barrier-based philosophy of interior point methods, enabling efficient and robust solutions to discrete-time, finite-horizon optimal control problems with nonlinear dynamics and general nonlinear equality and inequality constraints.

1. Formulation and Theoretical Foundations

IPDDP combines the recursive value function approximation of DDP with a primal–dual interior point framework. The objective is to solve constrained optimal control problems of the form: $$\min_{\{x_t, u_t\}} \sum_{t=0}^{N-1} \ell(x_t, u_t) + \ell_N(x_N)$$ subject to

$$x_{t+1} = f(x_t, u_t),\quad h(x_t, u_t) = 0,\quad g(x_t, u_t) \leq 0$$

where $h(\cdot)$ and $g(\cdot)$ represent nonlinear equality and inequality constraints, respectively.

Inequality constraints are incorporated using barrier methods, with the augmented Lagrangian or logarithmic penalty added to the stage cost, e.g.,

$$\phi_{\mu}(x_t, u_t) = \ell(x_t, u_t) - \mu \sum_{i=1}^m \ln(-g_i(x_t,u_t))$$

The current iterate maintains strict inequality ($g_i(x_t,u_t) < 0$), and the interior point/barrier parameter $\mu$ is decreased across the iteration sequence, tightening feasibility.

The backward pass linearizes the KKT conditions (including both equality and perturbed complementarity constraints) and solves a Newton-type system for search directions. Dual variables ("multipliers" or "slacks") are updated alongside primal variables, in contrast to active-set approaches.

The forward pass applies the computed affine policies, simulates the resulting state trajectories, and assesses step acceptance. A line-search or filter-based strategy ensures progress in either the barrier cost or in constraint violation, supporting both infeasible and feasible iterates (Xu et al., 11 Apr 2025).

2. Primal-Dual Interior Point Integration

Primal–dual methods appear at the core of modern IPDDP. For a generic set of dynamic, equality, and inequality constraints, the local stagewise KKT system is assembled as: $$\begin{bmatrix} Q_{uu}^t & Q_{us}^t \ Q_{su}^t & C_t \end{bmatrix} \begin{bmatrix} \delta u \ \delta s \end{bmatrix} = -\begin{bmatrix} Q_u^t \ r_t \end{bmatrix} -\begin{bmatrix} Q_{ux}^t \ Q_{sx}^t \end{bmatrix} \delta x$$ Slacks $s$, multipliers $y$, and (in some variants) additional slack variables for infeasible initializations are tracked in either perturbed primal-dual or directly log-barrier forms (Pavlov et al., 2020, Prabhu et al., 18 Sep 2024). This system explicitly avoids the combinatorial complexity of active sets.

Filter-type step acceptance makes use of sufficient decrease in either a merit function (cost + barrier) or in constraint residuals (Xu et al., 11 Apr 2025), enabling progress even from infeasible starting points.

The primal-dual “central path” is followed as $\mu \to 0$, so stationary points converge to solutions of the original (unperturbed) optimal control problem. Under standard assumptions (full row rank of equality constraints, positive-definite Hessian on null-spaces), local quadratic convergence is achieved.

3. Algorithmic Variants and Convergence

IPDDP encompasses several algorithmic variants:

• Feasible-IPDDP: Requires strictly primal-feasible initial trajectories, i.e., with all constraints satisfied and slack variables positive.
• Infeasible-IPDDP: Introduces additional slack variables, converting inequalities $c(x,u) \leq 0$ to equalities with slacks $y \geq 0$ and perturbed complementarity $S y - \mu = 0$. This variant requires no feasible initial guess and converges to KKT points in the limit $\mu \to 0$ (Pavlov et al., 2020).
• Stagewise Generalization: Recent advances permit stagewise equality, inequality, and even complementarity constraints (e.g., for contact dynamics).

Theoretical results show that once the solution is sufficiently close to a perturbed KKT point, quadratic convergence (superlinear reduction in trajectory error) is obtained: $\|w^{+} - w^{\star}\| \leq M \|w - w^{\star}\|^2$ where $w^{\star}$ is a (perturbed) KKT point.

4. Implementation Details and Computational Aspects

Implementations exploit the DDP structure, using condensed KKT system solvers in the backward pass (e.g., through Bunch–Kaufman LDL$^T$ decomposition with inertia correction and diagonal regularization (Xu et al., 11 Apr 2025)). Slack variables and dual multipliers are explicitly updated along with the state and control. For bound constraints, additional logarithmic barrier terms are included.

Forward passes leverage fraction-to-boundary rules and backtracking or filter line-search: a step is accepted if it reduces constraint violation or ensures sufficient (merit) cost decrease.

Applications to contact-implicit planning employ specialized treatment for complementarity constraints ($c \circ d = 0$); these are relaxed as $c \circ d - \mu e = 0$, which fits naturally within the IPDDP KKT framework (Xu et al., 11 Apr 2025).

Complexity per iteration is linear in the number of stages given DDP’s recursive structure. For polynomial trajectory generation, a state-space representation yields linear per-segment complexity, making IPDDP suitable for scalable applications (Cao et al., 2021).

5. Constraint Handling and Comparative Performance

IPDDP handles nonlinear inequalities directly and avoids the combinatorial complexities of active-set or augmented Lagrangian methods. Compared to log-barrier DDP, the primal–dual approach improves step size acceptance and robustness in ill-conditioned scenarios (e.g., bang–bang controls, multiple local minima) (Pavlov et al., 2020).

Comparative studies on robotic planning, car-parking, inverted pendulum, and contact-implicit acrobot swing-up show that IPDDP/Feasible-IPDDP and Infeasible-IPDDP require fewer iterations and exhibit more consistent convergence profiles than Control Limited DDP or pure log-barrier DDP, as well as outperforming AL-iLQR and general-purpose solvers such as IPOPT in time and robustness (Xu et al., 11 Apr 2025).

In hybrid sampling–gradient frameworks, such as MPPI-IPDDP, a coarse solution is generated via Model Predictive Path Integral (MPPI) control, followed by IPDDP-based smoothing within precomputed convex corridors. This two-phase approach combines global exploration with constraint-respecting smoothing, yielding collision-free, dynamically feasible, and highly smooth trajectories for autonomous robotics (Kim et al., 2022).

6. Recent Developments and Extensions

Recent directions in IPDDP include:

• Log-domain barrier formulations: Reformulation of central-path conditions in the log-domain allows all barrier constraints to be encoded within the variable substitution, resulting in strict positivity and superior convergence properties compared to classical barrier approximations (Permenter, 2022).
• Contact-implicit and complementarity constraints: IPDDP2 demonstrates robust handling of nonlinear complementarity constraints (e.g., contact modeling) with provable convergence and efficiency, critical for whole-body manipulation and locomotion.
• Matrix-free and continuous-time perspectives: The matrix-free augmented Lagrangian ODE approach suggests new discretization and search direction ideas for large-scale problems, avoiding ill-conditioned projections and improving scalability (Qian et al., 28 Dec 2024).
• Explicit Update Formulas: Algorithms now offer explicit formulas for all variable updates, enhanced regularization, and systematic handling of equality, inequality, and even arbitrary stagewise constraints in multiple-shooting or iLQR settings (Prabhu et al., 18 Sep 2024).

7. Applications and Impact

IPDDP is used in a variety of domains:

• Robotic Trajectory Optimization: Obstacle avoidance, car-parking, cartpole swing-up with equality constraints, inverse dynamics, and contact-implicit trajectory generation for locomotion.
• Embedded Real-Time Control: Model Predictive Control with fast convergence using dual fast-gradient warm starts and primal–dual Newton phases (Zhang et al., 2017).
• Trajectory Generation in Flat Systems: The DIRECT framework for polynomial trajectory generation in differentially flat systems leverages IPDDP for scalable, online planning (Cao et al., 2021).
• Autonomous Robotics: Real-time hybrid planning in complex, obstacle-rich environments (Kim et al., 2022).

IPDDP provides an algorithmic bridge between classical optimal control and modern barrier-based convex optimization, yielding methods with local quadratic convergence, robust constraint satisfaction, and strong practical performance on robotics benchmarks and dynamic systems.