Box-Constrained iLQR for Nonlinear Control

• Box-Constrained iLQR is an optimal control method that extends classical iLQR by explicitly enforcing upper and lower bounds on inputs and states in nonlinear systems.
• It employs interior-point arc-search, log-barrier, and hybrid penalty-barrier techniques to maintain feasibility and robust convergence during iterative optimization.
• The method has been effectively applied in robotics, autonomous driving, and quantum control, demonstrating superior real-time and high-dimensional control performance.

A Box-Constrained Iterative Linear Quadratic Regulator (iLQR) is an optimal control method extending classic iLQR to nonlinear systems with explicit upper and lower bounds—“box constraints”—on control inputs and, in some formulations, states. Modern variants of box-constrained iLQR leverage primal-dual interior-point methods, log-barrier relaxations, and hybrid penalty-barrier techniques to maintain feasibility and robust convergence. These methods have been justified by complexity-theoretic guarantees and demonstrated in domains such as robotics, quantum control, and autonomous driving.

1. Mathematical Formulation of Box-Constrained LQR and iLQR

Consider the finite-horizon, discrete-time system

$x_{k+1} = f(x_k, u_k), \quad x_0~\textrm{given}$

with $x_k \in \mathbb{R}^n$ and $u_k \in \mathbb{R}^m$. For LTI systems, $f(x_k, u_k) = A x_k + B u_k$.

The optimal control objective is quadratic: $$J(u) = \frac12 x_N^T P x_N + \frac12 \sum_{k=0}^{N-1} (x_k^T Q x_k + u_k^T R u_k)$$ with $P, Q \succ 0$, $R \succ 0$.

Box constraints enforce

$\underline{u} \leq u_k \leq \overline{u}.$

For state constraints, $x_{\min, i} \le x_{t,i} \le x_{\max,i}$ are imposed analogously.

Stacking all inputs, the finite-horizon control problem for the LTI case reduces to a convex quadratic program (QP) with box constraints: $$\min_{u\in\mathbb{R}^{n}} \tfrac12\,u^{T}H\,u + f^{T}u \quad\text{s.t.}\quad \underline u\le u\le\overline u$$ where $H$ is constructed from $A$, $B$, $Q$, $R$, $P$, and $f$ collects linear terms from the initial state and the cost (Yang, 2013).

The nonlinear case (iLQR) proceeds by linearizing dynamics and quadratizing the cost stagewise around a nominal trajectory, then solving the local box-constrained QP for increment $\delta u$, and iterating in outer Gauss–Newton fashion (Abhijeet et al., 4 Feb 2026, Yang, 2013).

2. Algorithmic Approaches: Interior-Point, Barrier, and Hybrid Methods

Several computational strategies for box-constrained iLQR are prominent:

2.1 Interior-Point Arc-Search Methods

An interior-point formulation introduces slack variables ($y = u - \underline{u} \ge 0$, $z = \overline{u} - u \ge 0$) and dual multipliers ($\lambda,\gamma \ge 0$), forming Karush-Kuhn-Tucker (KKT) conditions. Complementarity is relaxed ($\lambda_i y_i = \gamma_i z_i = \tau > 0$) and a central path is tracked by arc-search, using two linearizations ($\dot{w}$, $\ddot{w}$). Each iteration searches along an elliptical arc in variable space, stepping as far as possible while maintaining positivity, centrality, and sufficient duality-gap decrease.

The box structure enables a trivial strictly-interior initialization and the algorithm achieves polynomial complexity $O(\sqrt{n} \log(1/\epsilon))$ in problem size (Yang, 2013).

2.2 Log-Barrier Augmented iLQR

A log-barrier approach augments the cost: $$\omega(x_t,u_t) = -\sum_{i} \mu_i [\log(x_{t,i} - x_{min,i}) + \log(x_{max,i} - x_{t,i})] - \sum_j \sigma_j[\log(u_{t,j} - u_{min,j}) + \log(u_{max,j} - u_{t,j})]$$ with positive barrier weights $\mu, \sigma$. This regularizes the Hessian of the Q-function, ensuring positive definiteness ($Q_{uu} + \omega_{uu} \succ 0$) in the backward pass. The forward pass employs a line search to guarantee cost decrease and strict interior feasibility. When any control or state approaches a limit, the corresponding feedback gain vanishes, enforcing feedforward-only (bang–bang) behavior in saturated directions (Abhijeet et al., 4 Feb 2026).

2.3 Hybrid Augmented Lagrangian–Barrier Methods

A two-stage hybrid strategy comprises (i) an Augmented Lagrangian (AL) phase for rapid coarse constraint satisfaction and (ii) a Relaxed Log Barrier (RLB) refinement for high-precision feasibility. The AL phase introduces multipliers $\lambda,\mu$ and penalizes violations via $J_1(X,U;\lambda,\mu)$, updating multipliers and penalty weights at each outer iteration. The RLB stage uses a twice-differentiable relaxed barrier, with globalization via a coupled line-search and Levenberg–Marquardt regularization (CRLS) to maintain $Q_{uu}\succ0$. This enables robust convergence even from infeasible initial guesses, particularly when used with multiple shooting (Tang et al., 2021).

2.4 Direct QP Solution in Riccati-Like Backward Pass

An alternative, especially seen in quantum control, is to explicitly solve the local box-constrained QP at each stage via active-set or coordinate descent, or to “clamp” the unconstrained solution and set feedback gains for saturated controls to zero. This achieves a Riccati-style recursion compatible with box constraints, though the feedback law becomes locally piecewise affine (Heimann et al., 15 Apr 2025).

3. Algorithmic Structure and Complexity

A typical box-constrained iLQR iteration consists of:

1. Trajectory linearization and cost quadratization: Form local linear/quadratic approximations around the current trajectory.
2. Backward pass: Recursively compute local feedback/feedforward gains with QP or barrier incorporation, updating value-function approximations.
3. Forward pass with line search: Apply the updated policy to roll out a feasible (strictly interior) trajectory, possibly clamping controls to the box if necessary.
4. Penalty/barrier weight update: Decrease barrier or Lagrange multipliers per schedule.
5. Convergence check: Iterate until constraint violation is within tolerance and successive iterates change minimally.

Complexity per iLQR iteration is generally $O(N m^3 + N n m)$, where $N$ is horizon length, $n$ state dimension, and $m$ control dimension (Heimann et al., 15 Apr 2025). Arc-search interior-point methods achieve $O(\sqrt n \log(1/\epsilon))$ iteration complexity for the convex QP in the LTI case (Yang, 2013). Multiple shooting introduces $O(N n^2)$ overhead for assembling trajectory defects (Tang et al., 2021).

4. Properties Near Constraint Boundaries and Theoretical Guarantees

Log-barrier methods and explicit box-projections in iLQR exhibit the following structure:

• When the Hessian contribution from the barrier diverges (e.g., as $u_{t,j} \to u_{max,j}$), the corresponding row of the feedback matrix $K_t$ goes to zero; thus, saturated input channels become feedforward-only—no further feedback action is exerted (Abhijeet et al., 4 Feb 2026).
• For active state saturations, the local feedback which would drive the state off its bound is suppressed, and the value-function Hessian diverges in the saturated mode.
• Global convergence is maintained under a sufficiently strict penalty/barrier relaxation schedule. Newton-like convergence is observed locally under suitable regularity; however, when constraints are active, convergence slows to linear rate because of the induced piecewise-affine structure (Tang et al., 2021, Abhijeet et al., 4 Feb 2026, Heimann et al., 15 Apr 2025).
• Arc-search interior-point methods offer worst-case polynomial-time guarantees for the inner QP subproblem (Yang, 2013).

5. Application Domains

Box-constrained iLQR has been developed and benchmarked across a diversity of high-dimensional control settings:

• Nonlinear robot swing-up and manipulation: Benchmark tasks such as pendulum, cart-pole, and acrobot with control and state bounds demonstrate that the methods enforce box constraints strictly at each step and provide control policies that adapt feedback structure according to constraints (Abhijeet et al., 4 Feb 2026, Tang et al., 2021).
• Real-time motion planning for autonomous vehicles: Efficient collision-avoidance is achieved by encoding safe-distancing and input bounds as exponential or log-barrier augmented costs and updating the iLQR policy at rates sufficient for real-world applications (e.g., 50 Hz in simulation, ≥30 Hz in hardware). Both deterministic minimum-distance and chance-constrained minimum-risk formulations are supported (You, 2022).
• Quantum optimal control: For quantum gate synthesis, box bounds (including slew rate limits) impose physical feasibility of control pulses. The Riccati-style backward pass with saturation corrections yields high-fidelity, hardware-compliant pulse protocols (Heimann et al., 15 Apr 2025).

6. Implementation and Comparative Performance

Algorithmic implementations are accessible for a variety of architectures, with the following key features:

Method/Domain	Features	Reference
Arc-search interior-point	Strict box constraints, polynomial complexity, simple feasible initialization	(Yang, 2013)
Log-barrier iLQR	Strict feasibility, vanishing feedback on active bounds, intrinsic regularization	(Abhijeet et al., 4 Feb 2026)
Hybrid AL/RLB with multiple-shoot	Robust infeasible initialization, guaranteed feedback law, fast feasibility	(Tang et al., 2021)
Motion planning (autonomous car)	Real-time rate, barrier relaxation for collision+box constraints	(You, 2022)
Quantum control	Box + rate bounds, Riccati QP backward pass, piecewise-affine law	(Heimann et al., 15 Apr 2025)

Empirical studies report:

• Reliable convergence from feasible or infeasible initializations.
• Strict constraint satisfaction at all iterates (no post hoc projection).
• Superior real-time performance and convergence in comparison to active-set, ALTRO, SS/IPOPT, or direct-collocation methods, especially when constraint structure is exploited.
• Feedback law adapts by reducing gain on saturated constraints, as shown in benchmarks and analysis (Abhijeet et al., 4 Feb 2026, Tang et al., 2021).

7. Extensions, Limitations, and Comparative Remarks

Notable extensions include handling state constraints, rate/slew constraints, and risk-averse or chance-constrained settings through appropriate cost augmentations, barrier functions, and slack variable formulations. Multiple-shooting formulations further enhance numerical stability and allow for infeasible trajectory initialization (Tang et al., 2021).

A limitation of simple projection-based iLQR variants is loss of smooth gradient information at constraint boundaries, reducing asymptotic convergence rates. Log-barrier and interior-point methods maintain smoothness and allow strict interior iterates, avoiding chattering and ensuring robust step acceptance. However, barrier schedules and regularization/damping must be tuned to avoid ill-conditioning or slow convergence near active constraints (Abhijeet et al., 4 Feb 2026, Tang et al., 2021).

A plausible implication is that future research will integrate principled adaptive barrier or penalty strategies, scalable second-order solvers for large-scale nonlinear systems, and tighter coupling of model-predictive control with robust real-time constrained iLQR frameworks.

---

References:

(Yang, 2013): "Constrained LQR Design Using Interior-Point Arc-Search Method for Convex Quadratic Programming with Box Constraints" (Abhijeet et al., 4 Feb 2026): "Safe Optimal Control using Log Barrier Constrained iLQR" (Tang et al., 2021): "Inequality Constrained Trajectory Optimization with A Hybrid Multiple-shooting iLQR" (You, 2022): "Real Time Motion Planning Using Constrained Iterative Linear Quadratic Regulator for On-Road Self-Driving" (Heimann et al., 15 Apr 2025): "Iterative Linear Quadratic Regulator for Quantum Optimal Control"