On the Robustness in Data-Driven Nonlinear Optimal Control: From Stability to Optimality

Abstract: In data-driven nonlinear control, optimal controllers designed from learned models are inevitably subject to model mismatch when deployed on actual systems, potentially compromising both closed-loop stability and optimality. This paper investigates how the model mismatch propagates through the optimal control structure and alters the resulting optimality. First, we show that the nominal optimal value function remains a Lyapunov function under a quantifiable criterion, thereby preserving closed-loop robust stability. Building upon this foundation, we establish explicit characterizations for optimality deviations induced by model mismatch in both closed-loop performance and optimal controllers, and then reveal their consistency with classical linear-quadratic results. In addition, the proposed analysis admits a unified computational formulation with a provably convergent iterative algorithm, enabling quantitative assessment of optimality robustness in nonlinear optimal control. Numerical examples validate the theoretical analysis, reveal its intrinsic connection with classical results, and demonstrate its practical computability.

• The paper develops a unified framework showing that the nominal optimal controller stabilizes the true system when the model-error budget satisfies c₁L + (c₂²L²)/(4α₂) < α₁.
• The paper derives worst-case bounds on value-function and controller deviations using a Hamilton–Jacobi–Isaacs formulation, with performance estimates tied to gradient-energy integrals along perturbed trajectories.
• The paper recovers classical LQR robustness and Riccati sensitivity behavior, including an observed first-order scaling of ||P − P₀|| with uncertainty magnitude (log-log slope ≈ 1.0236), and validates a convergent Galerkin-based algorithm numerically.

Problem setting and motivation

The paper develops a unified robustness analysis framework for data-driven nonlinear optimal control. The setting is an infinite-horizon optimal control problem for a continuous-time nonlinear system $\dot{x}=f(x,u)$ whose dynamics are unknown; instead, only a surrogate model $\hat{f}$ learned from data is available for controller synthesis. The actual system is written as $\dot{x}=\hat{f}(x,u)+r(x,u)$, where $r$ is the approximation error. The central question is how this model mismatch propagates through the optimal control structure: does it destroy closed-loop stability, and by how much does it degrade optimality in terms of both achieved cost and feedback law?

The authors position their contribution against two strands of prior work. Robustness of stability has been studied within method-specific frameworks (Koopman-based designs, GP-MPC, etc.), yielding guarantees tied to particular modeling assumptions. Robustness of optimality—the degradation of performance and controller when an approximately optimal policy is deployed on the true system—has been largely overlooked. In the LQR setting, classical perturbation theory of the algebraic Riccati equation (ARE) provides explicit first-order sensitivity results [kenney1988sensitivity, konstantinov1993perturbation], but no analogous characterization exists for nonlinear systems because the HJB equation lacks the explicit perturbation structure of the ARE.

The analysis rests on three standing assumptions: (i) $\mathcal{C}^2$ nominal dynamics and running cost with $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$; (ii) classical ($\mathcal{C}^2$) solutions to the actual and nominal HJB equations with unique minimizers; and (iii) a semi-linear growth bound on the mismatch, $\|r(x,u)\|\leq c_1\|x\|+c_2\|u\|$ with uniformly bounded coefficients. Assumption (iii) is justified both theoretically (bounded Jacobians on compact domains imply it via local smoothness) and practically, since proportional state- and input-dependent error bounds have been established for Koopman surrogates and kernel-based learners. Notably, the value-function regularity assumption is a genuine restriction: existence of unique $\mathcal{C}^2$ HJB solutions is guaranteed only locally near the origin under stabilizability, and extension to viscosity solutions is left as future work.

Robust stability from the nominal value function

The first main result establishes that the nominal optimal value function $V_0^*$ remains a valid Lyapunov function on the true system under a quantifiable criterion. Using Lipschitz continuity of $\hat{f}$0 (with constant $\hat{f}$1 derived from the bounded Hessian on the compact state space), the time derivative of $\hat{f}$2 along the mismatched closed loop is bounded via Young's inequality, and feasibility of the resulting conditions on the free parameter reduces exactly to

$\hat{f}$3

When this holds, the nominal optimal controller asymptotically stabilizes the true system for every admissible error in the set $\hat{f}$4. Two implications follow. First, optimality and stabilization are not antagonistic under model mismatch: sufficient surrogate accuracy suffices to retain stability without redesigning the controller. Second, the criterion functions as an "uncertainty budget"—an explicit threshold on $\hat{f}$5 that translates directly into a required modeling accuracy for data-driven identification pipelines. A corollary confirms that the actual optimal controller trivially stabilizes the true system, which is used later in the controller-deviation proof.

Quantifying optimality deviation

Performance loss

The paper defines the worst-case value function $\hat{f}$6, characterized through a Hamilton–Jacobi–Isaacs equation. Because $\hat{f}$7 is linear in $\hat{f}$8 over a norm-bounded set, the worst-case error aligns with the value gradient, $\hat{f}$9. Solving the resulting quadratic inequality in $\dot{x}=\hat{f}(x,u)+r(x,u)$0 yields an explicit bound $\dot{x}=\hat{f}(x,u)+r(x,u)$1 on the extra cost incurred by deploying $\dot{x}=\hat{f}(x,u)+r(x,u)$2 on the true system:

$\dot{x}=\hat{f}(x,u)+r(x,u)$3

expressed in terms of the scalar $\dot{x}=\hat{f}(x,u)+r(x,u)$4 and a gradient-energy integral $\dot{x}=\hat{f}(x,u)+r(x,u)$5 evaluated along the worst-case trajectory. A companion proposition shows this integral is automatically finite whenever the robust stability criterion holds, bounded as $\dot{x}=\hat{f}(x,u)+r(x,u)$6 where $\dot{x}=\hat{f}(x,u)+r(x,u)$7 is the decay rate margin. This closes the loop between the stability result and the optimality analysis: the same condition that guarantees stability also ensures finiteness of the performance-deviation bound. By symmetry, deviations in either direction (cost increase or decrease relative to nominal) satisfy the same bound.

Controller deviation

Explicit characterization of controller deviation requires an additional structural assumption: the surrogate (not necessarily the actual system) is control-affine, $\dot{x}=\hat{f}(x,u)+r(x,u)$8, with quadratic input cost. Under this structure, the actual optimal controller takes the form $\dot{x}=\hat{f}(x,u)+r(x,u)$9, and the deviation satisfies

$r$0

A key analytical device here is the intermediate worst-case value function $r$1, which bridges the implicitly coupled pairs $r$2 and $r$3 that otherwise cannot be connected directly across distinct HJB equations. The authors argue this construct is conceptually necessary rather than merely auxiliary. It should be noted that the controller result inherits the conservatism of the worst-case formulation and applies only to control-affine surrogates; the generality claim of the framework is therefore asymmetric between performance and controller analyses.

Consistency with LQR theory

Specializing the framework to LQR ($r$4) yields two instructive correspondences. For stability, the general criterion recovers a condition structurally parallel to Patel–Toda-type Lyapunov bounds [Patel1977robustness]: in the pure state-perturbation case the boundary matches up to the term $r$5 added inside $r$6, a slight conservatism the authors attribute to the price of a unified nonlinear analysis; in the pure input-perturbation case both conditions admit the same upper bound in terms of $r$7 and $r$8.

For optimality, exploiting the quadratic structure $r$9 converts the gradient-energy integral into a Lyapunov characterization via an auxiliary matrix $\mathcal{C}^2$0 solving $\mathcal{C}^2$1. This produces a matrix perturbation bound on $\mathcal{C}^2$2, whose first-order asymptotics give

$\mathcal{C}^2$3

structurally consistent with classical ARE sensitivity analysis [kenney1988sensitivity]. The paper emphasizes that although the endpoints coincide, the viewpoints differ fundamentally: classical methods start from explicit matrix perturbations of the ARE, whereas this framework derives Riccati perturbation as a corollary of a general nonlinear optimality-robustness theory. Numerically, on a benchmark with parameterized mismatches $\mathcal{C}^2$4, the actual $\mathcal{C}^2$5 remains strictly below the theoretical bound over the entire uncertainty range, and a log-log fit of $\mathcal{C}^2$6 versus $\mathcal{C}^2$7 yields a slope of approximately 1.0236, quantitatively confirming the predicted first-order law $\mathcal{C}^2$8.

Unified computation and convergence

All deviation quantities reduce to solving PDEs of a common Hamilton–Jacobi–Isaacs type. The paper exploits this to propose Algorithm 1: alternate between updating the worst-case error along the current value gradient and solving the resulting linear PDE for the value function update. Convergence is proven via monotonicity—the update direction maximizes $\mathcal{C}^2$9, so the sequence $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$0 is monotonically increasing—and upper-boundedness follows from robust stability of every admissible iterate, so the sequence converges to the solution of the HJI equation. Each linear PDE solve is implemented by Galerkin projection onto polynomial bases, inheriting standard numerical convergence guarantees.

Simulations on a control-affine nonlinear benchmark with analytically known ground truth ($l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$1, $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$2) validate the pipeline end-to-end. With deliberately perturbed regression weights emulating SINDy- or physics-informed-NN-style identification errors, the uncertainty coefficients admit explicit forms $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$3, $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$4. Three findings stand out: the Galerkin coefficients converge within a few iterations; the theoretical stability budget region correctly separates a convergent case (inside the region) from a divergent one (violating the criterion); and over the full state space the actual deviation $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$5, the computed worst-case deviation, and the analytical bound exhibit a strict ordering, with the controller-deviation bounding gap nonnegative everywhere. The authors acknowledge that worst-case bounds are inherently conservative relative to pointwise behavior—an intrinsic gap they do not attempt to tighten.

Limitations and open questions

Several restrictions qualify the results. The classical-solution assumption on the HJB equations confines the analysis locally near the origin; extension to viscosity solutions is explicitly deferred. The semi-linear growth bound excludes mismatch with superlinear growth relative to $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$6 and $l(x,u)\geq\alpha_1\|x\|^2+\alpha_2\|u\|^2$7, and its coefficients must be estimated from data, introducing a further source of conservatism not analyzed here. The controller-deviation theorem requires a control-affine surrogate and quadratic input penalty, so fully general nonlinear actuation remains uncharacterized. Finally, all bounds are worst-case; whether tighter, distributionally aware characterizations of optimality deviation are possible under probabilistic error certificates is left open, as is the treatment of large-scale systems and adaptive/learning-based controller architectures mentioned in the conclusion.

Conclusion

This paper provides a self-contained theory connecting data-driven modeling accuracy to closed-loop guarantees in nonlinear optimal control: a quantitative stability criterion on the error bounds, explicit hierarchical characterizations of performance and controller deviation built on a worst-case value function, provable recovery of classical LQR robustness and Riccati sensitivity structures, and a convergent iterative algorithm making the analysis computationally tractable. Its principal limitation is the conservatism inherent to worst-case analysis combined with local regularity assumptions, which defines the natural scope within which the reported guarantees hold.