Kernel-Based LMI Approaches to Solving the Hamilton-Jacobi-Bellman Equation and Nonlinear Optimal Control

Abstract: We present a kernel-based linear matrix inequality (LMI) approach for solving Hamilton-Jacobi-Bellman (HJB) equations arising in nonlinear optimal control. The method converts the nonlinear HJB inequality into a convex semidefinite program through reproducing kernel Hilbert space (RKHS) representations and Schur complement reformulations. We provide complete theoretical results including convex reformulation, RKHS approximation, finite-dimensional discretization, suboptimality bounds, stability guarantees, and convergence rates. A critical component of our approach is the use of a Riccati Hessian constraint at the equilibrium to prevent trivial solutions while ensuring consistency with linearized optimal control theory. Numerical results on both 1D and 2D systems demonstrate the effectiveness of the approach, with comprehensive validation through multiple initial conditions showing that all closed-loop trajectories converge exponentially to the origin from various starting points, successfully stabilizing unstable equilibria with theoretical guarantees. The method maintains computational tractability while providing rigorous optimality and stability guarantees.