Stabilizing Optimal Control for Nonlinear Stochastic Systems: A Parametric Gradient-Based Approach

Abstract: This study proposes a method for designing stabilizing suboptimal controllers for nonlinear stochastic systems. These systems include time-invariant stochastic parameters that represent uncertainty of dynamics, posing two key difficulties in optimal control. Firstly, the time-invariant stochastic nature violates the principle of optimality and Hamilton-Jacobi equations, which are fundamental tools for solving optimal control problems. Secondly, nonlinear systems must be robustly stabilized against these stochastic parameters. To overcome these difficulties simultaneously, this study presents a parametric-gradient-based method with a penalty function. A controller and cost function are parameterized using basis functions, and a gradient method is employed to optimize the controller by minimizing the parameterized cost function. Crucial challenges in this approach are parameterizing the cost function appropriately and deriving the gradient of the cost. This study provides explicit formulations of an optimally parameterized cost and its gradient. Furthermore, a suitable penalty function is proposed to ensure robust stability, even when using the gradient method. Consequently, the gradient method produces a suboptimal feedback controller that guarantees the robust stability. The effectiveness of the proposed method is demonstrated through numerical simulations, highlighting its performance in comparison with other baseline methods.