Robust Optimal Safe and Stability Guaranteeing Reinforcement Learning Control for Quadcopter

Abstract: Recent advances in deep learning have provided new data-driven ways of controller design to replace the traditional manual synthesis and certification approaches. Employing neural network (NN) as controllers however, presents its own challenge: that of certifying stability due to their inherent complex nonlinearity, and while NN controllers have demonstrated high performance in complex systems, they often lack formal stability guarantees. This issue is further accentuated for critical nonlinear applications such as of unmanned aerial vehicles (UAVs), complicating their stability guarantees, whereas a lack of stability assurance raises the risk of critical damage or even complete failure under a loss of control. In this study, we improve a Robust, Optimal, Safe and Stability Guaranteed Training (ROSS-GT) method of [1] to design an NN controller for a quadcopter flight control. The approach ensures closed-loop system stability by finding a Lyapunov function, and providing a safe initial state domain that remains invariant under the control and guarantees stability to an equilibrium within it. Stability guaranteeing constraints are derived from the sector bound of the system nonlinearity and of its parameters and disturbance variations, in the form of a Lipschitz bound for a NN control. The control performance is further optimized by searching over the class of stability-guaranteeing controllers to minimize the reference tracking error and the control costs.