Self-Tuning PID Control via a Hybrid Actor-Critic-Based Neural Structure for Quadcopter Control (2307.01312v1)
Abstract: Proportional-Integrator-Derivative (PID) controller is used in a wide range of industrial and experimental processes. There are a couple of offline methods for tuning PID gains. However, due to the uncertainty of model parameters and external disturbances, real systems such as Quadrotors need more robust and reliable PID controllers. In this research, a self-tuning PID controller using a Reinforcement-Learning-based Neural Network for attitude and altitude control of a Quadrotor has been investigated. An Incremental PID, which contains static and dynamic gains, has been considered and only the variable gains have been tuned. To tune dynamic gains, a model-free actor-critic-based hybrid neural structure was used that was able to properly tune PID gains, and also has done the best as an identifier. In both tunning and identification tasks, a Neural Network with two hidden layers and sigmoid activation functions has been learned using Adaptive Momentum (ADAM) optimizer and Back-Propagation (BP) algorithm. This method is online, able to tackle disturbance, and fast in training. In addition to robustness to mass uncertainty and wind gust disturbance, results showed that the proposed method had a better performance when compared to a PID controller with constant gains.
- J. Kim, S. A. Gadsden, and S. A. Wilkerson, “A comprehensive survey of control strategies for autonomous quadrotors,” Canadian Journal of Electrical and Computer Engineering, vol. 43, no. 1, pp. 3–16, 2019.
- A. Zulu and S. John, “A review of control algorithms for autonomous quadrotors,” arXiv preprint arXiv:1602.02622, 2016.
- T.-h. Pan and S.-y. Li, “Adaptive pid control for nonlinear systems based on lazy learning,” Control Theory & Applications, vol. 10, p. 029, 2009.
- J. Yang, Z. Cai, Q. Lin, and Y. Wang, “Self-tuning pid control design for quadrotor uav based on adaptive pole placement control,” in 2013 Chinese Automation Congress, pp. 233–237, IEEE, 2013.
- A. Goel, A. M. Salim, A. Ansari, S. Ravela, and D. Bernstein, “Adaptive digital pid control of a quadcopter with unknown dynamics,” arXiv preprint arXiv:2006.00416, 2020.
- R. Hernández-Alvarado, L. G. García-Valdovinos, T. Salgado-Jiménez, A. Gómez-Espinosa, and F. Fonseca-Navarro, “Neural network-based self-tuning pid control for underwater vehicles,” Sensors, vol. 16, no. 9, p. 1429, 2016.
- S. Bari, S. S. Z. Hamdani, H. U. Khan, M. ur Rehman, and H. Khan, “Artificial neural network based self-tuned pid controller for flight control of quadcopter,” in 2019 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–5, IEEE, 2019.
- D. Park, T.-L. Le, N. V. Quynh, N. K. Long, and S. K. Hong, “Online tuning of pid controller using a multilayer fuzzy neural network design for quadcopter attitude tracking control,” Frontiers in Neurorobotics, p. 118, 2021.
- O. I. Abiodun, A. Jantan, A. E. Omolara, K. V. Dada, N. A. Mohamed, and H. Arshad, “State-of-the-art in artificial neural network applications: A survey,” Heliyon, vol. 4, no. 11, p. e00938, 2018.
- I. Carlucho, M. De Paula, S. A. Villar, and G. G. Acosta, “Incremental q-learning strategy for adaptive pid control of mobile robots,” Expert Systems with Applications, vol. 80, pp. 183–199, 2017.
- Q. Shi, H.-K. Lam, B. Xiao, and S.-H. Tsai, “Adaptive pid controller based on q-learning algorithm,” CAAI Transactions on Intelligence Technology, vol. 3, no. 4, pp. 235–244, 2018.
- Z. Guan and T. Yamamoto, “Design of a reinforcement learning pid controller,” IEEJ Transactions on Electrical and Electronic Engineering, vol. 16, no. 10, pp. 1354–1360, 2021.
- N. P. Lawrence, M. G. Forbes, P. D. Loewen, D. G. McClement, J. U. Backström, and R. B. Gopaluni, “Deep reinforcement learning with shallow controllers: An experimental application to pid tuning,” Control Engineering Practice, vol. 121, p. 105046, 2022.
- J. Yang, D. Chu, W. Peng, C. Sun, Z. Deng, L. Lu, and C. Wu, “A learning control method of automated vehicle platoon at straight path with ddpg-based pid,” Electronics, vol. 10, no. 21, p. 2580, 2021.
- Q. Sun, C. Du, Y. Duan, H. Ren, and H. Li, “Design and application of adaptive pid controller based on asynchronous advantage actor–critic learning method,” Wireless Networks, vol. 27, no. 5, pp. 3537–3547, 2021.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
- V. K. Tripathi, L. Behera, and N. Verma, “Design of sliding mode and backstepping controllers for a quadcopter,” in 2015 39th national systems conference (NSC), pp. 1–6, IEEE, 2015.
- R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018.
- L. Ding, Q. He, C. Wang, and R. Qi, “Disturbance rejection attitude control for a quadrotor: Theory and experiment,” International Journal of Aerospace Engineering, vol. 2021, 2021.