Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Lyapunov-Based Deep Residual Neural Network (ResNet) Adaptive Control (2404.07385v1)

Published 10 Apr 2024 in eess.SY and cs.SY

Abstract: Deep Neural Network (DNN)-based controllers have emerged as a tool to compensate for unstructured uncertainties in nonlinear dynamical systems. A recent breakthrough in the adaptive control literature provides a Lyapunov-based approach to derive weight adaptation laws for each layer of a fully-connected feedforward DNN-based adaptive controller. However, deriving weight adaptation laws from a Lyapunov-based analysis remains an open problem for deep residual neural networks (ResNets). This paper provides the first result on Lyapunov-derived weight adaptation for a ResNet-based adaptive controller. A nonsmooth Lyapunov-based analysis is provided to guarantee asymptotic tracking error convergence. Comparative Monte Carlo simulations are provided to demonstrate the performance of the developed ResNet-based adaptive controller. The ResNet-based adaptive controller shows a 64% improvement in the tracking and function approximation performance, in comparison to a fully-connected DNN-based adaptive controller.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. P. Kidger and T. Lyons, “Universal approximation with deep narrow networks,” in Conf. Learn. Theory, pp. 2306–2327, 2020.
  2. Cambridge University Press, 2019.
  3. F. Lewis, A. Yesildirek, and K. Liu, “Multilayer neural net robot controller: structure and stability proofs,” IEEE Trans. Neural Netw., vol. 7, no. 2, pp. 388–399, 1996.
  4. Philadelphia, PA: CRC Press, 1998.
  5. Boston, MA: Kluwer Academic Publishers, 2002.
  6. P. M. Patre, W. MacKunis, K. Kaiser, and W. E. Dixon, “Asymptotic tracking for uncertain dynamic systems via a multilayer neural network feedforward and RISE feedback control structure,” IEEE Trans. Autom. Control, vol. 53, no. 9, pp. 2180–2185, 2008.
  7. P. Patre, S. Bhasin, Z. D. Wilcox, and W. E. Dixon, “Composite adaptation for neural network-based controllers,” IEEE Trans. Autom. Control, vol. 55, no. 4, pp. 944–950, 2010.
  8. R. Sun, M. Greene, D. Le, Z. Bell, G. Chowdhary, and W. E. Dixon, “Lyapunov-based real-time and iterative adjustment of deep neural networks,” IEEE Control Syst. Lett., vol. 6, pp. 193–198, 2022.
  9. G. Joshi and G. Chowdhary, “Deep model reference adaptive control,” in Proc. IEEE Conf. Decis. Control, pp. 4601–4608, 2019.
  10. G. Joshi, J. Virdi, and G. Chowdhary, “Asynchronous deep model reference adaptive control,” in Conf. Robot Learn., 2020.
  11. D. Le, M. Greene, W. Makumi, and W. E. Dixon, “Real-time modular deep neural network-based adaptive control of nonlinear systems,” IEEE Control Syst. Lett., vol. 6, pp. 476–481, 2022.
  12. O. Patil, D. Le, M. Greene, and W. E. Dixon, “Lyapunov-derived control and adaptive update laws for inner and outer layer weights of a deep neural network,” IEEE Control Syst Lett., vol. 6, pp. 1855–1860, 2022.
  13. MIT press Cambridge, 2016.
  14. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 770–778, 2016.
  15. M. Hardt and T. Ma, “Identity matters in deep learning,” Int. Conf. Learn. Represent., 2017.
  16. K. Nar and S. Sastry, “Residual networks: Lyapunov stability and convex decomposition,” arXiv preprint arXiv:1803.08203, 2018.
  17. Y. Tai, J. Yang, and X. Liu, “Image super-resolution via deep recursive residual network,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 3147–3155, 2017.
  18. J. Li, F. Fang, K. Mei, and G. Zhang, “Multi-scale residual network for image super-resolution,” in Proc. Eur. Conf. Comput. Vis., pp. 517–532, 2018.
  19. M. Boroumand, M. Chen, and J. Fridrich, “Deep residual network for steganalysis of digital images,” IEEE Trans. Inf. Forensics Secur., vol. 14, no. 5, pp. 1181–1193, 2018.
  20. T. Tan, Y. Qian, H. Hu, Y. Zhou, W. Ding, and K. Yu, “Adaptive very deep convolutional residual network for noise robust speech recognition,” IEEE/ACM Trans. Audio, Speech, Language Process., vol. 26, no. 8, pp. 1393–1405, 2018.
  21. O. S. Patil, D. M. Le, E. Griffis, and W. E. Dixon, “Deep residual neural network (ResNet)-based adaptive control: A Lyapunov-based approach,” in Proc. IEEE Conf. Decis. Control, 2022.
  22. B. E. Paden and S. S. Sastry, “A calculus for computing Filippov’s differential inclusion with application to the variable structure control of robot manipulators,” IEEE Trans. Circuits Syst., vol. 34, pp. 73–82, Jan. 1987.
  23. D. S. Bernstein, Matrix Mathematics. Princeton university press, 2009.
  24. H. Lin and S. Jegelka, “ResNet with one-neuron hidden layers is a universal approximator,” Adv. Neural Inf. Process. Syst., vol. 31, 2018.
  25. P. Tabuada and B. Gharesifard, “Universal approximation power of deep residual neural networks via nonlinear control theory,” in Int. Conf. Learn. Represent., 2020.
  26. F. L. Lewis, A. Yegildirek, and K. Liu, “Multilayer neural-net robot controller with guaranteed tracking performance,” IEEE Trans. Neural Netw., vol. 7, pp. 388–399, Mar. 1996.
  27. A. M. Annaswamy, F. P. Skantze, and A.-P. Loh, “Adaptive control of continuous time systems with convex/concave parametrization,” Automatica, vol. 34, no. 1, pp. 33–49, 1998.
  28. A. Kojić, A. M. Annaswamy, A.-P. Loh, and R. Lozano, “Adaptive control of a class of nonlinear systems with convex/concave parameterization,” Syst. Control Lett., vol. 37, no. 5, pp. 267–274, 1999.
  29. W. Lin and C. Qian, “Adaptive control of nonlinearly parameterized systems: the smooth feedback case,” IEEE Trans. Autom. Control, vol. 47, no. 8, pp. 1249–1266, 2002.
  30. W. Lin and C. Qian, “Adaptive control of nonlinearly parameterized systems: a nonsmooth feedback framework,” IEEE Trans. Autom. Control, vol. 47, pp. 757–774, May 2002.
  31. Z. Qu, R. A. Hull, and J. Wang, “Globally stabilizing adaptive control design for nonlinearly-parameterized systems,” IEEE Trans. Autom. Control, vol. 51, pp. 1073–1079, June 2006.
  32. S. B. Roy, S. Bhasin, and I. N. Kar, “Robust gradient-based adaptive control of nonlinearly parametrized plants,” IEEE Control Syst. Lett., vol. 1, no. 2, pp. 352–357, 2017.
  33. N. Fischer, R. Kamalapurkar, and W. E. Dixon, “LaSalle-Yoshizawa corollaries for nonsmooth systems,” IEEE Trans. Autom. Control, vol. 58, pp. 2333–2338, Sep. 2013.
  34. F. H. Clarke, Optimization and nonsmooth analysis. SIAM, 1990.
  35. D. Shevitz and B. Paden, “Lyapunov stability theory of nonsmooth systems,” IEEE Trans. Autom. Control, vol. 39 no. 9, pp. 1910–1914, 1994.
  36. P. Ioannou and J. Sun, Robust Adaptive Control. Prentice Hall, 1996.
  37. E. Griffis, O. Patil, Z. Bell, and W. E. Dixon, “Lyapunov-based long short-term memory (Lb-LSTM) neural network-based control,” IEEE Control Syst. Lett., vol. 7, pp. 2976–2981, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets