Papers
Topics
Authors
Recent
Search
2000 character limit reached

Model-free Resilient Controller Design based on Incentive Feedback Stackelberg Game and Q-learning

Published 13 Mar 2024 in eess.SY, cs.GT, and cs.SY | (2403.08948v1)

Abstract: In the swift evolution of Cyber-Physical Systems (CPSs) within intelligent environments, especially in the industrial domain shaped by Industry 4.0, the surge in development brings forth unprecedented security challenges. This paper explores the intricate security issues of Industrial CPSs (ICPSs), with a specific focus on the unique threats presented by intelligent attackers capable of directly compromising the controller, thereby posing a direct risk to physical security. Within the framework of hierarchical control and incentive feedback Stackelberg game, we design a resilient leading controller (leader) that is adaptive to a compromised following controller (follower) such that the compromised follower acts cooperatively with the leader, aligning its strategies with the leader's objective to achieve a team-optimal solution. First, we provide sufficient conditions for the existence of an incentive Stackelberg solution when system dynamics are known. Then, we propose a Q-learning-based Approximate Dynamic Programming (ADP) approach, and corresponding algorithms for the online resolution of the incentive Stackelberg solution without requiring prior knowledge of system dynamics. Last but not least, we prove the convergence of our approach to the optimum.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (27)
  1. B. Dafflon, N. Moalla, and Y. Ouzrout, “The challenges, approaches, and used techniques of cps for manufacturing in industry 4.0: A literature review,” The International Journal of Advanced Manufacturing Technology, vol. 113, pp. 2395–2412, 2021.
  2. X. Ji, G. He, J. Xu, and Y. Guo, “Study on the mode of intelligent chemical industry based on cyber-physical system and its implementation,” Advances in Engineering software, vol. 99, pp. 18–26, 2016.
  3. H. Zhang, B. Liu, and H. Wu, “Smart grid cyber-physical attack and defense: A review,” IEEE Access, vol. 9, pp. 29 641–29 659, 2021.
  4. G. Narula, P. Nagrath, D. Hans, and A. Nayyar, “Novel defending and prevention technique for man-in-the-middle attacks in cyber-physical networks,” Cyber-Physical Systems: Foundations and Techniques, pp. 147–177, 2022.
  5. K. Evers, R. Oram, S. El-Tawab, M. H. Heydari, and B. B. Park, “Security measurement on a cloud-based cyber-physical system used for intelligent transportation,” in 2017 IEEE International Conference on Vehicular Electronics and Safety (ICVES).   IEEE, 2017, pp. 97–102.
  6. A. E. Elhabashy, L. J. Wells, and J. A. Camelio, “Cyber-physical security research efforts in manufacturing–a literature review,” Procedia manufacturing, vol. 34, pp. 921–931, 2019.
  7. Z. Kazemi, A. A. Safavi, M. M. Arefi, and F. Naseri, “Finite-time secure dynamic state estimation for cyber–physical systems under unknown inputs and sensor attacks,” IEEE transactions on systems, man, and cybernetics: systems, vol. 52, no. 8, pp. 4950–4959, 2021.
  8. Z. Li and Y. Mo, “Efficient secure state estimation against sparse integrity attack for regular linear system,” International Journal of Robust and Nonlinear Control, vol. 33, no. 1, pp. 209–236, 2023.
  9. Y. Yuan, Q. Zhu, F. Sun, Q. Wang, and T. Başar, “Resilient control of cyber-physical systems against denial-of-service attacks,” in 2013 6th International Symposium on Resilient Control Systems basar1979closed(ISRCS).   IEEE, 2013, pp. 54–59.
  10. Y. Yuan, F. Sun, and H. Liu, “Resilient control of cyber-physical systems against intelligent attacker: a hierarchal stackelberg game approach,” International Journal of Systems Science, vol. 47, no. 9, pp. 2067–2077, 2016.
  11. Q. Sun, K. Zhang, and Y. Shi, “Resilient model predictive control of cyber–physical systems under dos attacks,” IEEE Transactions on Industrial Informatics, vol. 16, no. 7, pp. 4920–4927, 2019.
  12. Y. Zhao, X. Du, C. Zhou, Y.-C. Tian, X. Hu, and D. E. Quevedo, “Adaptive resilient control of cyber-physical systems under actuator and sensor attacks,” IEEE Transactions on Industrial Informatics, vol. 18, no. 5, pp. 3203–3212, 2021.
  13. M. Cook, A. Marnerides, C. Johnson, and D. Pezaros, “A survey on industrial control system digital forensics: challenges, advances and future directions,” IEEE Communications Surveys & Tutorials, 2023.
  14. H. R. Ghaeini and N. O. Tippenhauer, “Hamids: Hierarchical monitoring intrusion detection system for industrial control systems,” in Proceedings of the 2nd ACM Workshop on Cyber-Physical Systems Security and Privacy, 2016, pp. 103–111.
  15. J. R. Hicks, “Marktform und gleichgewicht,” 1935.
  16. Y.-C. Ho, P. B. Luh, and G. J. Olsder, “A control-theoretic view on incentives,” Automatica, vol. 18, no. 2, pp. 167–179, 1982.
  17. M. Ahmed, H. Mukaidani, and T. Shima, “-constrained incentive stackelberg games for discrete-time stochastic systems with multiple followers,” IET Control Theory & Applications, vol. 11, no. 15, pp. 2475–2485, 2017.
  18. H. Mukaidani, R. Saravanakumar, and H. Xu, “Robust incentive stackelberg strategy for markov jump linear stochastic systems via static output feedback,” IET Control Theory & Applications, vol. 14, no. 9, pp. 1246–1254, 2020.
  19. T. Basar and H. Selbuz, “Closed-loop stackelberg strategies with applications in the optimal control of multilevel systems,” IEEE Transactions on Automatic Control, vol. 24, no. 2, pp. 166–179, 1979.
  20. M. Li, J. Cruz, and M. A. Simaan, “An approach to discrete-time incentive feedback stackelberg games,” IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 32, no. 4, pp. 472–481, 2002.
  21. Y. Lin, W. Gao, and W. Zhang, “Incentive feedback stackelberg strategy for stochastic systems with state-dependent noise,” Journal of the Franklin Institute, vol. 359, no. 5, pp. 2058–2072, 2022.
  22. J. Medanic, “Closed-loop stackelberg strategies in linear-quadratic problems,” IEEE Transactions on Automatic Control, vol. 23, no. 4, pp. 632–637, 1978.
  23. H. Mukaidani and H. Xu, “Incentive stackelberg games for stochastic linear systems with h-inf constraint,” IEEE Transactions on Cybernetics, vol. 49, no. 4, pp. 1463–1474, 2018.
  24. J. Brewer, “Kronecker products and matrix calculus in system theory,” IEEE Transactions on circuits and systems, vol. 25, no. 9, pp. 772–781, 1978.
  25. M. Z. Nashed and G. Wahba, “Convergence rates of approximate least squares solutions of linear integral and operator equations of the first kind,” Mathematics of Computation, vol. 28, no. 125, pp. 69–80, 1974.
  26. E. Fogel, “A fundamental approach to the convergence analysis of least squares algorithms,” IEEE Transactions on Automatic Control, vol. 26, no. 3, pp. 646–655, 1981.
  27. D. V. Prokhorov and D. C. Wunsch, “Adaptive critic designs,” IEEE transactions on Neural Networks, vol. 8, no. 5, pp. 997–1007, 1997.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free