Learning Recovery Strategies for Dynamic Self-healing in Reactive Systems (2401.12405v1)
Abstract: Self-healing systems depend on following a set of predefined instructions to recover from a known failure state. Failure states are generally detected based on domain specific specialized metrics. Failure fixes are applied at predefined application hooks that are not sufficiently expressive to manage different failure types. Self-healing is usually applied in the context of distributed systems, where the detection of failures is constrained to communication problems, and resolution strategies often consist of replacing complete components. Our proposal targets complex reactive systems, defining monitors as predicates specifying satisfiability conditions of system properties. Such monitors are functionally expressive and can be defined at run time to detect failure states at any execution point. Once failure states are detected, we use a Reinforcement Learning-based technique to learn a recovery strategy based on users' corrective sequences. Finally, to execute the learned strategies, we extract them as COP variations that activate dynamically whenever the failure state is detected, overwriting the base system behavior with the recovery strategy for that state. We validate the feasibility and effectiveness of our framework through a prototypical reactive application for tracking mouse movements, and the DeltaIoT exemplar for self-healing systems. Our results demonstrate that with just the definition of monitors, the system is effective in detecting and recovering from failures between 55%-92% of the cases in the first application, and at par with the predefined strategies in the second application.
- “Self-healing network for scalable fault-tolerant runtime environments” In Future Generation Computer Systems 26.3, 2010, pp. 479–485 DOI: https://doi.org/10.1016/j.future.2009.04.001
- “Generating Software Adaptations using Machine Learning” In Workshop on Machine Learning for Programming Languages, ML4PL’18, 2018, pp. 1–2
- “Next Generation Context-oriented Programming: Embracing Dynamic Generation of Adaptations” In Jour. of Object Technology 21.2, 2022, pp. 1–6 DOI: http://dx.doi.org/10.5381/jot.2022.21.2.a5
- “Auto-COP: Adaptation Generation in Context-Oriented Programming using Reinforcement Learning Options” In Information and Software Technology 164, 2023
- Nicolás Cardozo, Ivana Dusparic and Jorge H Castro “Peace COrP: Learning to solve conflicts between contexts” In Proc. of the 9th Intl. Workshop on Context-Oriented Programming, 2017, pp. 1–6
- Yuanshun Dai, Yanping Xiang and Gewei Zhang “Self-healing and Hybrid Diagnosis in Cloud Computing” In Cloud Computing Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 45–56
- Eric M Dashofy, André Van der Hoek and Richard N Taylor “Towards architecture-based self-healing systems” In Proc. of the first workshop on Self-healing systems, 2002, pp. 21–26
- Francis M David and Roy H Campbell “Building a self-healing operating system” In Third IEEE Intl. Symp. on Dependable, Autonomic and Secure Computing (DASC 2007), 2007, pp. 3–10 IEEE
- Edsger W. Dijkstra “Self-Stabilizing Systems in Spite of Distributed Control” In Commun. ACM 17.11 New York, NY, USA: Association for Computing Machinery, 1974, pp. 643–644 DOI: 10.1145/361179.361202
- Bahadir Dundar, Merve Astekin and Mehmet S Aktas “A big data processing framework for self-healing internet of things applications” In Intl. Conf. on Semantics, Knowledge and Grids, SKG, 2016, pp. 62–68 IEEE
- “Functional Reactive Animation” In Intl. Conf. on Functional Programming, 1997 URL: http://conal.net/papers/icfp97/
- “Model-Based Adaptation for Self-Healing Systems” In Proc. of the First Workshop on Self-Healing Systems, WOSS ’02 Charleston, South Carolina: Association for Computing Machinery, 2002, pp. 27–32 DOI: 10.1145/582128.582134
- “Option discovery in reinforcement learning using frequent common subsequences of actions” In Intl. Conf. on Computational Intelligence for Modelling, Control and Automation and Intl. Conf. on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06) 1, 2005, pp. 371–376 IEEE
- Robert Hirschfeld, Pascal Costanza and Oscar Nierstrasz “Context-oriented programming” In Jour. of Object technology 7.3 AITO, 2008, pp. 125–151
- “Deltaiot: A self-adaptive internet of things exemplar” In 2017 IEEE/ACM 12th Intl. Symp. on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2017, pp. 76–82 IEEE
- R. Kawamura, K.-I. Sato and I. Tokizawa “Self-healing ATM networks based on virtual path concept” In IEEE Jour. on Selected Areas in Communications 12.1, 1994, pp. 120–127 DOI: 10.1109/49.265711
- Philip Koopman “Elements of the Self-Healing System Problem Space”, 2003
- Henri Naccache and Gerald C Gannod “A self-healing framework for web services” In IEEE Intl. Conf. on Web Services (ICWS 2007), 2007, pp. 398–345 IEEE
- “A survey on self-healing systems: approaches and systems” In Computing 91.1 Springer ScienceBusiness Media LLC, 2010, pp. 43–73 DOI: 10.1007/s00607-010-0107-y
- Jette Randlov “Learning macro-actions in reinforcement learning” In Advances in Neural Information Processing Systems 11, 1998
- R. Razavi, S. Klein and H. Claussen “Self-optimization of capacity and coverage in LTE networks using a fuzzy reinforcement learning approach” In IEEE Intl. Symp. on Personal, Indoor and Mobile Radio Communications, 2010, pp. 1865–1870 DOI: 10.1109/PIMRC.2010.5671622
- “Self-Healing Systems: Foundations and Challenges.” In Self-Healing and Self-Adaptive Systems, 2009
- Sutton R. S. and Barto A. G. “Reinforcement Learning: An Introduction.” Bradford Book. The MIT Press, Cambridge, Massachusetts, 1998
- Arsalan Saeed, Osianoh Glenn Aliu and Muhammad Ali Imran “Controlling self healing cellular networks using fuzzy logic” In IEEE Wireless Communications and Networking Conf., WCNC’12, 2012, pp. 3080–3084 DOI: 10.1109/WCNC.2012.6214334
- Guido Salvaneschi, Carlo Ghezzi and Matteo Pradella “Context-oriented Programming: A Software Engineering Perspective” In Jour. of Systems and Software 85.8, 2012, pp. 1801–1817
- Guido Salvaneschi, Carlo Ghezzi and Matteo Pradella “ContextErlang: A language for distributed context-aware self-adaptive applications” In Science of Computer Programming 102 Elsevier, 2015, pp. 20–43
- Guido Salvaneschi, Gerold Hintz and Mira Mezini “REScala: Bridging between object-oriented and functional style in reactive applications” In Proc. of the 13th Intl. Conf. on Modularity, 2014, pp. 25–36
- Chris Schneider, Adam Barker and Simon Dobson “A survey of self-healing systems frameworks” In Software: Practice and Experience 45.10 Wiley Online Library, 2015, pp. 1375–1398
- “Learning options in reinforcement learning” In Intl. Symp. on abstraction, reformulation, and approximation, 2002, pp. 212–223 Springer
- Richard S Sutton, Doina Precup and Satinder P Singh “Intra-Option Learning about Temporally Abstract Actions.” In ICML 98, 1998, pp. 556–564
- Amitabh Trehan “Algorithms for Self-Healing Networks” In CoRR abs/1305.4675, 2013 arXiv: http://arxiv.org/abs/1305.4675
- “Functional reactive programming from first principles” In Proc. of the Conf. on Programming language design and implementation, 2000, pp. 242–252
- Danny Weyns “Software engineering of self-adaptive systems: an organised tour and future challenges” In Chapter in Handbook of Software Engineering Springer, 2017, pp. 2
- “Using Adaptive Neural Networks in Self-Healing Systems” In 2009 Second Intl. Conf. on Developments in eSystems Engineering, 2009, pp. 227–232 DOI: 10.1109/DeSE.2009.55
- “A meta reinforcement learning-based approach for self-adaptive system” In 2021 IEEE Intl. Conf. on Autonomic Computing and Self-Organizing Systems (ACSOS), 2021, pp. 1–10 IEEE
- “A reinforcement learning-based framework for the generation and evolution of adaptation rules” In 2017 IEEE Intl. Conf. on Autonomic Computing (ICAC), 2017, pp. 103–112 IEEE
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
