Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Repairing Learning-Enabled Controllers While Preserving What Works (2311.03477v2)

Published 6 Nov 2023 in eess.SY and cs.SY

Abstract: Learning-enabled controllers have been adopted in various cyber-physical systems (CPS). When a learning-enabled controller fails to accomplish its task from a set of initial states, researchers leverage repair algorithms to fine-tune the controller's parameters. However, existing repair techniques do not preserve previously correct behaviors. Specifically, when modifying the parameters to repair trajectories from a subset of initial states, another subset may be compromised. Therefore, the repair may break previously correct scenarios, introducing new risks that may not be accounted for. Due to this issue, repairing the entire initial state space may be hard or even infeasible. As a response, we formulate the Repair with Preservation (RwP) problem, which calls for preserving the already-correct scenarios during repair. To tackle this problem, we design the Incremental Simulated Annealing Repair (ISAR) algorithm, which leverages simulated annealing on a barriered energy function to safeguard the already-correct initial states while repairing as many additional ones as possible. Moreover, formal verification is utilized to guarantee the repair results. Case studies on an Unmanned Underwater Vehicle (UUV) and OpenAI Gym Mountain Car (MC) show that ISAR not only preserves correct behaviors from previously verified initial state regions, but also repairs 81.4% and 23.5% of broken state spaces in the two benchmarks. Moreover, the average signal temporal logic (STL) robustnesses of the ISAR repaired controllers are larger than those of the controllers repaired using baseline methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Openai gym. arXiv preprint arXiv:1606.01540, 2016.
  2. Custom memory management methodology: Exploration of memory organisation for embedded multimedia system design. Springer Science & Business Media, 2013.
  3. Lifelong machine learning, volume 1. Springer, 2018.
  4. Automated repair of neural networks. arXiv preprint arXiv:2207.08157, 2022.
  5. On the classical logarithmic barrier function method for a class of smooth convex programming problems. Journal of Optimization Theory and Applications, 73(1):1–25, 1992.
  6. Towards repairing neural networks correctly. arXiv preprint arXiv:2012.01872, 2020.
  7. Optimization of anemia treatment in hemodialysis patients via reinforcement learning. Artificial intelligence in medicine, 62(1):47–60, 2014.
  8. Robustness of temporal logic specifications for continuous-time signals. Theoretical Computer Science, 410(42):4262–4291, 2009.
  9. Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128–135, 1999.
  10. Sound and complete neural network repair with minimality and locality guarantees. arXiv preprint arXiv:2110.07682, 2021.
  11. Reglo: Provable neural network repair for global robustness properties. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022, 2022.
  12. A smooth robustness measure of signal temporal logic for symbolic control. IEEE Control Systems Letters, 5(1):241–246, 2020.
  13. Control from signal temporal logic specifications with smooth cumulative quantitative semantics. In 2019 IEEE 58th Conference on Decision and Control (CDC), pages 4361–4366. IEEE, 2019.
  14. Training agents to satisfy timed and untimed signal temporal logic specifications with reinforcement learning. In International Conference on Software Engineering and Formal Methods, pages 190–206. Springer, 2022.
  15. A barrier function method for the optimization of trajectory functionals with constraints. In Proceedings of the 45th IEEE Conference on Decision and Control, pages 864–869. IEEE, 2006.
  16. Service robots in the healthcare sector. Robotics, 10(1):47, 2021.
  17. Verisig 2.0: Verification of neural network controllers using taylor model preconditioning. In International Conference on Computer Aided Verification, pages 249–262. Springer, 2021.
  18. Verisig: verifying safety properties of hybrid systems with neural network controllers. In Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control, pages 169–178, 2019.
  19. Optimization by simulated annealing. science, 220(4598):671–680, 1983.
  20. Bing Liu. Lifelong machine learning: a paradigm for continuous learning. Frontiers of Computer Science, 11:359–361, 2017.
  21. Causal repair of learning-enabled cyber-physical systems. In 2023 IEEE International Conference on Assured Autonomy (ICAA), pages 1–10, 2023.
  22. Autorepair: Automated repair for ai-enabled cyber-physical systems under safety-critical conditions. arXiv preprint arXiv:2304.05617, 2023.
  23. Local repair of neural networks using optimization. arXiv preprint arXiv:2109.14041, 2021.
  24. Monitoring temporal properties of continuous signals. In International Symposium on Formal Techniques in Real-Time and Fault-Tolerant Systems, pages 152–166. Springer, 2004.
  25. Enabling fully automated insulin delivery through meal detection and size estimation using artificial intelligence. npj Digital Medicine, 6(1):39, 2023.
  26. Roman Polyak. Modified barrier functions (theory and methods). Mathematical programming, 54:177–222, 1992.
  27. Advantages of monte carlo confidence intervals for indirect effects. Communication Methods and Measures, 6(2):77–98, 2012.
  28. Monte carlo integration. Monte Carlo statistical methods, pages 71–138, 1999.
  29. Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123–146, 1995.
  30. Confidence composition for monitors of verification assumptions. In 2022 ACM/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS), pages 1–12. IEEE, 2022.
  31. Active task selection for lifelong machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 27, pages 862–868, 2013.
  32. Safe-by-repair: A convex optimization approach for repairing unsafe two-level lattice neural network controllers. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 3383–3388. IEEE, 2022.
  33. Paolo Serafini. Simulated annealing for multi objective optimization problems. In Multiple Criteria Decision Making: Proceedings of the Tenth International Conference: Expand and Enrich the Domains of Thinking and Application, pages 283–292. Springer, 1994.
  34. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning, pages 4548–4557. PMLR, 2018.
  35. Lifelong machine learning systems: Beyond learning algorithms. In 2013 AAAI spring symposium series, 2013.
  36. Search based repair of deep neural networks. arXiv preprint arXiv:1912.12463, 2019.
  37. Provable repair of deep neural networks. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pages 588–603, 2021.
  38. A survey of simulated annealing as a tool for single and multiobjective optimization. Journal of the operational research society, 57:1143–1160, 2006.
  39. Causality-based neural network repair. In Proceedings of the 44th International Conference on Software Engineering, pages 338–349, 2022.
  40. Neurecover: Regression-controlled repair of deep neural networks with training history. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 1111–1121. IEEE, 2022.
  41. Reasoning about safety of learning-enabled components in autonomous cyber-physical systems. In Proceedings of the 55th Annual Design Automation Conference, pages 1–6, 2018.
  42. Nn repair: Constraint-based repair of neural network classifiers. In Computer Aided Verification: 33rd International Conference, CAV 2021, Virtual Event, July 20–23, 2021, Proceedings, Part I 33, pages 3–25. Springer, 2021.
  43. Cognitive computing and wireless communications on the edge for healthcare service robots. Computer Communications, 149:99–106, 2020.
  44. Neural network repair with reachability analysis. In International Conference on Formal Modeling and Analysis of Timed Systems, pages 221–236. Springer, 2022.
  45. Reinforcement learning in healthcare: A survey. ACM Computing Surveys (CSUR), 55(1):1–36, 2021.
  46. Runtime-safety-guided policy repair. In Runtime Verification: 20th International Conference, RV 2020, Los Angeles, CA, USA, October 6–9, 2020, Proceedings 20, pages 131–150. Springer, 2020.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now