Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Runtime Verification and Field-based Testing for ROS-based Robotic Systems (2404.11498v3)

Published 17 Apr 2024 in cs.SE and cs.RO

Abstract: Robotic systems are becoming pervasive and adopted in increasingly many domains, such as manufacturing, healthcare, and space exploration. To this end, engineering software has emerged as a crucial discipline for building maintainable and reusable robotic systems. The robotics software engineering research field has received increasing attention, fostering autonomy as a fundamental goal. However, robotics developers are still challenged to achieve this goal because simulation cannot realistically deliver solutions to emulate real-world phenomena. Robots also need to operate in unpredictable and uncontrollable environments, which require safe and trustworthy self-adaptation capabilities implemented in software. Typical techniques to address the challenges are runtime verification, field-based testing, and mitigation techniques that enable fail-safe solutions. However, no clear guidance exists for architecting ROS-based systems to enable and facilitate runtime verification and field-based testing. This paper aims to fill this gap by providing guidelines to help developers and quality assurance (QA) teams develop, verify, or test their robots in the field. These guidelines are carefully tailored to address the challenges and requirements of testing robotics systems in real-world scenarios. We conducted (i) a literature review on studies addressing runtime verification and field-based testing for robotic systems, (ii) mined ROS-based applications repositories, and (iii) validated the applicability, clarity, and usefulness via two questionnaires with 55 answers overall. We contribute 20 guidelines: 8 for developers and 12 for QA teams formulated for researchers and practitioners in robotic software engineering. Finally, we map our guidelines to open challenges in runtime verification and field-based testing for ROS-based systems, and we outline promising research directions in the field.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (155)
  1. G.-Z. Yang, J. Bellingham et al., “The grand challenges of science robotics,” Science Robotics, vol. 3, no. 14, p. eaar7650, 2018.
  2. D. Brugali and E. Prassler, “Software engineering for robotics [from the guest editors],” IEEE Robotics & Automation Magazine, vol. 16, no. 1, pp. 9–15, 2009.
  3. S. García, D. Strüber et al., “Robotics software engineering: A perspective from the service robotics domain,” in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, pp. 593–604.
  4. N. Blümlein, “Function-based cost estimation for service robot prototypes in early design phases,” Ph.D. dissertation, University of Stuttgart, 2014.
  5. J. Cámara, B. Schmerl et al., “Software architecture and task plan co-adaptation for mobile service robots,” in Proceedings of the IEEE/ACM 15th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2020, pp. 125–136.
  6. M. Askarpour, C. Tsigkanos et al., “Robomax: Robotic mission adaptation exemplars,” in 2021 International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).   IEEE, 2021, pp. 245–251.
  7. S. Dragule, S. G. Gonzalo et al., “Languages for specifying missions of robotic applications,” Software Engineering for Robotics, pp. 377–411, 2021.
  8. C. Menghi, C. Tsigkanos et al., “Mission specification patterns for mobile robots: Providing support for quantitative properties,” IEEE Transactions on Software Engineering, 2022.
  9. ——, “Specification patterns for robotic missions,” IEEE Transactions on Software Engineering, vol. 47, no. 10, pp. 2208–2224, oct 2021.
  10. S. Dragule, T. Berger et al., “A survey on the design space of end-user oriented languages for specifying robotic missions,” International Journal of Software and Systems Modeling, vol. 20, no. 4, pp. 1123–1158, 2021.
  11. D. Kortenkamp, R. Simmons et al., “Robotic systems architectures and programming,” Springer handbook of robotics, pp. 283–306, 2016.
  12. G. Rodrigues, R. Caldas et al., “An architecture for mission coordination of heterogeneous robots,” Journal of Systems and Software, vol. 191, p. 111363, 2022.
  13. D. Brugali and P. Scandurra, “Component-based robotic engineering (part i) [tutorial],” IEEE Robotics & Automation Magazine, vol. 16, no. 4, pp. 84–96, 2009.
  14. D. Brugali and A. Shakhimardanov, “Component-based robotic engineering (part ii),” IEEE Robotics & Automation Magazine, vol. 17, no. 1, pp. 100–112, 2010.
  15. R. Ghzouli, T. Berger et al., “Behavior trees and state machines in robotics applications,” IEEE Transactions on Software Engineering, 2023.
  16. A. Gotlieb, D. Marijan et al., “Testing industrial robotic systems: A new battlefield!” Software Engineering for Robotics, pp. 109–137, 2021.
  17. A. Cavalcanti, W. Barnett et al., “Robostar technology: A roboticist’s toolbox for combined proof, simulation, and testing,” Software Engineering for Robotics, pp. 249–293, 2021.
  18. F. Ingrand, “Verification of autonomous robots: A roboticist’s bottom-up approach,” Software engineering for robotics, pp. 219–248, 2021.
  19. D. Bozhinoski, D. Di Ruscio et al., “Safety for mobile robotic systems: A systematic mapping study from a software engineering perspective,” Journal of Systems and Software, vol. 151, pp. 150–179, 2019.
  20. S. García, D. Strüber et al., “Software variability in service robotics,” Empirical Softw. Engg., vol. 28, no. 2, dec 2022.
  21. C. Hildebrandt, M. von Stein, and S. Elbaum, “Physcov: Physical test coverage for autonomous vehicles,” in Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, 2023, pp. 449–461.
  22. M. Rizwan, R. Caldas, C. Reichenbach, and M. Mayr, “Ezskiros: A case study on embedded robotics dsls to catch bugs early,” in 2023 IEEE/ACM 5th International Workshop on Robotics Software Engineering (RoSE).   IEEE, 2023, pp. 61–68.
  23. C. Hildebrandt and S. Elbaum, “World-in-the-loop simulation for autonomous systems validation,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 10 912–10 919.
  24. A. Alami, Y. Dittrich et al., “Influencers of quality assurance in an open source community,” in Proceedings of the 11th International Workshop on Cooperative and Human Aspects of Software Engineering, 2018, pp. 61–68.
  25. A. Santos, A. Cunha et al., “The high-assurance ros framework,” in 2021 IEEE/ACM 3rd International Workshop on Robotics Software Engineering (RoSE).   IEEE, 2021, pp. 37–40.
  26. I. Malavolta, G. A. Lewis et al., “Mining guidelines for architecting robotics software,” Journal of Systems and Software, vol. 178, p. 110969, 2021.
  27. K. Havelund and A. Goldberg, “Verify your runs,” in Working Conference on Verified Software: Theories, Tools, and Experiments.   Springer, 2005, pp. 374–383.
  28. E. Bartocci, Y. Falcone et al., “Introduction to runtime verification,” Lectures on Runtime Verification: Introductory and Advanced Topics, pp. 1–33, 2018.
  29. Y. Falcone, S. Krstić et al., “A taxonomy for classifying runtime verification tools,” International Journal on Software Tools for Technology Transfer, vol. 23, no. 2, pp. 255–284, 2021.
  30. S. Azzopardi, N. Piterman et al., “Runtime verification meets controller synthesis,” in International Symposium on Leveraging Applications of Formal Methods.   Springer, 2022, pp. 382–396.
  31. M. Rizwan, C. Reichenbach et al., “Rossmarie: A domain-specific language to express dynamic safety rules and recovery strategies for autonomous robots,” in Second Workshop on Quality and Reliability Assessment of Robotic Software Architectures and Components, 2023.
  32. M. Vierhauser, A. Garmendia et al., “Grum—a flexible model-driven runtime monitoring framework and its application to automated aerial and ground vehicles,” Journal of Systems and Software, vol. 203, p. 111733, 2023.
  33. S. G. Brunner, P. Lehner et al., “Design, execution, and postmortem analysis of prolonged autonomous robot operations,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 1056–1063, 2018.
  34. A. Bertolino, P. Braione et al., “A survey of field-based testing techniques,” ACM Computing Surveys (CSUR), vol. 54, no. 5, pp. 1–39, 2021.
  35. S. Eberlin, M. S. d. Silva et al., “The ex vivo skin model as an alternative tool for the efficacy and safety evaluation of topical products,” Alternatives to Laboratory Animals, vol. 48, no. 1, pp. 10–22, 2020.
  36. R. Wieringa, “Design science as nested problem solving,” in Proceedings of the 4th international conference on design science research in information systems and technology, 2009, pp. 1–12.
  37. R. Baskerville, A. Baiyere et al., “Design science research contributions: Finding a balance between artifact and theory,” Journal of the Association for Information Systems, vol. 19, no. 5, pp. 358–376, 2018.
  38. R. Caldas, J. A. P. García et al., “Verification of ROS-based Systems: Guidelines for Developers and QA teams,” 2024. [Online]. Available: https://ros-rvft.github.io
  39. B. Kitchenham, O. P. Brereton et al., “Systematic literature reviews in software engineering–a systematic literature review,” Information and software technology, vol. 51, no. 1, pp. 7–15, 2009.
  40. Dronecode, “PX4 Autopilot Project,” 2023. [Online]. Available: https://px4.io/
  41. M. Luckcuck, M. Farrell et al., “Formal specification and verification of autonomous robotic systems: A survey,” ACM Computing Surveys (CSUR), vol. 52, no. 5, pp. 1–41, 2019.
  42. M. Proetzsch, T. Luksch et al., “The behaviour-based control architecture ib2c for complex robotic systems,” in KI 2007: Advances in Artificial Intelligence: 30th Annual German Conference on AI, KI 2007, Osnabrück, Germany, September 10-13, 2007. Proceedings 30.   Springer, 2007, pp. 494–497.
  43. Y. Falcone, K. Havelund et al., “A tutorial on runtime verification,” Engineering dependable software systems, pp. 141–175, 2013.
  44. Y. Falcone, M. Jaber et al., “Runtime verification of component-based systems,” in Software Engineering and Formal Methods: 9th International Conference, SEFM 2011, Montevideo, Uruguay, November 14-18, 2011. Proceedings 9.   Springer, 2011, pp. 204–220.
  45. M. Beul, N. Krombach et al., “Autonomous navigation in a warehouse with a cognitive micro aerial vehicle,” in Robot Operating System (ROS).   Springer, 2017, pp. 487–524.
  46. T. Lewis and K. Bhaganagar, “Configurable simulation strategies for testing pollutant plume source localization algorithms using autonomous multisensor mobile robots,” International Journal of Advanced Robotic Systems, vol. 19, no. 2, p. 17298806221081325, 2022.
  47. Z. Li, A. Hasegawa et al., “Autoware_perf: A tracing and performance analysis framework for ros 2 applications,” Journal of Systems Architecture, vol. 123, p. 102341, 2022.
  48. M. Albonico, M. DJević et al., “Software engineering research on the robot operating system: A systematic mapping study,” Journal of Systems and Software, vol. 197, p. 111574, 2023.
  49. C. Hu, W. Dong et al., “Runtime verification on hierarchical properties of ros-based robot swarms,” IEEE Transactions on Reliability, vol. 69, no. 2, pp. 674–689, 2019.
  50. J. Huang, C. Erdogan et al., “Rosrv: Runtime verification for robots,” in International Conference on Runtime Verification.   Springer, 2014, pp. 247–254.
  51. V. Mayoral-Vilches, R. White et al., “Sros2: Usable cyber security tools for ros 2,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 11 253–11 259.
  52. S.-Y. Jeong, I.-J. Choi et al., “A study on ros vulnerabilities and countermeasure,” in Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, 2017, pp. 147–148.
  53. A. Afzal, C. Le Goues et al., “A study on challenges of testing robotic systems,” in 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST).   IEEE, 2020, pp. 96–107.
  54. S. Adam, M. Larsen et al., “Towards rule-based dynamic safety monitoring for mobile robots,” in Simulation, Modeling, and Programming for Autonomous Robots: 4th International Conference, SIMPAR 2014, Bergamo, Italy, October 20-23, 2014. Proceedings 4.   Springer, 2014, pp. 207–218.
  55. M. Stadler and M. Vierhauser, “Romosu: Flexible runtime monitoring support for ros-based applications,” in RoSE’23: 5th International Workshop on Robotics Software Engineering Proceedings, 2023, p. xx.
  56. C. Hartsell, N. Mahadevan et al., “Model-based design for cps with learning-enabled components,” in Proceedings of the Workshop on Design Automation for CPS and IoT, 2019, pp. 1–9.
  57. C. Hutchison, M. Zizyte et al., “Robustness testing of autonomy software,” in 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP).   IEEE, 2018, pp. 276–285.
  58. S. Limsoonthrakul, M. N. Dailey et al., “A modular system architecture for autonomous robots based on blackboard and publish-subscribe mechanisms,” in 2008 IEEE International conference on robotics and biomimetics.   IEEE, 2009, pp. 633–638.
  59. M. Efatmaneshnik and M. Ryan, “A study of the relationship between system testability and modularity,” INSIGHT, vol. 20, no. 1, pp. 20–24, 2017.
  60. F. Rovida, M. Crosby et al., “Skiros—a skill-based robot control platform on top of ros,” Robot Operating System (ROS) The Complete Reference (Volume 2), pp. 121–160, 2017.
  61. A. Albore, D. Doose et al., “Skill-based design of dependable robotic architectures,” Robotics and Autonomous Systems, vol. 160, p. 104318, 2023.
  62. E. B. Gil, R. Caldas et al., “Body sensor network: A self-adaptive system exemplar in the healthcare domain,” in 2021 International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).   IEEE, 2021, pp. 224–230.
  63. R. Halder, J. Proença et al., “Formal verification of ros-based robotic applications using timed-automata,” in 2017 IEEE/ACM 5th International FME Workshop on Formal Methods in Software Engineering (FormaliSE).   IEEE, 2017, pp. 44–50.
  64. K. Chaaban, “A new algorithm for real-time scheduling and resource mapping for robot operating systems (ros),” Applied Sciences, vol. 13, no. 3, p. 1532, 2023.
  65. H. Choi, Y. Xiang et al., “Picas: New design of priority-driven chain-aware scheduling for ros2,” in 2021 IEEE 27th Real-Time and Embedded Technology and Applications Symposium (RTAS).   IEEE, 2021, pp. 251–263.
  66. T. Blaß, D. Casini et al., “A ros 2 response-time analysis exploiting starvation freedom and execution-time variance,” in 2021 IEEE Real-Time Systems Symposium (RTSS).   IEEE, 2021, pp. 41–53.
  67. G. Conte, D. Scaradozzi et al., “Development and experimental tests of a ros multi-agent structure for autonomous surface vehicles,” Journal of Intelligent & Robotic Systems, vol. 92, no. 3, pp. 705–718, 2018.
  68. K. Belsare, A. C. Rodriguez et al., “Micro-ros,” in Robot Operating System (ROS) The Complete Reference (Volume 7).   Springer, 2023, pp. 3–55.
  69. A. Nordmann, R. Lange et al., “System modes-digestible system (re-) configuration for robotics,” in 2021 IEEE/ACM 3rd International Workshop on Robotics Software Engineering (RoSE).   IEEE, 2021, pp. 19–24.
  70. J. Staschulat, I. Lütkebohle et al., “The rclc executor: Domain-specific deterministic scheduling mechanisms for ros applications on microcontrollers: work-in-progress,” in 2020 International Conference on Embedded Software (EMSOFT).   IEEE, 2020, pp. 18–19.
  71. A. Ferrando, R. C. Cardoso et al., “Rosmonitoring: a runtime verification framework for ros,” in Annual Conference Towards Autonomous Robotic Systems.   Springer, 2020, pp. 387–399.
  72. S. Elbaum and J. C. Munson, “Software black box: an alternative mechanism for failure analysis,” in Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.   IEEE, 2000, pp. 365–376.
  73. A. Mitrevski, S. Thoduka et al., “Deploying robots in everyday environments: Towards dependable and practical robotic systems,” arXiv preprint arXiv:2206.12719, 2022.
  74. J. A. Duraes and H. S. Madeira, “Emulation of software faults: A field data study and a practical approach,” Ieee transactions on software engineering, vol. 32, no. 11, pp. 849–867, 2006.
  75. R. Barbosa, J. Karlsson et al., “Fault injection,” Resilience Assessment and Evaluation of Computing Systems, pp. 263–281, 2012.
  76. R. K. Lenka, S. Padhi et al., “Fault injection techniques-a brief review,” in 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN).   IEEE, 2018, pp. 832–837.
  77. U. Yayan and C. Baglum, “Tailored mutation-based software fault injection tool (im-fit),” SoftwareX, p. 101463, 2023.
  78. S. Macenski, A. Soragna et al., “Impact of ros 2 node composition in robotic systems,” IEEE Robotics and Automation Letters, 2023.
  79. C. Bédard, P.-Y. Lajoie et al., “Message flow analysis with complex causal links for distributed ros 2 systems,” Robotics and Autonomous Systems, vol. 161, p. 104361, 2023.
  80. S. Rivera and R. State, “Securing robots: An integrated approach for security challenges and monitoring for the robotic operating system (ros),” in 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM).   IEEE, 2021, pp. 754–759.
  81. A. Bertolino, G. De Angelis et al., “Enhancing service federation trustworthiness through online testing,” Computer, vol. 45, no. 1, pp. 66–72, 2011.
  82. M. Lahami, M. Krichen et al., “Safe and efficient runtime testing framework applied in dynamic and distributed systems,” Science of Computer Programming, vol. 122, pp. 1–28, 2016.
  83. B. Breiling, B. Dieber et al., “Secure communication for the robot operating system,” in 2017 annual IEEE international systems conference (SysCon).   IEEE, 2017, pp. 1–6.
  84. E. Guerrero, F. Bonin-Font et al., “Adaptive visual information gathering for autonomous exploration of underwater environments,” IEEE Access, vol. 9, pp. 136 487–136 506, 2021.
  85. C. Schlegel, T. Haßler et al., “Robotic software systems: From code-driven to model-driven designs,” in 2009 International Conference on Advanced Robotics.   IEEE, 2009, pp. 1–8.
  86. J. C. Kirchhof, J. Michael et al., “Model-driven digital twin construction: synthesizing the integration of cyber-physical systems with their information systems,” in Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, 2020, pp. 90–101.
  87. T. Sotiropoulos, H. Waeselynck et al., “Can robot navigation bugs be found in simulation? an exploratory study,” in 2017 IEEE International conference on software quality, reliability and security (QRS).   IEEE, 2017, pp. 150–159.
  88. C. Saavedra Sueldo, I. Perez Colo et al., “Ros-based architecture for fast digital twin development of smart manufacturing robotized systems,” Annals of Operations Research, pp. 1–25, 2022.
  89. A. Fend and D. Bork, “Cpsaml: a language and code generation framework for digital twin based monitoring of mobile cyber-physical systems,” in Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, 2022, pp. 649–658.
  90. J. Ernits, E. Halling et al., “Model-based integration testing of ros packages: A mobile robot case study,” in 2015 European Conference on Mobile Robots (ECMR).   IEEE, 2015, pp. 1–7.
  91. K. G. Larsen, M. Mikucionis et al., “Testing real-time embedded software using uppaal-tron: an industrial case study,” in Proceedings of the 5th ACM international conference on Embedded software, 2005, pp. 299–306.
  92. N. Hammoudeh Garcia, M. Lüdtke et al., “Bootstrapping mde development from ros manual code - part 1: Metamodeling,” in 2019 Third IEEE International Conference on Robotic Computing (IRC), 2019, pp. 329–336.
  93. N. Hammoudeh García, H. Deshpande et al., “Bootstrapping mde development from ros manual code: Part 2—model generation and leveraging models at runtime,” Software and Systems Modeling, vol. 20, no. 6, pp. 2047–2070, 2021.
  94. A. F. F. Santos, “Safety verification for ros software,” Ph.D. dissertation, Universidade do Minho, 2021.
  95. S. Aldegheri, N. Bombieri et al., “A containerized ros-compliant verification environment for robotic systems,” in 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE).   IEEE, 2021, pp. 222–225.
  96. C. Lesire, S. Roussel et al., “Synthesis of real-time observers from past-time linear temporal logic and timed specification,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 597–603.
  97. D. Ničković and T. Yamaguchi, “Rtamt: Online robustness monitors from stl,” in International Symposium on Automated Technology for Verification and Analysis.   Springer, 2020, pp. 564–571.
  98. S. Adam, M. Larsen et al., “Rule-based dynamic safety monitoring for mobile robots,” Journal of Software Engineering for Robotics, vol. 7, no. 1, pp. 120–141, 2016.
  99. A. Balakrishnan, J. Deshmukh et al., “Percemon: Online monitoring for perception systems,” in International Conference on Runtime Verification.   Springer, 2021, pp. 297–308.
  100. X. Duan, Q. Wang et al., “Implementing trajectory tracking control algorithm for autonomous vehicles,” in 2021 IEEE International Conference on Unmanned Systems (ICUS).   IEEE, 2021, pp. 947–953.
  101. S. B. Liu, H. Roehm et al., “Provably safe motion of mobile robots in human environments,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2017, pp. 1351–1357.
  102. R. Queiroz, D. Sharma et al., “A driver-vehicle model for ads scenario-based testing,” arXiv preprint arXiv:2205.02911, 2022.
  103. D. J. Fremont, E. Kim et al., “Scenic: A language for scenario specification and data generation,” Machine Learning, pp. 1–45, 2022.
  104. C. Xu, H. Huang et al., “Sil simulation test platform based on prescan and ros,” in 2022 International Conference on Cyber-Physical Social Intelligence (ICCSI).   IEEE, 2022, pp. 240–244.
  105. Y.-S. Hsiao, Z. Wan et al., “Mavfi: An end-to-end fault analysis framework with anomaly detection and recovery for micro aerial vehicles,” arXiv preprint arXiv:2105.12882, 2021.
  106. S. Kim and T. Kim, “Robofuzz: Fuzzing robotic systems over robot operating system (ros) for finding correctness bugs,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2022.   New York, NY, USA: Association for Computing Machinery, 2022, p. 447–458.
  107. B. Dieber, R. White et al., “Penetration testing ros,” Robot Operating System (ROS), p. 183, 2020.
  108. A. Ortega, N. Hochgeschwender et al., “Testing service robots in the field: An experience report,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 165–172.
  109. A. Afzal, C. Le Goues et al., “Mithra: Anomaly detection as an oracle for cyberphysical systems,” IEEE Transactions on Software Engineering, vol. 48, no. 11, pp. 4535–4552, 2021.
  110. C. Hildebrandt, S. Elbaum et al., “Feasible and stressful trajectory generation for mobile robots,” in Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2020, pp. 349–362.
  111. A. Santos, A. Cunha et al., “Property-based testing for the robot operating system,” in Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation, 2018, pp. 56–62.
  112. Z. Han, T. Williams et al., “Mixed-reality robot behavior replay: A system implementation,” arXiv preprint arXiv:2210.00075, 2022.
  113. B. Zahn, J. Fountain et al., “Optimization of robot movements using genetic algorithms and simulation,” in RoboCup 2019: Robot World Cup XXIII 23.   Springer, 2019, pp. 466–475.
  114. R. D. Caldas, A. Rodrigues et al., “A hybrid approach combining control theory and ai for engineering self-adaptive systems,” in Proceedings of the IEEE/ACM 15th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2020, pp. 9–19.
  115. A. Afzal, D. S. Katz et al., “Simulation for robotics test automation: Developer perspectives,” in 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST).   IEEE, 2021, pp. 263–274.
  116. C. S. Timperley, A. Afzal et al., “Crashing simulated planes is cheap: Can simulation detect robotics bugs early?” in 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST).   IEEE, 2018, pp. 331–342.
  117. V. Estivill-Castro, R. Hexel et al., “Continuous integration for testing full robotic behaviours in a gui-stripped simulation.” in MoDELS (Workshops), 2018, pp. 453–464.
  118. C. Berger, “An open continuous deployment infrastructure for a self-driving vehicle ecosystem,” in Open Source Systems: Integrating Communities: 12th IFIP WG 2.13 International Conference, OSS 2016, Gothenburg, Sweden, May 30-June 2, 2016, Proceedings 12.   Springer, 2016, pp. 177–183.
  119. F. Roman, R. G. Maidana et al., “Overseer: A multi robot monitoring infrastructure,” in International Conference on Informatics in Control, Automation and Robotics (ICINCO), 2018, Brasil., 2018.
  120. D. Kirchner, S. Niemczyk et al., “Rosha: A multi-robot self-healing architecture,” in Robot Soccer World Cup.   Springer, 2013, pp. 304–315.
  121. M. A. Hossen, S. Kharade et al., “Care: Finding root causes of configuration issues in highly-configurable robots,” IEEE Robotics and Automation Letters, 2023.
  122. M. G. Morais, F. R. Meneguzzi et al., “Distributed fault diagnosis for multiple mobile robots using an agent programming language,” in 2015 International Conference on Advanced Robotics (ICAR).   IEEE, 2015, pp. 395–400.
  123. C. Tampier, A. Tiderko et al., “Field test tool: automatic reporting and reliability evaluation for autonomous ground vehicles,” in 2022 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).   IEEE, 2022, pp. 73–78.
  124. P. Kaveti and H. Singh, “Ros rescue: fault tolerance system for robot operating system,” Robot Operating System (ROS) The Complete Reference (Volume 5), pp. 381–397, 2021.
  125. S. Silva, A. Bertolino et al., “Self-adaptive testing in the field: are we there yet?” in Proceedings of the 17th Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2022, pp. 58–69.
  126. J. Cohen, “A power primer,” Psychological Bulletin, vol. 112, pp. 155–159, 1992.
  127. C. Sánchez, G. Schneider et al., “A survey of challenges for runtime verification from advanced application domains (beyond software),” Formal Methods in System Design, vol. 54, pp. 279–335, 2019.
  128. G. Dos Reis, J. Garcia et al., “A contract design,” C++ Standards Committee Working Group ISOCPP, 2016.
  129. M. Luckcuck, M. Farrell et al., “A compositional approach to verifying modular robotic systems,” arXiv preprint arXiv:2208.05507, 2022.
  130. M. Autili, L. Grunske et al., “Aligning qualitative, real-time, and probabilistic property specification patterns using a structured english grammar,” IEEE Transactions on Software Engineering, vol. 41, no. 7, pp. 620–638, 2015.
  131. C. Menghi, C. Tsigkanos et al., “Mission specification patterns for mobile robots: Providing support for quantitative properties,” IEEE Trans. Softw. Eng., vol. 49, no. 4, p. 2741–2760, dec 2022.
  132. S. Silva, P. Pelliccione et al., “Self-adaptive testing in the field,” ACM Trans. Auton. Adapt. Syst., vol. 19, no. 1, feb 2024.
  133. S. Keele et al., “Guidelines for performing systematic literature reviews in software engineering,” 2007.
  134. Q. Song, E. Engström, and P. Runeson, “Concepts in testing of autonomous systems: Academic literature and industry practice,” in 2021 IEEE/ACM 1st Workshop on AI Engineering-Software Engineering for AI (WAIN).   IEEE, 2021, pp. 74–81.
  135. M. A. Brito, S. R. Souza, and P. S. Souza, “Integration testing for robotic systems,” Software Quality Journal, vol. 30, no. 1, pp. 3–35, 2022.
  136. J. Laval, L. Fabresse, and N. Bouraqadi, “A methodology for testing mobile autonomous robots,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.   IEEE, 2013, pp. 1842–1847.
  137. A. Babić, G. Vasiljević, and N. Mišković, “Vehicle-in-the-loop framework for testing long-term autonomy in a heterogeneous marine robot swarm,” IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4439–4446, 2020.
  138. L. Gazzola, L. Mariani, M. Orrú, M. Pezzè, and M. Tappler, “Testing software in production environments with data from the field,” in IEEE Conference on Software Testing, Verification and Validation (ICST), 2022.
  139. D. Weyns, I. Gerostathopoulos et al., “Guidelines for artifacts to support industry-relevant research on self-adaptation,” SIGSOFT Softw. Eng. Notes, vol. 47, no. 4, p. 18–24, sep 2022.
  140. A. Bombarda, S. Bonfanti et al., “Guidelines for the development of a critical software under emergency,” Information and Software Technology, vol. 152, p. 107061, 2022.
  141. A. Avizienis, J.-C. Laprie et al., “Basic concepts and taxonomy of dependable and secure computing,” IEEE transactions on dependable and secure computing, vol. 1, no. 1, pp. 11–33, 2004.
  142. J. Bengtsson, K. Larsen et al., “Uppaal—a tool suite for automatic verification of real-time systems,” in Proceedings of the DIMACS/SYCON Workshop on Hybrid Systems III: Verification and Control: Verification and Control.   Berlin, Heidelberg: Springer-Verlag, 1996, p. 232–243.
  143. T. Yamaguchi, B. Hoxha et al., “Rtamt–runtime robustness monitors with application to cps and robotics,” International Journal on Software Tools for Technology Transfer, pp. 1–21, 2023.
  144. A. Rodrigues, R. D. Caldas et al., “A learning approach to enhance assurances for real-time self-adaptive systems,” in 2018 IEEE/ACM 13th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).   IEEE, 2018, pp. 206–216.
  145. A. Dosovitskiy, G. Ros et al., “Carla: An open urban driving simulator,” in Conference on robot learning.   PMLR, 2017, pp. 1–16.
  146. D. Tommaso, D. J. Fremont et al., “Verifai: A toolkit for the design and analysis of artificial intelligence-based systems,” in Proceedings of the 31st Conference on Computer Aided Verification, CAV 2019, 2019.
  147. M. Tideman, “Scenario-based simulation environment for assistance systems,” ATZautotechnology, vol. 10, no. 1, pp. 28–32, 2010.
  148. S. Documentation, “Simulation and model-based design,” 2020. [Online]. Available: https://www.mathworks.com/products/simulink.html
  149. E. Rohmer, S. P. N. Singh et al., “V-rep: A versatile and scalable robot simulation framework,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 1321–1326.
  150. E. T. Barr, M. Harman et al., “The oracle problem in software testing: A survey,” IEEE transactions on software engineering, vol. 41, no. 5, pp. 507–525, 2014.
  151. S. Reid, “Software and systems engineering software testing part 1: Concepts and definitions,” ISO/IEC/IEEE 29119-1, Tech. Rep., 2013.
  152. Y. Koo and S. Kim, “Distributed logging system for ros-based systems,” in 2019 IEEE International Conference on Big Data and Smart Computing (BigComp).   IEEE, 2019, pp. 1–3.
  153. C. Pinciroli, V. Trianni et al., “ARGoS: a modular, parallel, multi-engine simulator for multi-robot systems,” Swarm Intelligence, vol. 6, no. 4, pp. 271–295, 2012.
  154. L. Pitonakova, M. Giuliani et al., “Feature and performance comparison of the v-rep, gazebo and argos robot simulators,” in Towards Autonomous Robotic Systems: 19th Annual Conference, TAROS 2018, Bristol, UK July 25-27, 2018, Proceedings 19.   Springer, 2018, pp. 357–368.
  155. C. Jung, A. Ahad et al., “Swarmbug: debugging configuration bugs in swarm robotics,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2021, pp. 868–880.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Ricardo Caldas (6 papers)
  2. Matei Schiopu (1 paper)
  3. Patrizio Pelliccione (32 papers)
  4. Thorsten Berger (21 papers)
  5. Juan Antonio Pinera Garcia (1 paper)
  6. Genaina Rodrigues (2 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now