Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Coupled Requirements-driven Testing of CPS: From Simulation To Reality (2403.16287v2)

Published 24 Mar 2024 in cs.SE

Abstract: Failures in safety-critical Cyber-Physical Systems (CPS), both software and hardware-related, can lead to severe incidents impacting physical infrastructure or even harming humans. As a result, extensive simulations and field tests need to be conducted, as part of the verification and validation of system requirements, to ensure system safety. However, current simulation and field testing practices, particularly in the domain of small Unmanned Aerial Systems (sUAS), are ad-hoc and lack a thorough, structured testing process. Furthermore, there is a dearth of standard processes and methodologies to inform the design of comprehensive simulation and field tests. This gap in the testing process leads to the deployment of sUAS applications that are: (a) tested in simulation environments which do not adequately capture the real-world complexity, such as environmental factors, due to a lack of tool support; (b) not subjected to a comprehensive range of scenarios during simulation testing to validate the system requirements, due to the absence of a process defining the relationship between requirements and simulation tests; and (c) not analyzed through standard safety analysis processes, because of missing traceability between simulation testing artifacts and safety analysis artifacts. To address these issues, we have developed an initial framework for validating CPS, specifically focusing on sUAS and robotic applications. We demonstrate the suitability of our framework by applying it to an example from the sUAS domain. Our preliminary results confirm the applicability of our framework. We conclude with a research roadmap to outline our next research goals along with our current proposal.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. A. Orso and G. Rothermel, “Software testing: a research travelogue (2000–2014),” in Future of Software Eng. Proc., 2014, pp. 117–132.
  2. Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Trans. on Software Eng., vol. 37, no. 5, pp. 649–678, 2010.
  3. T. Y. Chen, F.-C. Kuo, H. Liu, P.-L. Poon, D. Towey, T. Tse, and Z. Q. Zhou, “Metamorphic testing: A review of challenges and opportunities,” ACM Computing Surveys (CSUR), vol. 51, no. 1, pp. 1–27, 2018.
  4. M. Harman and P. McMinn, “A theoretical and empirical study of search-based testing: Local, global, and hybrid search,” IEEE Trans. on Software Eng., vol. 36, no. 2, pp. 226–247, 2009.
  5. C. Patrício, R. Pinto, and G. Marques, “A study on software testing standard using iso/iec/ieee 29119-2: 2013,” Recent Advances in Intelligent Systems and Smart Applications, pp. 43–62, 2021.
  6. R. Hawkins, I. Habli, T. Kelly, and J. McDermid, “Assurance cases and prescriptive software safety certification: A comparative study,” Safety science, vol. 59, pp. 55–71, 2013.
  7. S. Ali and T. Yue, “U-Test: Evolving, modelling and testing realistic uncertain behaviours of cyber-physical systems,” in Proc. of the 8th Int’l Conference on Software Testing, Verification and Validation.   IEEE, 2015, pp. 1–2.
  8. E. Bringmann and A. Krämer, “Model-based testing of automotive systems,” in 2008 1st international conference on software testing, verification, and validation.   IEEE, 2008, pp. 485–493.
  9. A. Boeing and T. Bräunl, “Leveraging multiple simulators for crossing the reality gap,” in 2012 12th international conference on control automation robotics & vision (ICARCV).   IEEE, 2012, pp. 1113–1119.
  10. A. Agrawal, B. Zhang, Y. Shivalingaiah, M. Vierhauser, and J. Cleland-Huang, “A requirements-driven platform for validating field operations of small uncrewed aerial vehicles,” in Proc. of the 31st Int’l Requirements Engineering Conf.   IEEE, 2023, pp. 29–40.
  11. M. Vierhauser, M. N. A. Islam, A. Agrawal, J. Cleland-Huang, and J. Mason, “Hazard analysis for human-on-the-loop interactions in suas systems,” in Proc. of the 29th ACM Joint Meeting on European Software Engineering Conf. and Symp. on the Foundations of Software Engineering, 2021, pp. 8–19.
  12. “Facebook aquila drone accident: Gust induced structural failure - aerossurance,” https://aerossurance.com/safety-management/facebook-aquila-drone-accident/, (Accessed on 07/31/2023).
  13. Open Robotics, “Gazebo,” https://gazebosim.org, 2023, [Online: accessed 8-14-2023].
  14. S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in Field and Service Robotics: Results of the 11th Int’l Conf.   Springer, 2018, pp. 621–635.
  15. S. Abbaspour Asadollah, R. Inam, and H. Hansson, “A survey on testing for cyber physical system,” in Proc. of the 27th Int’l Conference on Testing Software and System.   Springer, 2015, pp. 194–207.
  16. X. Zhou, X. Gou, T. Huang, and S. Yang, “Review on testing of cyber physical systems: Methods and testbeds,” IEEE Access, vol. 6, pp. 52 179–52 194, 2018.
  17. A. Pretschner, “Model-based testing,” in Proc. of the 27th Int’l Conf. on Software Engineering, 2005, pp. 722–723.
  18. S. R. Dalal, A. Jain, N. Karunanithi, J. Leaton, C. M. Lott, G. C. Patton, and B. M. Horowitz, “Model-based testing in practice,” in Proc. of the 21st Int’l Conf. on Software Engineering, 1999, pp. 285–294.
  19. H. G. Gurbuz and B. Tekinerdogan, “Model-based testing for software safety: a systematic mapping study,” Software Quality Journal, vol. 26, pp. 1327–1372, 2018.
  20. F. Dalpiaz, X. Franch, and J. Horkoff, “istar 2.0 language guide,” arXiv preprint arXiv:1605.07767, 2016.
  21. P. Giorgini, M. Kolp, J. Mylopoulos, and M. Pistore, “The tropos methodology: An overview,” Methodologies and software engineering for agent systems: the agent-oriented software engineering handbook, pp. 89–106, 2004.
  22. A. Dardenne, A. Van Lamsweerde, and S. Fickas, “Goal-directed requirements acquisition,” Science of computer programming, vol. 20, no. 1-2, pp. 3–50, 1993.
  23. S. Greenspan, J. Mylopoulos, and A. Borgida, “On formal requirements modeling languages: Rml revisited,” in Proc. of the 16th Int’l Conf. on Software Engineering.   IEEE, 1994, pp. 135–147.
  24. M. M. Awan, F. Azam, M. W. Anwar, and Y. Rasheed, “Formal requirements specification: Z notation meta model facilitating model to model transformation,” in Proc. of the 9th Int’l Conf. on Software and Information Engineering, 2020, pp. 61–66.
  25. A. Cimatti, M. Roveri, A. Susi, and S. Tonetta, “From informal requirements to property-driven formal validation,” in Proc. of the 3th International Workshop on Formal Methods for Industrial Critical Systems.   Springer, 2009, pp. 166–181.
  26. M. von der Beeck, T. Margaria, and B. Steffen, “A formal requirements engineering method for specification, synthesis, and verification,” in Proc. 8th Conf. on Software Engineering Environments.   IEEE, 1997, pp. 131–144.
  27. M. Greiler, A. Van Deursen, and M.-A. Storey, “Automated detection of test fixture strategies and smells,” in Proc. of the 6th Int’l Conf. on Software Testing, Verification and Validation.   IEEE, 2013, pp. 322–331.
  28. E. Denney, G. Pai, and I. Habli, “Dynamic safety cases for through-life safety assurance,” in Proc. of the 37th IEEE Int’l Conf. on Software Engineering, vol. 2.   IEEE, 2015, pp. 587–590.
  29. A. Mavin, P. Wilkinson, A. Harwood, and M. Novak, “Easy approach to requirements syntax (ears),” in Proc. of the 17th Int’l Requirements Engineering Conf.   IEEE, 2009, pp. 317–322.
  30. L. P. Gaspary, L. F. Balbinot, R. Storch, F. Wendt, and L. R. Tarouco, “Distributed management of high-layer protocols and network services through a programmable agent-based architecture,” in Proc. of the 1st Int’l Conf. on Networking.   Springer, 2001, pp. 204–217.
  31. “Cesium for unreal – cesium,” https://cesium.com/platform/cesium-for-unreal/, [Accessed on 01/08/2023].
  32. M. N. Al Islam, M. T. Chowdhury, A. Agrawal, M. Murphy, R. Mehta, D. Kudriavtseva, J. Cleland-Huang, M. Vierhauser, and M. Chechik, “Configuring mission-specific behavior in a product line of collaborating small unmanned aerial systems,” Journal of Systems and Software, vol. 197, p. 111543, 2023.
  33. L. He, P. Glogowski, K. Lemmerz, B. Kuhlenkötter, and W. Zhang, “Method to integrate human simulation into gazebo for human-robot collaboration,” in IOP Conference Series: Materials Science and Engineering, vol. 825, no. 1.   IOP Publishing, 2020, p. 012006.
  34. A. Agrawal and J. Cleland-Huang, “Leveraging traceability to integrate safety analysis artifacts into the software development process,” in 2023 IEEE 31st International Requirements Engineering Conference Workshops (REW).   IEEE, 2023, pp. 475–478.
  35. P. Mäder, P. L. Jones, Y. Zhang, and J. Cleland-Huang, “Strategic traceability for safety-critical projects,” IEEE software, vol. 30, no. 3, pp. 58–66, 2013.
  36. I. Galvao and A. Goknil, “Survey of traceability approaches in model-driven engineering,” in Proc. of the 11th IEEE Int’l Enterprise Distributed Object Computing Conf.   IEEE, 2007, pp. 313–313.
  37. C. C. Raţiu, C. Mayr-Dorn, W. K. Assunção, and A. Egyed, “Taming cross-tool traceability in the wild,” in Proc. of the 31st Int’l Requirements Engineering Conf.   IEEE, 2023, pp. 233–243.
  38. J. H. Hill, J. White, S. Eade, D. Schmidt, and T. Denton, “Towards a solution for synchronizing disparate models of ultra-large-scale systems,” in Proc. of the 2nd Int’l Workshop on Ultra-large-scale software-intensive systems, 2008, pp. 19–22.
  39. P. Zech, M. Felderer, P. Kalb, and R. Breu, “A generic platform for model-based regression testing,” in Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change: 5th Int’l Symp.   Springer, 2012, pp. 112–126.
  40. B. Succar, “Building information modelling framework: A research and delivery foundation for industry stakeholders,” Automation in construction, vol. 18, no. 3, pp. 357–375, 2009.
  41. A. Olechowski, S. D. Eppinger, and N. Joglekar, “Technology readiness levels at 40: A study of state-of-the-art use, challenges, and opportunities,” in Proc. of the Int’l Conf on Management of Engineering and Technology.   IEEE, 2015, pp. 2084–2094.
  42. E. Thorn, S. C. Kimmel, M. Chaka, B. A. Hamilton et al., “A framework for automated driving system testable cases and scenarios,” United States. Department of Transportation. National Highway Traffic Safety …, Tech. Rep., 2018.
  43. N. Jakobi, P. Husbands, and I. Harvey, “Noise and the reality gap: The use of simulation in evolutionary robotics,” in Proc. of the Third European Conference on Artificial Life.   Springer, 1995, pp. 704–720.
  44. E. Salvato, G. Fenu, E. Medvet, and F. A. Pellegrino, “Crossing the reality gap: A survey on sim-to-real transferability of robot controllers in reinforcement learning,” IEEE Access, vol. 9, pp. 153 171–153 187, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now