2000 character limit reached
Taming the Beast: Fully Automated Unit Testing with Coyote C++ (2401.01073v2)
Published 2 Jan 2024 in cs.PL and cs.SE
Abstract: In this paper, we present Coyote C++, a fully automated white-box unit testing tool for C and C++. Whereas existing tools have struggled to realize unit test generation for C++, Coyote C++ is able to produce high coverage results from unit test generation at a testing speed of over 10,000 statements per hour. This impressive feat is made possible by the combination of a powerful concolic execution engine with sophisticated automated test harness generation. Additionally, the GUI of Coyote C++ displays detailed code coverage visualizations and provides various configuration features for users seeking to manually optimize their coverage results. Combining potent one-click automated testing with rich support for manual tweaking, Coyote C++ is the first automated testing tool that is practical enough to make automated testing of C++ code truly viable in industrial applications.
- C. Cadar, D. Dunbar, D. R. Engler et al., “KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs.” in OSDI, vol. 8, 2008, pp. 209–224.
- K. Sen, D. Marinov, and G. Agha, “CUTE: A concolic unit testing engine for C,” ACM SIGSOFT Software Engineering Notes, vol. 30, no. 5, pp. 263–272, 2005.
- G. Fraser and A. Arcuri, “A large-scale evaluation of automated unit test generation using EvoSuite,” ACM Trans. Softw. Eng. Methodol., vol. 24, no. 2, dec 2014. [Online]. Available: https://doi.org/10.1145/2685612
- M. Tufano, D. Drain, A. Svyatkovskiy, S. K. Deng, and N. Sundaresan, “Unit test case generation with transformers,” ArXiv, vol. abs/2009.05617, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:221655653
- M. Schäfer, S. Nadi, A. Eghbali, and F. Tip, “An empirical evaluation of using large language models for automated unit test generation,” arXiv preprint arXiv:2302.06527, 2023.
- K. Sen and G. Agha, “Cute and jcute: Concolic unit testing and explicit path model-checking tools,” in Computer Aided Verification, T. Ball and R. B. Jones, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 419–423.
- Y. Kim, D. Lee, J. Baek, and M. Kim, “Concolic testing for high test coverage and reduced human effort in automotive industry,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 2019, pp. 151–160.
- P. Godefroid, N. Klarlund, and K. Sen, “DART: Directed automated random testing,” in Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, 2005, pp. 213–223.
- N. Tillmann and J. de Halleux, “Pex–white box test generation for .net,” in Tests and Proofs, B. Beckert and R. Hähnle, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 134–153.
- A. Giantsios, N. Papaspyrou, and K. Sagonas, “Concolic testing for functional languages,” Science of Computer Programming, vol. 147, pp. 109–134, 2017, selected and Extended papers from the International Symposium on Principles and Practice of Declarative Programming 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167642317300837
- K. Sen, S. Kalasapur, T. Brutch, and S. Gibbs, “Jalangi: A selective record-replay and dynamic analysis framework for javascript,” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ser. ESEC/FSE 2013. New York, NY, USA: Association for Computing Machinery, 2013, p. 488–498. [Online]. Available: https://doi.org/10.1145/2491411.2491447
- R. S. Herlim, Y. Kim, and M. Kim, “CITRUS: Automated unit testing tool for real-world C++ programs,” in 2022 IEEE Conference on Software Testing, Verification and Validation (ICST), 2022, pp. 400–410.
- D. Ivanov, A. Babushkin, S. Grigoryev, P. Iatchenii, V. Kalugin, E. Kichin, E. Kulikov, A. Misonizhnik, D. Mordvinov, S. Morozov et al., “UnitTestBot: Automated unit test generation for C code in integrated development environments,” in 2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 2023, pp. 380–384.
- S. Rho, P. Martens, S. Shin, Y. Kim, H. Heo, and S. Oh, “Coyote c++: An industrial-strength fully automated unit testing tool,” 2023.
- R. Baldoni, E. Coppa, D. C. D’elia, C. Demetrescu, and I. Finocchi, “A survey of symbolic execution techniques,” ACM Comput. Surv., vol. 51, no. 3, may 2018. [Online]. Available: https://doi.org/10.1145/3182657