2000 character limit reached
Using Neural Networks for Novelty-based Test Selection to Accelerate Functional Coverage Closure (2207.00445v3)
Published 1 Jul 2022 in cs.SE and cs.LG
Abstract: Novel test selectors used in simulation-based verification have been shown to significantly accelerate coverage closure regardless of the number of coverage holes. This paper presents a configurable and highly-automated framework for novel test selection based on neural networks. Three configurations of this framework are tested with a commercial signal processing unit. All three convincingly outperform random test selection with the largest saving of simulation being 49.37% to reach 99.5% coverage. The computational expense of the configurations is negligible compared to the simulation reduction. We compare the experimental results and discuss important characteristics related to the performance of the configurations.
- Online selection of effective functional test programs based on novelty detection. In 2010 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pages 762–769. IEEE, 2010.
- Novel test detection to improve simulation efficiency—a commercial experiment. In 2012 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pages 101–108. IEEE, 2012.
- F. Chollet et al. Keras. https://keras.io, 2015.
- Towards automating simulation-based design verification using ilp. In S. Muggleton, R. Otero, and A. Tamaddoni-Nezhad, editors, Inductive Logic Programming, pages 154–168, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg.
- Automatic scalable system for the coverage-directed generation (cdg) problem. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 206–211, 2021.
- Late breaking results: Friends - finding related interesting events via neighbor detection. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1–2, 2020.
- Machine learning-guided stimulus generation for functional verification. In Proceedings of the Design and Verification Conference (DVCON-USA), Virtual Conference, pages 2–5, 2020.
- Functional test selection based on unsupervised support vector analysis. In 2008 45th ACM/IEEE Design Automation Conference, pages 262–267. IEEE, 2008.
- Verifying artificial neural network classifier performance using dataset dissimilarity measures. In 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 115–121, 2020.
- C. Ioannides and K. I. Eder. Coverage-directed test generation automated by machine learning–a review. ACM Transactions on Design Automation of Electronic Systems (TODAES), 17(1):1–21, 2012.
- A. Mandelbaum and D. Weinshall. Distance-based confidence score for neural network classifiers. arXiv preprint arXiv:1709.09844, 2017.
- Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
- A review of novelty detection. Signal processing, 99:215–249, 2014.
- R. H. Tim Blackmore and S. Schaal. Novelty-driven verification: Using machine learning to identify novel stimuli and close coverage. https://www.dropbox.com/s/iulpk8kba3f7xxn/DVCon%20US%202021_Proceedings-FINAL.zip?dl=0&file_subpath=%2FPapers%2F7060.pdf, 2021. Proceedings of the 2021 Design and Verification Conference (Virtual).
- Reluplex made more practical: Leaky relu. In 2020 IEEE Symposium on Computers and Communications (ISCC), pages 1–7. IEEE, 2020.