Towards Practical Requirement Analysis and Verification: A Case Study on Software IP Components in Aerospace Embedded Systems (2404.00795v1)
Abstract: IP-based software design is a crucial research field that aims to improve efficiency and reliability by reusing complex software components known as intellectual property (IP) components. To ensure the reusability of these components, particularly in security-sensitive software systems, it is necessary to analyze the requirements and perform formal verification for each IP component. However, converting the requirements of IP components from natural language descriptions to temporal logic and subsequently conducting formal verification demands domain expertise and non-trivial manpower. This paper presents a case study on software IP components derived from aerospace embedded systems, with the objective of automating the requirement analysis and verification process. The study begins by employing LLMs to convert unstructured natural language into formal specifications. Subsequently, three distinct verification techniques are employed to ascertain whether the source code meets the extracted temporal logic properties. By doing so, five real-world IP components from the China Academy of Space Technology (CAST) have been successfully verified.
- Software ip in embedded systems. Virtual Components Design and Reuse, pages 47–64, 2001.
- Strategies for the integration of hardware and software ip components in embedded systems-on-chip. Integration, 37(4):223–252, 2004.
- On-chip data security against untrustworthy software and hardware ips in embedded systems. In 2018 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pages 644–649. IEEE, 2018.
- Design space exploration with automatic generation of ip-based embedded software. In IFIP Working Conference on Distributed and Parallel Embedded Systems, pages 237–246. Springer, 2004.
- Joachim Kunkel. Toward ip-based system-level soc design. Computer, 36(05):88–89, 2003.
- Software control and intellectual property protection in cyber-physical systems. EURASIP Journal on Information Security, 2016:1–14, 2016.
- nl2spec: Interactively translating unstructured natural language to temporal logics with large language models. arXiv preprint arXiv:2303.04864, 2023.
- Towards improving verification productivity with circuit-aware translation of natural language to systemverilog assertions. In First International Workshop on Deep Learning-aided Verification, 2023.
- C2s: translating natural language comments to formal program specifications. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 25–37, 2020.
- A classification framework for software component models. IEEE Transactions on Software Engineering, 37(5):593–615, 2010.
- Herve C Lefevre. The fiber-optic gyroscope. Artech house, 2022.
- Synthesis of ltl formulas from natural language texts: State of the art and research directions. In 26th International symposium on temporal representation and reasoning (TIME 2019). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.
- Madjid Maidi. The common fragment of ctl and ltl. In Proceedings 41st Annual Symposium on Foundations of Computer Science, pages 643–652. IEEE, 2000.
- Vaughan R Pratt. Semantical considerations on floyd-hoare logic. In 17th Annual Symposium on Foundations of Computer Science (sfcs 1976), pages 109–121. IEEE, 1976.
- Mingsheng Ying. Floyd–hoare logic for quantum programs. ACM Transactions on Programming Languages and Systems (TOPLAS), 33(6):1–49, 2012.
- A brief overview of chatgpt: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5):1122–1136, 2023.
- Chatgpt: Jack of all trades, master of none. Information Fusion, page 101861, 2023.
- Sudharsan Ravichandiran. Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT. Packt Publishing Ltd, 2021.
- Enchanting program specification synthesis by large language models using static analysis and program verification. In International Conference on Computer Aided Verification. Springer, 2024.
- Automatically inspecting thousands of static bug warnings with large language model: How far are we? ACM Transactions on Knowledge Discovery from Data, 2024.
- Impact of large language models on generating software specifications. arXiv preprint arXiv:2306.03324, 2023.
- Temporal logic specification mining of programs. Theoretical Computer Science, 857:29–42, 2021.
- Pptl specification mining based on lnfg. Theoretical Computer Science, 937:85–95, 2022.
- Cbmc–c bounded model checker: (competition contribution). In Tools and Algorithms for the Construction and Analysis of Systems: 20th International Conference, TACAS 2014, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2014, Grenoble, France, April 5-13, 2014. Proceedings 20, pages 389–391. Springer, 2014.
- Symbolic model checking using sat procedures instead of bdds. In Proceedings of the 36th annual ACM/IEEE Design Automation Conference, pages 317–320, 1999.
- Cpachecker: A tool for configurable software verification. In Computer Aided Verification: 23rd International Conference, CAV 2011, Snowbird, UT, USA, July 14-20, 2011. Proceedings 23, pages 184–190. Springer, 2011.
- Cpachecker with sequential combination of explicit-value analyses and predicate analyses: (competition contribution). In Tools and Algorithms for the Construction and Analysis of Systems: 20th International Conference, TACAS 2014, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2014, Grenoble, France, April 5-13, 2014. Proceedings 20, pages 392–394. Springer, 2014.
- What is a trace? a runtime verification perspective. In International Symposium on Leveraging Applications of Formal Methods, pages 339–355. Springer, 2016.
- Trace-based deductive verification. In Proceedings of 24th International Conference on Logic, volume 94, pages 73–95, 2023.
- Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, volume 8, pages 209–224, 2008.
- Klee symbolic execution engine in 2019. International Journal on Software Tools for Technology Transfer, 23:867–870, 2021.
- Charles Antony Richard Hoare. An axiomatic basis for computer programming. Communications of the ACM, 12(10):576–580, 1969.
- Fast: Formal specification driven test harness generation. In Tenth ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMCODE2012), pages 33–42. IEEE, 2012.
- A method for model based test harness generation for component testing. Journal of the Brazilian Computer Society, 14:7–23, 2008.
- Software verification: Testing vs. model checking: A comparative evaluation of the state of the art. In Hardware and Software: Verification and Testing: 13th International Haifa Verification Conference, HVC 2017, Haifa, Israel, November 13-15, 2017, Proceedings 13, pages 99–114. Springer, 2017.
- Vst-floyd: A separation logic tool to verify correctness of c programs. Journal of Automated Reasoning, pages 367–422, 2018.
- Model-based variance-stabilizing transformation for illumina microarray data. Nucleic Acids Research, 36:e11–e11, 2008.
- Safe to the last instruction: automated verification of a type-safe operating system. Communications of the ACM, 54:123–131, 2011.
- Model checking boot code from aws data centers. In Computer Aided Verification: 30th International Conference, CAV 2018, Held as Part of the Federated Logic Conference, FloC 2018, Oxford, UK, July 14-17, 2018, Proceedings, Part II 30, pages 467–486. Springer, 2018.
- Code-level model checking in the software development workflow at amazon web services. Software: Practice and Experience, 51(4):772–797, 2021.
- Unit testing of flash memory device driver through a sat-based model checker. In 2008 23rd IEEE/ACM International Conference on Automated Software Engineering, pages 198–207. IEEE, 2008.
- Linking functional requirements and software verification. In 2009 17th IEEE International Requirements Engineering Conference, pages 295–302. IEEE, 2009.
- Software verification in the google app-engine cloud. In Computer Aided Verification: 26th International Conference, CAV 2014, Held as Part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18-22, 2014. Proceedings 26, pages 327–333. Springer, 2014.
- Dirk Beyer. Status report on software verification: (competition summary sv-comp 2014). In Tools and Algorithms for the Construction and Analysis of Systems: 20th International Conference, TACAS 2014, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2014, Grenoble, France, April 5-13, 2014. Proceedings 20, pages 373–388. Springer, 2014.
- Understanding programming bugs in ansi-c software using bounded model checking counter-examples. In Integrated Formal Methods: 9th International Conference, IFM 2012, Pisa, Italy, June 18-21, 2012. Proceedings 9, pages 128–142. Springer, 2012.
- Comparison between cpbpv, esc/java, cbmc, blast, eureka and why for bounded program verification. arXiv preprint arXiv:0808.1508, 2008.
- Scalable verification framework for c program. In 2018 25th Asia-Pacific Software Engineering Conference (APSEC), pages 129–138. IEEE, 2018.
- Runtime verification prediction for traces with data. In International Conference on Runtime Verification, pages 148–167. Springer, 2023.
- Hardware-based runtime verification with embedded tracing units and stream processing. In International Conference on Runtime Verification, pages 43–63. Springer, 2018.