ConFL: Constraint-guided Fuzzing for Machine Learning Framework (2307.05642v1)
Abstract: As machine learning gains prominence in various sectors of society for automated decision-making, concerns have risen regarding potential vulnerabilities in ML frameworks. Nevertheless, testing these frameworks is a daunting task due to their intricate implementation. Previous research on fuzzing ML frameworks has struggled to effectively extract input constraints and generate valid inputs, leading to extended fuzzing durations for deep execution or revealing the target crash. In this paper, we propose ConFL, a constraint-guided fuzzer for ML frameworks. ConFL automatically extracting constraints from kernel codes without the need for any prior knowledge. Guided by the constraints, ConFL is able to generate valid inputs that can pass the verification and explore deeper paths of kernel codes. In addition, we design a grouping technique to boost the fuzzing efficiency. To demonstrate the effectiveness of ConFL, we evaluated its performance mainly on Tensorflow. We find that ConFL is able to cover more code lines, and generate more valid inputs than state-of-the-art (SOTA) fuzzers. More importantly, ConFL found 84 previously unknown vulnerabilities in different versions of Tensorflow, all of which were assigned with new CVE ids, of which 3 were critical-severity and 13 were high-severity. We also extended ConFL to test PyTorch and Paddle, 7 vulnerabilities are found to date.
- Effective Loop Fusion in Polyhedral Compilation Using Fusion Conflict Graphs. ACM Trans. Archit. Code Optim. 17, 4 (2020), 26:1–26:26.
- AIVul. 2022. vulsinfo. https://sites.google.com/view/aivul/.
- Restler: Stateful rest api fuzzing. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 748–758.
- Fudge: fuzz driver generation at scale. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 975–985.
- NTFuzz: Enabling type-aware kernel fuzzing on windows with static binary analysis. In 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 677–693.
- NtFuzz: Enabling Type-Aware Kernel Fuzzing on Windows with Static Binary Analysis. In 2021 IEEE Symposium on Security and Privacy (SP). 677–693. https://doi.org/10.1109/SP40001.2021.00114
- IvySyn: Automated Vulnerability Discovery in Deep Learning Frameworks. In USENIX Security Symposium (SEC).
- Difuze: Interface aware fuzzing for kernel drivers. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2123–2138.
- Fuzzing deep-learning libraries via automated relational API inference. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 44–56.
- Google. 2021. Tensorflow. https://www.tensorflow.org/api_docs/python/tf/dtypes.
- Google. 2022a. AFL. https://github.com/google/AFL.
- Google. 2022b. BoostedTreesCalculateBestFeatureSplit. https://www.tensorflow.org/api_docs/python/tf/raw_ops/BoostedTreesCalculateBestFeatureSplit.
- Google. 2022c. BoostedTreesCreateQuantileStreamResource. https://www.tensorflow.org/api_docs/python/tf/raw_ops/BoostedTreesCreateQuantileStreamResource.
- Google. 2022d. Building the Future of TensorFlow. https://blog.tensorflow.org/2022/10/building-the-future-of-tensorflow.html.
- HyungSeok Han and Sang Kil Cha. 2017. Imf: Inferred model-based fuzzer. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2345–2358.
- Chris Lattner and Vikram Adve. 2004. LLVM: A compilation framework for lifelong program analysis & transformation. In International Symposium on Code Generation and Optimization, 2004. CGO 2004. IEEE, 75–86.
- LLVM. 2022a. Clang. https://clang.llvm.org.
- LLVM. 2022b. libfuzzer. https://llvm.org/docs/LibFuzzer.html.
- Graph-Based Fuzz Testing for Deep Learning Inference Engines. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 288–299. https://doi.org/10.1109/ICSE43902.2021.00037
- PeachTech. 2022. peach. https://www.peach.tech/.
- SWIG. 2022. SWIG. https://github.com/swig/swig/.
- An exploratory study of deep learning supply chain. In Proceedings of the 44th International Conference on Software Engineering. 86–98.
- Automatic Unit Test Generation for Machine Learning Libraries: How Far Are We?. In Proceedings of the 43rd International Conference on Software Engineering (Madrid, Spain) (ICSE ’21). IEEE Press, 1548–1560. https://doi.org/10.1109/ICSE43902.2021.00138
- Free lunch for testing: Fuzzing deep-learning libraries from open source. arXiv preprint arXiv:2201.06589 (2022).
- Security risks in deep learning implementations. In 2018 IEEE Security and privacy workshops (SPW). IEEE, 123–128.
- Docter: Documentation-guided fuzzing for testing deep learning api functions. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 176–188.
- DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (Beijing, China) (ISSTA 2019). Association for Computing Machinery, New York, NY, USA, 146–157. https://doi.org/10.1145/3293882.3330579
- APICraft: Fuzz Driver Generation for Closed-source SDK Libraries. In 30th USENIX Security Symposium, USENIX Security 2021, August 11-13, 2021, Michael Bailey and Rachel Greenstadt (Eds.). USENIX Association, 2811–2828. https://www.usenix.org/conference/usenixsecurity21/presentation/zhang-cen