NEUZZ: Efficient Fuzzing with Neural Program Smoothing (1807.05620v4)

Abstract: Fuzzing has become the de facto standard technique for finding software vulnerabilities. However, even state-of-the-art fuzzers are not very efficient at finding hard-to-trigger software bugs. Most popular fuzzers use evolutionary guidance to generate inputs that can trigger different bugs. Such evolutionary algorithms, while fast and simple to implement, often get stuck in fruitless sequences of random mutations. Gradient-guided optimization presents a promising alternative to evolutionary guidance. Gradient-guided techniques have been shown to significantly outperform evolutionary algorithms at solving high-dimensional structured optimization problems in domains like machine learning by efficiently utilizing gradients or higher-order derivatives of the underlying function. However, gradient-guided approaches are not directly applicable to fuzzing as real-world program behaviors contain many discontinuities, plateaus, and ridges where the gradient-based methods often get stuck. We observe that this problem can be addressed by creating a smooth surrogate function approximating the discrete branching behavior of target program. In this paper, we propose a novel program smoothing technique using surrogate neural network models that can incrementally learn smooth approximations of a complex, real-world program's branching behaviors. We further demonstrate that such neural network models can be used together with gradient-guided input generation schemes to significantly improve the fuzzing efficiency. Our extensive evaluations demonstrate that NEUZZ significantly outperforms 10 state-of-the-art graybox fuzzers on 10 real-world programs both at finding new bugs and achieving higher edge coverage. NEUZZ found 31 unknown bugs that other fuzzers failed to find in 10 real world programs and achieved 3X more edge coverage than all of the tested graybox fuzzers for 24 hours running.

An Analysis of NEUZZ: Efficient Fuzzing with Neural Program Smoothing

The paper "NEUZZ: Efficient Fuzzing with Neural Program Smoothing" introduces a novel approach to software fuzzing by leveraging neural networks (NNs) to improve the efficiency of bug detection. The central idea is to employ surrogate neural network models to create smooth approximations of a program's discrete control flow, enabling the application of gradient-guided optimization techniques to the fuzzing process.

Key Contributions

The authors make several significant contributions:

1. Program Smoothing: The paper identifies program smoothing as a critical step for applying gradient-guided techniques to fuzzing. Utilizing neural networks, they approximate a program’s branching behavior, allowing for smooth gradient computations, which are traditionally hindered by the discontinuous nature of program execution paths.
2. Incremental Learning with NNs: The paper proposes using feed-forward neural networks to model program behaviors and incrementally refine these models as new training data becomes available. This allows NEUZZ to adaptively enhance its fuzzing strategy over time.
3. Gradient-Guided Optimization: By computing the gradient from the neural network's predictions, NEUZZ guides the mutation of input data to efficiently explore new execution paths in the program, facilitating the discovery of software vulnerabilities.
4. Extensive Evaluation: NEUZZ is evaluated against ten state-of-the-art fuzzers, such as AFL and AFLFast. On ten real-world programs, NEUZZ achieved significantly better results in both bug discovery and edge coverage. Notably, it uncovered 31 previously unknown bugs and exceeded edge coverage of all tested fuzzers by a factor of three.

Numerical Results and Implications

NEUZZ's results are quantifiably strong, particularly in its ability to find previously undetected vulnerabilities. It identified 31 unknown bugs, including two CVEs, and achieved three times the edge coverage of other graybox fuzzers during 24-hour testing sessions. Furthermore, the evaluation on benchmark datasets like LAVA-M and DARPA CGC further underscores NEUZZ’s ability to outperform existing solutions.

The implications of this research are wide-ranging. Practically, the increased efficiency and effectiveness of NEUZZ in identifying software vulnerabilities can lead to more secure software development practices. Theoretically, it demonstrates the potential of integrating machine learning techniques, particularly neural networks, with traditional software testing methods. This fusion can be pioneered in other domains of software engineering that rely on structured input exploration.

Future Directions

Opportunities for the further development of AI-enhanced fuzzing tools are evident. Future work could explore:

• Scalability: Extending NEUZZ to handle even larger software projects and more complex application environments.
• Integration: Combining NEUZZ with other AI-driven techniques, such as reinforcement learning, which could dynamically adjust fuzzing strategies based on ongoing results.
• Optimization Algorithms: Delving deeper into the types of gradient-guided optimization algorithms to further improve input mutation strategies.

In conclusion, the NEUZZ paper presents a compelling case for the integration of neural networks into fuzzing to enhance bug discovery processes. Its contributions could herald a new era in software testing methodologies, leveraging the strengths of AI to optimize and streamline the handling of complex, real-world software systems.