This paper, "Fuzzing Frameworks for Server-side Web Applications: A Survey" (Dharmaadi et al., 5 Jun 2024 ), provides a comprehensive review of the state-of-the-art in fuzzing techniques applied to server-side web applications via their Web APIs. It highlights the growing importance of securing web applications due to their widespread use and the increasing complexity of their APIs, noting that traditional binary fuzzing techniques are often not directly applicable. Web APIs, especially RESTful ones, present unique challenges, such as requiring syntactically and semantically valid inputs to interact effectively and the need to handle request dependencies and state. The survey aims to bridge this gap by analyzing how existing frameworks address these challenges and identifying areas for future research.
The authors collected 53 relevant papers published between 2013 and 2023 from various research repositories. They filtered papers based on their focus, specifically targeting studies that propose or improve web API fuzzers for server-side applications. The analysis categorizes existing web API fuzzers into two main approaches:
- Standard Fuzzing: Focuses on finding any bug or crash by exploring as much of the application's code and states as possible.
- Vulnerability-driven Fuzzing: Targets specific types of vulnerabilities using tailored mutators and checkers.
The paper details a general workflow for web API fuzzers, breaking it down into four crucial processes:
- Request Template Generation: This process involves creating the structure of HTTP requests. A major challenge is understanding request dependencies (e.g., creating a resource before deleting it). Solutions include building dependency graphs (inferring relationships from OpenAPI specs or CRUD semantics), using tree-based dependency models for efficiency, employing pre-defined templates, annotating OpenAPI specifications, or simply having no explicit dependency model for single-request fuzzing. Another problem is generating long request sequences to reach deeper states, addressed by techniques like length-oriented selection and smart sampling. For web applications without OpenAPI specifications, frameworks use HTML or JavaScript crawlers to discover endpoints and parameters or rely on manual input/corpus. Inferring specific data types beyond basic strings/numbers (Format-encoded Type inference) is also discussed to narrow down input space effectively.
- Template Rendering: This step populates the generated templates with concrete values. A key issue is obtaining initial valid values, often solved by using default dictionaries or extracting example values from OpenAPI specifications. Generating semantically valid values is challenging as OpenAPI typically lacks this information. Approaches include leveraging external knowledge bases like DBPedia or grouping semantically similar parameters. Handling inter-parameter dependencies (where one parameter's value depends on another's) is also critical and addressed through NLP-based dependency inference, using specific languages like IDL, or employing deep learning models to predict request validity.
- Execution and Getting Feedback: Fuzzers send generated requests and analyze the responses. Feedback is vital for guiding the mutation process. Common feedback types include HTTP response codes (especially 5xx for crashes), code coverage (requires instrumentation), branch distance (how close an input is to satisfying a condition), taint feedback (tracking input flow), Test Coverage Level (TCL, a black-box metric), and sometimes specific application-level messages like XML responses. Instrumentation, necessary for grey/white-box feedback like code coverage or branch distance, is often done via source code augmentation (e.g., using Java agents, Babel plugins for JavaScript, or AST parsing for PHP) or interpreter augmentation (modifying the web server's interpreter). Regarding observed vulnerabilities, most standard fuzzers report crashes (5xx responses). Vulnerability-driven fuzzers look for specific issues like resource violations (use-after-free, resource leaks), mass assignment, XSS, SQL injection, command injection, etc., often using custom checkers or analyzing responses for specific patterns or payloads.
- Mutation: This is the core fuzzing process where existing inputs are modified to explore the input space. Simple value mutation (random changes) is adapted for web parameters (string/number modifications). Generating more valid requests often uses response dictionaries (reusing values from successful responses, especially for generated IDs), corpus mutation (blending parameters from stored requests), adaptive hypermutation (focusing mutation on parameters with higher impact), attention-based models (learning parameter relationships), or Many Independent Objective (MIO) algorithms for covering multiple objectives. Generating more invalid requests, crucial for triggering error handling code, uses techniques like constraint violation (ignoring schema rules), rule-based schema mutators (modifying the structure of request bodies), vulnerability dictionaries (injecting known exploit payloads), and tracked fault generators (systematically inserting faults).
The paper reviews how researchers evaluate their fuzzers, primarily using:
- Public WUTs (43% of studies): Online services like those on APIS.Guru, Azure, GitLab, GitHub. Challenges include black-box nature and rate limiting/authentication issues.
- Self-developed benchmarks (39%): Open-source web applications (Java, PHP, Node.js, etc.), sometimes with manually injected bugs (e.g., EvoMaster Benchmark, PHP benchmarks with XSS).
- Third-party benchmarks (18%): Known vulnerable test-bed applications like WebGoat, Gruyere, Nodegoat, DVWA.
Finally, the survey identifies several open challenges:
- Effectiveness of Instrumentation: While beneficial for coverage, instrumentation introduces overhead and complexity, requiring recompilation/rebuilding, which can be slow in fast-paced development environments.
- Complexity of Handling Microservice Architecture: Testing interactions between independent services is difficult, especially in identifying which specific service is responsible for a failure when testing the API gateway.
- Difficulty of Testing Public WUTs: Lack of source code access limits deep testing, and rate limits/authentication complexity hinders extensive campaigns. Testing large open-source WUTs is also difficult due to their complexity.
- Low-Quality Corpus: Initial corpora derived solely from documentation might be incomplete, limiting the fuzzer's exploration. Minimizing the corpus to keep it powerful is a potential area for improvement.
- Lack of Web API Fuzzing Benchmarks: Unlike binary fuzzing, there is no widely accepted, comprehensive benchmark with annotated bugs and standardized metrics for fair comparison of web API fuzzers.
Potential future research directions highlighted include extending fuzzing to web client programming (e.g., WebAssembly), mobile web applications, leveraging generative AI for input generation, and developing strategies to detect a wider variety of web security vulnerabilities beyond crashes and common injection/XSS flaws, including novel or complex bugs.