Web Fuzzing Commons (WFC) Framework

• Web Fuzzing Commons (WFC) is an open-source framework offering declarative schemas for authentication and fault reporting in REST API fuzzing.
• It standardizes authentication handling by auto-generating parsers for static and dynamic token flows (e.g., JWT, Basic Auth), thereby enabling consistent fuzzer comparisons.
• Combined with the Web Fuzzing Dataset (WFD), WFC provides a reproducible and scalable environment for evaluating and benchmarking diverse REST API fuzzing tools.

Web Fuzzing Commons (WFC) is a set of open-source libraries and schema definitions to declaratively specify authentication info and catalog different types of faults that fuzzers can automatically detect. It was introduced together with Web Fuzzing Dataset (WFD), a collection of 36 open-source APIs with all necessary scaffolding to easily run experiments with fuzzers, supported by WFC. The framework is motivated by three major issues that hinder further progress in REST API fuzzing: how to deal with API authentication; how to catalog and compare different fault types found by different fuzzers; and what to use as case study to facilitate fair comparisons among fuzzers (Sahin et al., 1 Sep 2025).

1. Scope and stated objectives

WFC is described as a small, open-source library of declarative schema definitions (JSON Schema) for specifying API authentication configurations in a fuzzer-agnostic way, reporting the results of fuzzing runs according to a unified “fault taxonomy,” and a ready-to-use HTML-based visualization front end. In this design, the common unit of exchange is not an implementation-specific script or report format, but a machine-readable schema that multiple fuzzers can share.

The stated goals are operational rather than theoretical. WFC is intended to remove “reinventing the wheel” every time a new fuzzer needs a way to feed authentication tokens or secrets; allow different fuzzers to share a single, machine-readable format for authentication setup and fuzzer-detected faults; enable standardized, side-by-side comparisons of fault-finding capabilities; and improve fuzzer usability in industrial settings by bundling common utilities such as parsers and report viewers. The accompanying paper shows the usefulness of WFC/WFD with EvoMaster, a state-of-the-art fuzzer for Web APIs, and compares EvoMaster with ARAT-RL, EmRest, LLamaRestTest, RESTler, and Schemathesis; however, the framework is explicitly presented as beneficial to any fuzzer (Sahin et al., 1 Sep 2025).

2. Declarative authentication model

A central component of WFC is a single JSON Schema that describes how users can declare, in YAML or JSON, both static and dynamic authentication flows. The motivation is that real-world REST APIs almost always enforce authentication—static (Basic Auth), dynamic (JWT/OAuth2), cookie sessions, and related mechanisms—while each fuzzer has traditionally invented its own mini-DSL or script-based approach. WFC replaces this fragmentation with a declarative schema from which fuzzer libraries can auto-generate parsers, for example via jsonschema2pojo in Java (Sahin et al., 1 Sep 2025).

The paper provides two illustrative configurations. One concerns form-login with cookies:

auth: - name: ADMIN loginEndpointAuth: payloadRaw: "username=admin&password=admin" - name: user1 loginEndpointAuth: payloadRaw: "username=user1&password=password" authTemplate: loginEndpointAuth: endpoint: /login verb: POST contentType: application/x-www-form-urlencoded expectCookies: true

The other concerns an API returning a Bearer-token JSON field:

auth: - name: admin loginEndpointAuth: payloadRaw: '{"usernameOrEmail":"admin","password":"bar123"}' - name: user loginEndpointAuth: payloadRaw: '{"usernameOrEmail":"user","password":"bar123"}' authTemplate: loginEndpointAuth: endpoint: /api/auth/signin verb: POST contentType: application/json token: extractFromField: /accessToken httpHeaderName: Authorization headerPrefix: "Bearer "

Under the hood, WFC provides an auth.jsonschema whose top-level structure includes schema_version, auth, and authTemplate, with auth required and AuthenticationInfo defined in $defs. The paper further states that an AuthenticationInfo object must have either staticHttpAuth { username, password }, or loginEndpointAuth { endpoint, verb, payloadRaw or payloadTemplate, tokenExtraction }. This gives WFC a declarative representation for both credentials and dynamic token specifications, while keeping the concrete execution logic in reusable library code rather than in per-tool ad hoc scripts (Sahin et al., 1 Sep 2025).

3. Fault taxonomy and report schema

The second core artifact is a report schema for recording faults according to a unified taxonomy. The stated motivation is that different fuzzers detect oracles beyond HTTP 500—schema mismatches, robustness failures, SQL-injection, access-control violations, and related conditions—but each reports them in ad hoc formats. WFC addresses this with a report.jsonschema and a fault_categories.json listing unique fault codes (Sahin et al., 1 Sep 2025).

The report schema excerpt given in the paper includes the following fields:

{ "$schema":"https://json-schema.org/draft/2020-12/schema", "$id":".../wfc/schemas/report.yaml", "title":"Web Fuzzing Commons Report", "type":"object", "properties": { "schema_version": { "type":"string" }, "tool_name": { "type":"string" }, "tool_version": { "type":"string" }, "creation_time": { "type":"string","format":"date-time" }, "faults": { "%%%%0%%%%defs/Faults" }, "problem_details": { "rest": { "%%%%1%%%%defs/RESTReport" } }, "total_tests": { "type":"integer","minimum":0 }, "test_file_paths": { "type":"array","items":{ "%%%%2%%%%defs/TestFilePath" } }, "test_cases": { "type":"array","items":{ "%%%%3%%%%defs/TestCase" } } }, "required":["schema_version","tool_name","faults",...], "$defs": { /* definitions of Faults, TestCase, etc. */ } }

The companion taxonomy maps integer codes to semantic labels. The examples given are:

{ "100": "HTTP 500 Internal Server Error", "101": "Response not valid according to schema", "200": "SQL injection vulnerability", "204": "Access-control bypass (unauthenticated request)" }

The text also refers to codes such as F100="HTTP 500 internal server error", F101="Schema-validation failure", and F204="RBAC violation". Each fault is reported with fault_code, endpoint path, and optional context, such as a stack-trace line or schema path. This standardization is significant because it moves comparison beyond raw HTTP 500 counts and permits side-by-side analysis of heterogeneous oracle sets, while still recognizing that reports remain self-reported and may need external cross-checking (Sahin et al., 1 Sep 2025).

4. Runtime components and adoption workflow

WFC is not only a schema repository. It ships as a small Maven (JVM) artifact—or equivalent Python/PIP module in future—containing the JSON Schemas (auth.yaml, report.yaml, fault_categories.json), code generators via jsonschema2pojo for Java parsers, a minimal runtime, and a static web directory (index.html plus JS/CSS assets). The runtime reads an auth-config file, drives HTTP calls to obtain tokens, injects tokens into each test request, and exposes an API for fuzzers to record detected faults according to the standardized taxonomy. Given a report.json conforming to WFC, the static assets generate an interactive HTML report (Sahin et al., 1 Sep 2025).

The paper describes six steps for incorporating WFC into a new or existing fuzzer:

1. add the WFC library as a dependency;
2. parse auth.yaml to instantiate user credentials/token-flows;
3. before sending each request, consult the parsed auth info for each user to refresh or reuse tokens;
4. whenever an oracle fires, call the WFC report API to record fault code, endpoint, and context;
5. at the end, serialize all collected data into report.json;
6. drop in the WFC static folder so a test engineer can simply run webreport.py and get a browsable HTML summary.

The code fragments in the paper instantiate this workflow concretely. Authentication parsing is shown with AuthConfigParser.fromFile("ocvn-auth.yaml"); token acquisition is illustrated with WfcHttpClient.login(auth,"ADMIN").extractCookies(); fault recording is exemplified by reporter.recordFault(100,response.getPath(), context="line 353"); and report generation includes setting tool_name, tool_version, and creation_time, followed by reporter.writeReport(Paths.get("report.json")). In this arrangement, WFC functions as a common support layer for authentication, fault recording, and post-run visualization rather than as a fuzzing algorithm in itself (Sahin et al., 1 Sep 2025).

5. Web Fuzzing Dataset (WFD)

WFD is a companion GitHub repository and Zenodo archive that bundles 36 open-source, JVM-based REST APIs together with the scaffolding needed to run large experimental campaigns. The repository includes Docker-Compose files to bring up each API and its external dependencies, preloaded initialization scripts such as user/admin accounts in the database, JaCoCo instrumentation out-of-the-box, mitmproxy logging, WFC authentication config files, and scripts in Bash and Python to launch a large grid of experiments over APIs, repetitions, and fuzzers (Sahin et al., 1 Sep 2025).

The numerical profile reported for WFD is as follows:

Attribute	Value
Open-source APIs	36
Source files	6 465
LOC	657 162
Endpoints	1487
APIs requiring auth	15

The paper characterizes WFD as the largest publicly available corpus of REST APIs with complete “run, fuzz, analyze” scaffolding. It also describes the dataset as a living repository in which new community-driven APIs can be added, strengthening representativeness. A plausible implication is that WFD is intended not merely as a benchmark snapshot but as a community-maintained experimental substrate for reproducible REST API fuzzing studies (Sahin et al., 1 Sep 2025).

6. Experimental role, comparison criteria, and methodological cautions

The usefulness of WFC/WFD is demonstrated through an empirical study comprising 2 160 runs across 6 fuzzers and 36 APIs. On this basis, the authors identify common pitfalls in tool comparisons and propose guidelines intended to support large, fair, and reproducible REST-API-fuzzer comparisons (Sahin et al., 1 Sep 2025).

One guideline concerns artifact selection. The recommendation is to always start from the previous studies’ corpus, denoted $z_n$, and add new APIs, denoted $k$, rather than dropping APIs, to avoid bias. The paper also states that community-managed corpora such as WFD minimize cherry-picking. Another guideline concerns tool usability requirements. A candidate fuzzer should satisfy the following conditions:

1. open-source or deposited in long-term archive;
2. one-time install cost, $O(1)$ wrt #APIs;
3. at least three CLI args: schema path, API base URL, timeout;
4. optional: WFC auth config support;
5. optional: custom output-folder flag;
6. if using LLM, must support a local model for reproducibility and cost control;
7. must generate executable test suites, for example PyTest or JUnit.

The paper treats authentication and fault reporting as particularly important sources of unfairness in comparisons. It recommends not relying on per-tool ad hoc scripts, but adopting WFC’s auth.yaml so all fuzzers share the same credentials. It likewise recommends standardizing on WFC’s fault codes to compare diverse oracles beyond HTTP 500, while emphasizing that such reports are self-reported and should be cross-checked with external measures such as coverage and proxy logs.

The most explicit methodological warning concerns black-box fuzzing. The paper states that in black-box fuzzing, the only artifacts a test engineer ultimately uses are the generated test suites; therefore, comparisons of coverage during the search, such as in-memory JaCoCo, are misleading in pure black-box mode. A fuzzer must minimize its half-million HTTP calls into a small, high-quality test suite. On this basis, the comparative metrics proposed by the authors include code coverage of the generated tests, mutation score as future work, flakiness or failures when executing the suite, and number and types of assertions. These recommendations frame WFC less as a narrow exchange format than as a methodological attempt to standardize authentication handling, fault cataloging, and experimental scaffolding across REST API fuzzing studies (Sahin et al., 1 Sep 2025).