FlashFuzz: Adaptive Greybox and DL Fuzzing

• FlashFuzz is a dual-fuzzing approach that targets both complex C/C++ programs and deep learning APIs with distinct mechanisms.
• The system employs adaptive greybox fuzzing using quantitative branch-distance bitmaps and neural network-guided mutations to overcome hard predicates.
• It leverages LLM-synthesized harnesses to generate valid inputs for DL libraries like PyTorch and TensorFlow, enhancing coverage and bug discovery.

FlashFuzz is a name used for two distinct fuzzing systems in recent literature. In one line of work, the name is associated with the system presented as Finch, which addresses greybox fuzzing bottlenecks by learning quantitative branch-distance bitmaps and adaptively identifying hot-bytes on a min-Pareto seed frontier (Nguyen et al., 2023). In a later line of work, FlashFuzz denotes an LLM-assisted, coverage-guided fuzzing approach that automatically synthesizes C++ libFuzzer harnesses for PyTorch and TensorFlow APIs, translating raw byte streams into semantically valid deep-learning arguments and measuring backend branch coverage with llvm-cov (Qin et al., 18 Sep 2025). Both systems retain the core coverage-guided fuzzing premise, but they operationalize guidance at different layers: one at the level of branch predicates in instrumented programs, the other at the level of API harness generation and backend-kernel exploration.

1. Terminological scope and problem setting

The literature does not use FlashFuzz for a single monolithic framework. One system targets conventional greybox fuzzing of C/C++ programs with complicated branching conditions such as magic-byte comparisons, checksum tests, and nested if-statements. The other targets individual deep-learning library APIs, where the central obstacle is the absence of per-API harnesses that can map byte-level fuzzer inputs into valid tensors, shapes, dtypes, devices, enums, and numeric ranges (Nguyen et al., 2023, Qin et al., 18 Sep 2025).

Aspect	Quantitative/adaptive hot-bytes system	DL-library API system
Primary setting	10 real-world programs and LAVA-M	PyTorch and TensorFlow backend APIs
Core mechanism	Neural network predicts a branch-distance bitmap; mutation follows gradient-derived hot-bytes on a min-Pareto seed pool	LLM synthesizes C++ libFuzzer harnesses from templates, helpers, and API documentation
Main bottleneck addressed	Random byte-level mutation stalls on composite predicates	Coverage-guided fuzzing needs valid API-level inputs and backend-facing harnesses

In the branch-distance system, the critique is directed primarily at AFL’s random mutation strategy and at assumptions made by prior work such as Steelix and NEUZZ. Steelix assumes byte-to-byte satisfaction of branch conditions, which does not generalize to multi-byte arithmetic, checksums, or non-linear relations. NEUZZ assumes that hot-bytes can be identified once and remain useful throughout fuzzing, even though the set of uncovered branches changes over time. In the DL-library system, the critique is directed at model-level fuzzers, frontend API fuzzers, and constraint-driven approaches that do not use coverage guidance, thereby limiting path diversity, backend observability, throughput, or all three.

2. Quantitative branch-distance fuzzing

The branch-distance FlashFuzz system models a program $P$ as a CFG with basic blocks $B$ and edges $E$. An edge $e=(x,y)\in E$ is covered by input $t$ iff execution transitions from block $x$ to $y$, and overall coverage is defined as

$C=\bigcup_{t\in T} E_t.$

AFL’s edge encoding is written as

$$\mathrm{ID}_{(x,y)}=(\mathrm{ID}_x \gg 1)\oplus \mathrm{ID}_y.$$

The system’s central move is to replace binary coverage labels with a quantitative branch-distance bitmap. For an uncovered branch $br_n$ at node $B$0 with condition $B$1, branch distance is defined as

$B$2

where $B$3 is a constant denoting maximum distance. If $B$4 is true or false, then $B$5. The implementation optimizes runtime computation of $B$6 using $B$7, i.e. $B$8, as an efficient distance proxy with fine granularity (Nguyen et al., 2023).

For $B$9 uncovered branches, the branch-distance bitmap is

$E$0

Fuzzing is then cast as the multi-objective optimization problem

$E$1

Pareto dominance is defined in the standard way: $E$2 dominates $E$3 if $E$4 and $E$5.

Hot-bytes are the input positions whose changes most reduce one or more $E$6. Rather than relying on taint offsets or fixed byte importance, the system trains a neural network to predict normalized branch distances

$E$7

and then uses gradients with respect to input bytes. Gradient magnitudes identify importance, while gradient signs indicate whether increasing or decreasing a byte’s value is predicted to reduce distance to branch satisfaction.

This formalization is designed to handle conditions that are hard for unguided mutation. For a magic-byte-like predicate such as $E$8, the branch distance becomes

$E$9

The intended effect is to convert an exact-match barrier into a guided search problem with monotone distance reduction.

3. Adaptive objective selection, learning, and mutation

The branch-distance system does not optimize against all uncovered branches uniformly. It focuses on “just-missed branches,” defined by rule $e=(x,y)\in E$0: for any branch $e=(x,y)\in E$1 of node $e=(x,y)\in E$2 never visited by any seed, objective $e=(x,y)\in E$3 is dropped. The remaining objectives correspond to reachable branching nodes whose outgoing branches are not yet covered. This removes “future” objectives and concentrates learning on the current bottlenecks (Nguyen et al., 2023).

Seed scheduling is governed by a min-Pareto frontier. Rule $e=(x,y)\in E$4 removes a Pareto-optimal input $e=(x,y)\in E$5 if for every objective there exists some other input $e=(x,y)\in E$6 with $e=(x,y)\in E$7. The remaining seeds are those that are uniquely best for at least one objective. A greedy approximation computes, for each $e=(x,y)\in E$8, the set

$e=(x,y)\in E$9

sorts $t$0 by $t$1 in descending order, and iteratively selects seeds with non-empty $t$2 into the compact frontier $t$3. The reported complexity is $t$4.

The predictive model is a feed-forward MLP with one hidden fully-connected layer; the input layer uses ReLU and the output uses Sigmoid. It is implemented in PyTorch 1.6.0 and trained for 200 epochs per iteration. Inputs are byte vectors normalized by zero-padding to the longest seed length, and outputs are normalized branch-distance bitmaps $t$5 for the current set of just-missed branches. The loss is binary cross-entropy,

$t$6

with the convention that if $t$7 (maximum distance $t$8), then $t$9 to avoid meaningless loss for unreachable nodes.

Mutation is gradient-guided and grouped. After training, the system computes gradient scores $x$0 over input bytes. Larger $x$1 indicates hotter bytes, and $x$2 indicates the direction predicted to decrease distance. Mutation proceeds over ranges [(0, 2), (2, 4), (4, 8), …]: smaller groups are used for higher-importance bytes, and larger groups for lower-importance bytes. For each group, the mutator selects the top positions by importance and applies directional updates according to the gradient signs until bytes saturate at $x$3 or $x$4.

Implementation is split between a C executor and a Python mutator. An LLVM Pass instruments BranchInst and SwitchInst to capture operands $x$5 at runtime and computes xor-based distances. A separate shared memory bitmap stores branch distances alongside AFL’s coverage bitmap. Seed queue management uses directories, with pareto/ holding the current min-Pareto seeds and tmp/ holding generated mutants.

The mechanism is explicitly intended to support several hard predicate classes. For checksum tests, gradients indicate which bytes the model correlates with the checksum branch distance, and group mutations adjust many bytes jointly following gradient signs. For nested conditions, rule $x$6 delays tracking deeper branches until predecessor nodes become reachable, while min-Pareto scheduling retains seeds that are best for specific inner objectives. An illustrative branch-distance bitmap is given for two test inputs:

$x$7

showing why seeds that optimize different objectives remain valuable even before they add new coverage.

4. Empirical behavior of the branch-distance system

Evaluation was conducted on 10 real-world programs—drawn from binutils, libjpeg-turbo, libpng, bzip2, harfbuzz, libxml2, and tiff—and on the LAVA-M dataset. The real-world programs were fuzzed for 24 hours, with repeated runs except for NEUZZ, which used one run due to GPU constraints. All runs used Ubuntu 16.04 LTS, Intel Core i9, and 32GB RAM. Finch and Angora coverage were normalized on Finch-instrumented binaries using xor distance instrumentation (Nguyen et al., 2023).

The reported cumulative 24-hour edge coverage shows that Finch achieves the highest edge coverage on 6 of 10 programs and is second best on most of the remaining ones, with harfbuzz as the stated exception. Selected figures include xmllint at 12329 edges for Finch versus 11779 for MOPT and 10601 for NEUZZ; libpng at 3041 for Finch versus 2958 for MOPT and 2949 for Angora; objdump at 8363 for Finch versus 7377 for MOPT and 6234 for Angora; and nm at 6586 for Finch versus 5772 for MOPT and 5566 for Angora. Overall ranking across programs is reported as Finch > Angora > MOPT > FairFuzz > NEUZZ > AFL > TortoiseFuzz, and compared to NEUZZ, Finch achieves on average about 1.2× more edge coverage.

Stability across runs is described as relatively good. For objdump, Finch is reported at 8005/8655/std 270; for nm, at 6168/6837/std 298. The ablation on training labels compares branch-distance bitmaps with coverage bitmaps on the same min-Pareto data. Branch-distance labels yield higher new edge coverage in 6 of 9 programs, with examples including djpeg: 7 vs 2, libpng: 56 vs 13, nm: 104 vs 69, and strip: 147 vs 135. Coverage labels can help when many easy-to-cover branches remain, with examples tiffcp: 353 vs 227 and objdump: 298 vs 237.

The seed-pool ablation reports that the min-Pareto pool size is approximately 11.72% of AFL’s seed pool on average. Executions are approximately 11.11% of AFL’s, yet the system achieves approximately 87.48% of AFL’s edge coverage and sometimes surpasses AFL, as noted for libpng. This is presented as evidence of reduced redundancy and improved efficiency from focused training data. Training overhead over a 24-hour campaign averages approximately 0.54 hours, ranging from negligible on small programs such as bzip2 to approximately 1.19 hours on xmllint, where many just-missed branches are present.

Bug discovery on LAVA-M is reported as strongest in 3/4 programs, with a unique result on md5sum at 61 bugs while other tools reach at most 60. The full Finch counts are base64 48, md5sum 61, uniq 29, and who 2229. The paper states that many unlisted LAVA-M bugs were discovered and that md5sum and uniq bugs were triggered rapidly, with 48 and 29 respectively in less than one hour and 61 in md5sum by six hours. On real-world applications with ASAN, Finch discovered a previously unknown bug in tiffcp; Angora found a bug in strip; and no other unique bugs were reported in the 24-hour window.

The stated limitations are also significant. The xor proxy may not precisely reflect semantic distance for complex operations or certain data types. Some comparisons fall back to distance 0 when operands cannot be derived. Early guidance is limited, and performance curves often start below AFL variants before overtaking them after sufficient learning, approximately 15 hours in several programs. The implementation targets C/C++ with LLVM instrumentation, and extension to other languages or binary-only scenarios is described as requiring additional engineering.

5. LLM-synthesized harnesses for DL library APIs

In the later work, FlashFuzz is an LLM-assisted, coverage-guided fuzzing approach for testing individual deep-learning library APIs in PyTorch and TensorFlow. Its core claim is that if one can automatically generate per-API fuzz harnesses that translate raw libFuzzer byte streams into semantically valid API inputs, then established coverage-guided fuzzing engines can scale to DL libraries and achieve higher coverage, higher validity, faster throughput, and more bug discoveries than existing DL fuzzers that lack coverage guidance (Qin et al., 18 Sep 2025).

The architecture centers on automatic synthesis of C++ harnesses for backend APIs. Each harness follows the libFuzzer entry-point convention $y$1 and is structured to parse bytes into valid DL arguments, call the target operator in the backend, and optionally run differential oracles such as CPU-vs-GPU comparison. Templates provide includes, namespace setup, backend session or scope initialization, and error handling. Helper functions perform reusable parsing steps: parseDataType(uint8_t selector), parseRank(uint8_t byte), parseShape(data, offset, total_size, rank), and createTensor(...).

Constraint extraction is documentation-driven rather than grammar-driven. The system fetches API documentation via __doc__ and includes it in the LLM prompt, together with the template, helper functions, and explicit instructions such as mapping bytes to dtype via modulo, prefilling required stride ends with 1, deriving filter channels from input channels, and rejecting invalid seeds by returning -1. If generated code fails to compile, compiler diagnostics are appended to the prompt. If the harness does not call the correct focal API, the prompt is updated with the expected call signature. During fuzzing, frequent runtime exceptions or low validity trigger stronger constraints in helper functions, such as masking shapes to nonzero or enforcing nonzero divisors.

The system integrates with libFuzzer in-process using -fsanitize=fuzzer, with coverage instrumentation through -fprofile-instr-generate and -fcoverage-mapping. During harness development, -fsanitize=fuzzer-no-link is used; final fuzzing uses -fsanitize=fuzzer so that libFuzzer can drive mutations using sanitizer coverage hooks. Coverage is filtered to backend kernel directories only: aten/src/ATen/native for PyTorch and tensorflow/core/kernels for TensorFlow.

Byte-to-argument mapping is intentionally simple and constraint-aware. Dtype selection typically uses selector % |allowed_dtypes|. Rank is computed as byte % (MAX_RANK - MIN_RANK + 1) + MIN_RANK, often with a small maximum such as 4. Shapes are built with each dimension length as next_byte % MAX_DIM_LEN, often 16, to avoid enormous allocations. Tensor values are read according to dtype size. If insufficient bytes remain, missing bytes default to 1. Dependent arguments are reconstructed from already parsed values; for example, filter in_channels can be derived from an input tensor’s last dimension, and both endpoints of Conv2D strides can be fixed at 1.

The framework also supports differential oracles. Output divergence is flagged when relative tolerance exceeds 1e-2 or absolute tolerance exceeds 1e-3 across devices. Exception divergence is flagged when one device throws and the other succeeds. Validity is defined as

$x$8

where valid inputs are those that execute successfully without input-validation errors at the target operator. Backend branch coverage is defined as

$x$9

and input-generation speedup under identical time budgets is

$y$0

A concrete worked example is given for torch.fmod. The first byte may determine dtype, the next byte rank, the next bytes tensor shape, and the remaining bytes tensor values. Mutating dtype or rank changes both semantics and buffer consumption. This variability is presented as beneficial because it allows coverage-guided mutation to explore diverse operator behaviors while still respecting API constraints.

6. Evaluation, bug discovery, and limitations in DL-library fuzzing

The DL-library FlashFuzz study synthesizes harnesses for 1,151 PyTorch APIs out of 1,576 candidates and 662 TensorFlow APIs out of 1,452 candidates. Coverage and validity experiments use PyTorch 2.2.0 and TensorFlow 2.16.1; bug detection experiments use PyTorch 2.7.0 and TensorFlow 2.19.0. Coverage calculation and most experiments run on two CPU-only machines with AMD EPYC 9684X, while TitanFuzz input generation uses two machines with AMD EPYC 7742 and 8× NVIDIA RTX V100 GPUs. Time budgets are 10 minutes per API for coverage and validity, and 8 hours per API for long “in the wild” campaigns. FlashFuzz uses Claude Sonnet 4.0 for harness generation, and TitanFuzz’s original Incoder-1B is updated to qwen2.5-coder:7b for comparison (Qin et al., 18 Sep 2025).

Across common API sets, FlashFuzz is reported to achieve 101.13%–212.88% higher branch coverage than ACETest, PathFinder, and TitanFuzz. The throughput and validity tables are particularly large. For PyTorch, compared with ACETest, FlashFuzz increases total inputs from 9,267.4K to 158,027.2K (17.1×), valid inputs from 4,918.4K to 104,971.4K (21.3×), and validity from 53.07% to 66.43% (1.3×). Compared with PathFinder, total inputs increase from 58,333.2K to 270,083.6K (4.6×) and valid inputs from 36,126.7K to 170,117.1K (4.7×), while validity rises from 61.93% to 62.99% (1.0×). Compared with TitanFuzz, total inputs increase from 206.8K to 244,433.4K (1182.0×) and valid inputs from 126.6K to 160,943.9K (1271.3×), with validity rising from 61.22% to 65.84% (1.1×).

For TensorFlow, compared with ACETest, total inputs move from 38,451.8K to 39,047.2K (1.0×), but valid inputs rise from 14,843.5K to 37,183.2K (2.5×) and validity from 38.60% to 95.23% (2.5×). Compared with PathFinder, total inputs increase from 20,392.2K to 34,662.0K (1.7×), valid inputs from 3,605.1K to 32,866.4K (9.1×), and validity from 17.68% to 94.82% (5.4×). Compared with TitanFuzz, total inputs increase from 110.7K to 59,252.4K (535.3×), valid inputs from 60.0K to 54,549.7K (909.2×), and validity from 54.23% to 92.06% (1.7×).

Bug discovery in the 8-hour campaigns yields 42 previously unknown bugs, split as 20 in PyTorch and 22 in TensorFlow. Developers confirmed approximately 87%, specifically 20/25 PyTorch reports and 21/22 TensorFlow reports, and 8 bugs were fixed, namely 7 in PyTorch and 1 in TensorFlow. Reported categories among confirmed and pending bugs are Aborted (10), Segfault (8), Floating-Point Exception (7), Memory Overflow (1), Internal Exception (2), and CPU-GPU inconsistency (14). Representative examples include a torch.fmod Floating Point Exception with large negative modulo value, a torch.combinations performance hang due to Cartesian construction, a ResourceSparseApplyProximalAdagrad crash under large magnitude inputs, and a BiasAdd GPU abort for 2D input with data_format="NCHW" while CPU succeeds.

The ablation study attributes a substantial fraction of this performance to documentation and helper functions. Removing documentation decreases coverage and validity, with TensorFlow validity dropping by approximately 31% and TensorFlow coverage halving. Removing helpers also reduces coverage and crashes found. Removing both yields the lowest validity and coverage.

The stated limitations are nontrivial. Harness generation success rates are 73% for PyTorch and 46% for TensorFlow. Some validity checks exist only in Python frontends, so backend-only harnesses may admit inputs that the frontend would reject; triage therefore includes reproducing bugs via Python API calls before reporting. Stateful APIs and GPU-only kernels are harder to exercise consistently, and nondeterminism can obscure differential oracles. The throughput-oriented configuration uses max_len=128, mutation depth 8, an in-process runner, and two threads overlapping I/O, but further domain-specific mutators are described as a possible improvement. The study focuses on PyTorch and TensorFlow, though it states that the lessons likely generalize to similar backends such as JAX.

Taken together, the two FlashFuzz lines show a shared methodological pattern: conventional coverage-guided fuzzing is retained, but the major source of leverage is shifted from undirected mutation toward learned or synthesized structure. In the branch-distance system, that structure is a quantitative model of predicate satisfaction over a dynamically maintained min-Pareto frontier. In the DL-library system, it is an automatically generated, documentation-aware harness that makes byte-level mutation semantically meaningful for backend APIs.