GPU-native Fuzzing Pipeline

• GPU-native fuzzing pipeline is a dynamic software testing architecture that instruments live device execution to uncover deep bugs in GPU codebases.
• It employs domain-specific input mutations and formal constraint modeling to enhance coverage and detect memory issues in frameworks like TensorFlow and CUDA.
• The approach integrates device-level instrumentation, sanitization, and persistent execution models to significantly boost bug discovery and code coverage.

A GPU-native fuzzing pipeline is a dynamic software testing architecture that performs automated, feedback-guided input generation and error detection directly on GPU-accelerated code, maintaining architectural faithfulness by instrumenting actual device-side execution. Recent advances in this field have been driven by the need to assure memory safety and correctness within the rapidly diversifying GPU stack—including machine learning frameworks (TensorFlow, PyTorch, PaddlePaddle), CUDA libraries, and modern browser shader pipelines. Unlike traditional approaches focused on CPU or on CPU-simulated GPU code, GPU-native pipelines integrate device-level coverage, memory sanitization, and domain-specific input modeling to efficiently discover deep bugs in heterogeneous, high-performance codebases (Li et al., 11 Feb 2026, Ziad et al., 12 Mar 2026, Li et al., 5 Mar 2026, Bernhard et al., 2024).

1. Architectural Principles of GPU-Native Fuzzing

GPU-native fuzzing architectures fundamentally differ from CPU-oriented fuzzing in two respects: (1) instrumenting actual device-side execution for coverage and error monitoring, and (2) maintaining semantic faithfulness concerning GPU concurrency, memory hierarchies, and timing hazards. The typical pipeline orchestrates the following stages:

1. Initialization: Target binaries or frameworks are loaded, and a binary instrumentation layer (e.g., NVIDIA NVBit) is attached to observe GPU activity and inject runtime checks (Li et al., 5 Mar 2026, Ziad et al., 12 Mar 2026).
2. Test Input Generation: Inputs are generated or mutated either in a type-aware (e.g., operator parameter modeling) or generic (e.g., seed file mutation) manner.
3. Kernel Dispatch & Execution: Mutated or synthesized inputs are transferred to GPU memory, and the relevant kernels or device code are launched under instrumentation.
4. Inline Device Instrumentation: Every relevant memory or control-flow instruction is optionally rewritten to check memory access safety, gather coverage, and trap on violations.
5. Coverage & Error Feedback Collection: Upon kernel completion or error, metadata and diagnostic logs are retrieved to guide further fuzzing.
6. Corpus Management & Scheduling: New inputs are retained if they increase coverage or trigger sanitizer violations.

While specific implementations (e.g., GPU-Fuzz, cuFuzz, DarthShader) differ in the downstream analysis and domain-specialized stages, maintaining GPU execution fidelity is a non-negotiable design constraint (Li et al., 5 Mar 2026).

2. Input Modeling and Mutation Strategies

Coverage and semantic depth are determined by how input test cases are synthesized or mutated:

• Formal Constraint Modeling (GPU-Fuzz): Abstracts operator parameters (shape, stride, dilation) as symbolic variables, encoding their interdependencies as SMT constraints. An SMT solver (e.g., Z3) is used to systematically explore legal and boundary parameter settings, often focusing on edge cases likely to provoke memory violations. For example, the output dimension of a 2D convolution is computed by

$$H_{out} = \frac{H_{in} + 2P_h - D_h (K_h-1) - 1}{S_h} + 1$$

This approach rapidly discovers boundary errors in tensor operators (Li et al., 11 Feb 2026).

• Type-aware Mutations (Challenges/Design paper): Input integers are perturbed with boundary values (e.g., 0, INT_MAX), floating-point bitwise mutations (sign, mantissa, NaN/Inf injection), pointer region swaps, and combinatorial reshaping of arrays, ensuring deep coverage of hardware-specific access checks. Dimensional and type-consistency is prioritized to avoid artifacts of invalid host-side rejection (Li et al., 5 Mar 2026).
• Generic Seed File Mutation (cuFuzz): Conventional coverage-guided fuzzing (AFL++) mutates input bitstreams of various formats (images, integer sequences) using random and dictionary-driven mutators. Device and host coverage joint feedback prunes the corpus for maximum semantic novelty (Ziad et al., 12 Mar 2026).
• Dual-Level Language Mutators (DarthShader): In shader compilation pipelines, complementary mutation engines operate on the Abstract Syntax Tree (AST) and statically-typed Intermediate Representation (IR). AST mutations stress parser and front-end code paths, while IR mutations expose faults in deep optimizer passes (e.g., constant folding, vector flattening), leveraging staged representations for maximal end-to-end fault discovery (Bernhard et al., 2024).

The use of staged, domain-specific mutators and/or constraint modeling, as opposed to naïve randomization, has been empirically shown to increase both coverage and unique bug discovery.

3. Instrumentation, Sanitization, and Coverage Feedback

Effective GPU-native fuzzing depends on robust device-side monitoring and cross-host/device feedback loops:

• Binary Instrumentation: Dynamic rewriting via NVBit (SASS/PTX level) injects coverage counters on control-flow transitions and inserts memory safety checks (bounds, allocation tags) for every device memory instruction (LDG, STS, etc.) (Ziad et al., 12 Mar 2026, Li et al., 5 Mar 2026).
• Sanitizer Decoupling: Given incompatibilities between instrumentation frameworks and sanitizers, sanitization (ASan for host, Compute Sanitizer’s memcheck/initcheck/racecheck on device) is typically decoupled into separate processes or runs. Only coverage-increasing or error-triggering inputs are promoted for full sanitization (Ziad et al., 12 Mar 2026).
• Coverage Feedback: Edge or basic block coverage is computed and merged across host and device. For instance, cuFuzz uses a 64 KB bitmap, with the lower half reserved for host edges and the upper for device edges. Any new edge observed by either is considered “new coverage” (Ziad et al., 12 Mar 2026).
• Persistent-Mode Support: To avoid costly CUDA runtime initializations, persistent harnessing maintains device state across iterations, with synchronization points and device coverage resets triggered through notification kernels, leading to significant throughput gains (Ziad et al., 12 Mar 2026).
• Memory Hierarchy Awareness: Explicit checks are implemented for global, shared, and local memory regions. Out-of-bounds (OOB), misaligned accesses, uninitialized reads, and shared memory races are all monitored using hardware-level semantics—not via CPU simulation—to ensure detection of real-world error conditions (Li et al., 11 Feb 2026, Li et al., 5 Mar 2026).

4. Evaluation, Metrics, and Bug Categories

Empirical evaluation across frameworks, libraries, and shader compilers demonstrates significant improvements in both test-case throughput and unique bug discovery relative to CPU-native or naïve approaches.

Table: Representative Metrics from Recent GPU-Native Fuzzers

Fuzzer	Test-case Throughput	Unique Bugs Found	Edge Coverage (24h)
GPU-Fuzz	12,000/hr (H100 GPU)	26 ± 5 memory-safety	~85% specialized checks
cuFuzz	36 K (persistent)	43 (19 closed-source)	34 K (full)
DarthShader	–	39 (39 browsers)	+11–24% over SOTAs

• Bug Types: Heap and stack buffer overflows, OOB global/shared memory access, use-after-free, uninitialized reads, data races, kernel launch failures, and CPU-side assertion errors are among the faults catalogued. For instance, GPU-Fuzz surfaced integer overflows in convolution operator gridDims and silent memory corruption in major DL frameworks (Li et al., 11 Feb 2026).
• Coverage Delta: High-precision modeling and device-level feedback contribute to a marked lift in boundary check (GPU-Fuzz: 85%) and unique device-side edge coverage (cuFuzz: up to 289% improvement for closed source).
• Throughput and Overhead: Persistent harnesses and batched test dispatch significantly increase executions per second, although instrumentation with NVBit and full Compute Sanitizer checks remains a bottleneck (e.g., ~3× execution slowdown) (Ziad et al., 12 Mar 2026, Li et al., 5 Mar 2026).

5. Faithfulness and Domain-Specific Considerations

Assuring that detected errors are faithful to genuine GPU semantics is critical:

• Execution Faithfulness: All major designs conduct sanitization and coverage on live GPU hardware, capturing effects of true scheduling, coalescing, and device-specific variance—circumventing pitfalls of CPU translation or emulation, which often miss concurrency bugs or memory hazards unique to the GPU’s execution model (Li et al., 5 Mar 2026).
• Domain-Specific Knowledge: DL operator fuzzing leverages explicit parameter and kernel shape modeling, while shader pipeline fuzzers (e.g., DarthShader) exploit the multistage nature of compiler passes. Mutation policies are orchestrated to stress both surface and deep stages relevant to production vulnerabilities (Bernhard et al., 2024).
• Realistic Attack Models: End-to-end pipeline fuzzing is mandated for security-relevant contexts (e.g., remote shader compilation in browsers) to avoid false negatives from API-specific testing (Bernhard et al., 2024).

A plausible implication is that GPU-native approaches constitute the only viable methodology for discovering classes of bugs inherently tied to device architectures, particularly those involving concurrency, precise memory layouts, or inter-kernel state.

6. Limitations, Open Problems, and Future Directions

While recent GPU-native pipelines have exposed major blind spots in DL and GPU stack correctness, notable challenges remain:

• Incomplete Race Detection: Full device-side race and bank conflict detection is either in early prototype or not yet implemented. Efficient, scalable mechanisms for per-warp or per-address timestamping remain an open research area (Li et al., 5 Mar 2026).
• Harness Automation: Current methods often require bespoke harnesses or semi-automated slicing of application samples. Addressing this requires routine integration of code analysis, possibly with LLM-assisted slicing or programmatic harness generation (Li et al., 5 Mar 2026).
• Multi-GPU and Heterogeneous Contexts: Fuzzing remains primarily single-device; peer-to-peer, unified memory, and NVLink support are proposed as future extensions.
• Performance Counter and State Feedback: Hardware sampling integration for performance-guided fuzzing and adaptive mutation remains unexplored at scale.
• Corpus Management: Seed queue management strategies are basic; deeper heuristics may be needed to target complex GPU state changes or long-tail coverage (Li et al., 5 Mar 2026).

This suggests that production-scale GPU fuzzing will increasingly rely on joint advances in device-level instrumentation, automated test harness generation, state-aware mutation strategies, and the continued evolution of GPU sanitizer runtime support.

7. Impact, Empirical Outcomes, and Lessons Learned

The deployment of GPU-native fuzzing pipelines has resulted in substantive improvements in bug discovery in both open-source and proprietary codebases. Examples include:

• Memory-Safety and Security: cuFuzz discovered 43 unique bugs in a 24-hour campaign, with 19 in commercial libraries and up to 13 invalid global memory accesses captured in production code (Ziad et al., 12 Mar 2026). DarthShader reported 39 bugs in all major browsers, 15 of which received CVEs for vulnerabilities in real-world shader translation and optimization (Bernhard et al., 2024).
• Code Coverage and Test-Case Diversity: Coverage-guided, dual-stage mutation infrastructures (e.g., DarthShader) achieve significant, quantifiable coverage lifts (+11–24% branches), with empirical studies confirming the necessity of staged, domain-specific input mutation strategies.
• Lessons for the Community: AST and IR-level mutations each stress distinct (and often non-overlapping) validation/code paths; neither suffices alone for end-to-end pipeline robustness (Bernhard et al., 2024). Fuzzing infrastructure must operate at the same abstraction level as the software under test for maximal efficacy.

Overall, GPU-native fuzzing pipelines now represent a key methodology for uncovering silent, high-severity bugs in the critical software stacks underlying modern AI, scientific workloads, and client-facing GPU deployment environments (Li et al., 11 Feb 2026, Ziad et al., 12 Mar 2026, Li et al., 5 Mar 2026, Bernhard et al., 2024).