Graph-Aware Fuzzing for Graph Database Management Systems

Abstract: Graph Database Management Systems (GDBMSs) are essential infrastructure for managing interconnected data. Existing GDBMS testing methods primarily rely on differential and metamorphic testing. The result consistency oracles of these methods constrain inputs to queries that are comparable across engines or transformations, leaving single engine runtime failures, such as crashes and memory errors, insufficiently explored. Developing dedicated fuzzers for GDBMSs faces two key challenges: (1) generating valid and structurally diverse queries under complex graph constraints, and (2) guiding exploration to capture topology dependent execution behavior. To address these challenges, we propose GRAF, a black box fuzzing framework for GDBMS query engines. First, GRAF introduces graph context aware query generation based on cascading dependency resolution. It instantiates parameterized Cypher skeletons generated by a LLM by jointly resolving labels, relationship types, properties, values, and variable scopes against the active graph state. This process produces structurally diverse queries while eliminating syntactic and semantic violations. Second, GRAF applies five graph specific mutation operators guided by execution state feedback, including execution time, result size, and system status. This feedback steers exploration away from unproductive queries and expensive traversals, while prioritizing local mutations around abnormal executions. We evaluated GRAF against three existing approaches on six widely used GDBMSs. GRAF consistently improves line coverage by 31.6% to 41.1% over the strongest baseline on each target. In 12 hour fuzzing, it triggered 25 unique bugs, compared to six from all baselines combined. Overall, GRAF discovered 34 previously unknown bugs, with 32 confirmed by developers and 23 assigned CVEs.

• The paper presents GRAF, a black-box fuzzing framework that decouples abstract query structures from concrete graph data to systematically trigger runtime failures.
• It employs LLM-generated query skeletons and cascading dependency resolution to maintain 61.8%-76.7% query validity and achieve a 31.6%-41.1% increase in code coverage.
• Empirical results demonstrate that GRAF discovers 25 unique bugs—a >4× improvement over existing fuzzers—validating its impact on enhancing GDBMS reliability.

Graph-Aware Fuzzing for GDBMSs: Overview and Analysis

Introduction

This paper introduces GRAF, a black-box, context-aware fuzzing framework designed for Graph Database Management Systems (GDBMSs). Existing GDBMS fuzzing approaches are limited by their reliance on result consistency oracles, which restrict input diversity and fail to reach engine-specific runtime failures such as crashes and OOM errors. GRAF overcomes these challenges through a novel combination of graph-context-aware query generation and dynamic, feedback-guided mutation. Core to GRAF's methodology is the decoupling of abstract query structure from concrete graph data, leveraging LLM-based skeleton generation and runtime-dependent instantiation to maintain validity and maximize diversity.

Figure 1: Core elements of a labeled property graph: labeled nodes, directed relationships, and typed properties.

System Architecture and Methodology

GRAF operates in two principal phases: (1) graph-context-aware query generation and (2) execution-state-guided mutation, as depicted in its system architecture.

Figure 3: Architecture of GRAF. The left component generates valid and diverse query seeds by combining skeleton filtering with graph context instantiation. The right component performs feedback guided mutation to explore deep GDBMS execution behavior.

Graph Context Modeling and Cascading Dependency Resolution

GRAF builds an explicit context by extracting topology, schema, and value metadata from the active graph. Query generation is initiated using a set of parameterized skeletons produced via an LLM, where placeholders denote abstract constructs such as labels, relationships, properties, and values. At instantiation, these skeletons are transformed to executable Cypher queries through cascading resolution: the binding of placeholders is performed in a dependency-aware sequence, ensuring every step is topologically and semantically valid.

Figure 5: Cascading dependency resolution. (a) A parameterized skeleton where arrows indicate binding dependencies. (b) The graph initialized in the target. (c) Metadata extracted from the graph to constrain instantiation.

This separation between abstract query form and concrete runtime binding enables effective exploration of diverse, deeply executable queries while minimizing syntactic or semantic rejection.

Execution State Feedback and Mutation Operators

Unlike code-coverage-guided fuzzing, GRAF uses a runtime-emergent feedback vector $\langle t, |R|, s\rangle$—capturing execution time, result cardinality, and status—to dynamically adapt mutation strategies. Five graph-specific mutation operators (topology mutation, depth/volume mutation, predicate mutation, clause insertion, and value mutation) are selected and parameterized based on this feedback. This mechanism suppresses mutations producing nonproductive or expensive queries, while prioritizing modifications around abnormal executions to maximize backend code coverage and defect exposure.

Empirical Evaluation

The evaluation comprises a suite of experiments on six prominent GDBMSs (Neo4j, Memgraph, RedisGraph, NebulaGraph, FalkorDB, and KuzuDB), comparing GRAF against structurally diverse (Dinkel), mutation-based (BUZZBEE), and generic (AFL++) fuzzers.

Code Coverage Results

GRAF achieves a 31.6%–41.1% increase in code coverage relative to the strongest baseline across all targets, with a sustained growth profile even during extended fuzzing periods. While byte-level and grammar-based fuzzers plateau due to syntactic invalidity and limited structural diversity, GRAF's skeleton plus instantiation strategy ensures ongoing discovery of deep behavioral paths.

Figure 2: Coverage growth over 12 hours. The curves report the median number of covered code lines, and shaded regions denote 95% confidence intervals.

Figure 8: Line coverage overlap at 12 hours. GRAF covers the baseline intersection and achieves the highest unique coverage.

Query Validity and Component Ablation

GRAF maintains a valid query rate of 61.8%–76.7%. Ablation studies reveal that the instantiation mechanism is indispensable: removing it reduces validity to 10.7% and coverage by over 80%. Execution feedback not only reduces slow queries (by up to 52.8%) but boosts effective exploration, as demonstrated by a sharp drop in timeouts with feedback enabled.

Figure 4: Slow executions with and without execution state feedback.

Bug Discovery

During systematic 12-hour fuzzing runs, GRAF triggers 25 unique bugs—a >4× improvement over all baselines combined—with 32 of 34 total discovered bugs confirmed by developers and 23 assigned CVEs. The fault spectrum includes segmentation faults, use-after-free, stack overflows, assertion failures, resource exhaustion, and OOM events, confirming GRAF's deep backend coverage.

Implications and Future Directions

The GRAF framework demonstrates that decoupling query abstract structure from graph-dependent instantiation combined with orthogonal feedback-guided mutation is critical for systematically exercising GDBMS runtime and uncovering severe implementation bugs. The use of LLMs for skeleton generation enables scalable coverage of complex, realistic query forms across dialects, while cascading instantiation adapts to evolving graph schemas without manual engineering.

Practically, this methodology provides a blueprint for future fuzzers in irregular, context-dependent domains—such as property graphs, knowledge graphs, or other semistructured data engines—where frontend constraints and backend complexity align. The strong empirical results and confirmed impact (via CVEs) highlight the efficacy of external-execution feedback in steering test generation toward meaningful failures, rather than exhaustively relying on coverage metrics that may not reflect semantic depth.

Theoretically, GRAF suggests new design patterns for “oracle-free” or semi-oracle fuzzing in database and data-management systems, motivating further research on semantic bug detection beyond runtime failures—e.g., leveraging learned or derived invariants for logical correctness, or integrating symbolic analysis with context-guided mutation.

Conclusion

GRAF marks a significant methodological advancement in the fuzzing of GDBMSs by unifying LLM-powered, context-aware query generation with execution state feedback-driven mutation. The empirical evidence supports that abstraction-aware instantiation and external feedback are effective levers for maximizing both coverage and defect exposure in complex data management software. Future research will benefit from extending GRAF’s context-guided paradigm to additional query languages, heterogeneous graph domains, and deep semantic bug detection approaches.

---

Reference: "Graph-Aware Fuzzing for Graph Database Management Systems" (2607.03741)