GraphIP-Bench: How Hard Is It to Steal a Graph Neural Network, and Can We Stop It?

Abstract: Graph neural networks (GNNs) deployed as cloud services can be \emph{stolen} through \emph{model-extraction attacks}, which train a surrogate from query responses to reproduce the target's behaviour, and a growing line of ownership defenses tries to prevent or trace such theft. The title of this paper asks two questions: \emph{how hard is it to steal a GNN?}, and \emph{can we stop it?} Prior work cannot answer either, because experiments use inconsistent datasets, threat models, and metrics. We introduce \emph{GraphIP-Bench}, a unified benchmark which evaluates both sides under a single black-box protocol. It integrates twelve extraction attacks, twelve defenses spanning watermarking, output-perturbation, and query-pattern-detection families, ten public graphs covering homophilic, heterophilic, and large-scale regimes, three GNN backbones, and three graph-learning tasks, and it reports fidelity, task utility, ownership verification, and computational cost on shared splits, queries, and budgets. We further add a joint attack-and-defense track which runs every attack on every defended target and measures watermark verification on the resulting surrogate, which exposes the protection that a defense retains after extraction. The empirical picture is short: stealing a GNN is easy at medium query budgets and most defenses do not change this; several watermarks verify reliably on the protected model but lose most of their verification signal on the extracted surrogate, which exposes a gap that single-model evaluations miss; and heterophilic graphs are systematically harder to steal, while a cross-architecture mismatch between target and surrogate reduces but does not prevent extraction. Code: \href{https://github.com/LabRAI/GraphIP-Bench}{LabRAI/GraphIP-Bench}.

• The paper reveals that GNN extraction can achieve over 90% fidelity on homophilic graphs with moderate query budgets.
• It introduces a unified benchmark integrating 12 attack and 12 defense methods across diverse graph datasets and architectures.
• The study highlights that robust ownership verification requires defenses to preserve watermark signals even in extracted surrogate models.

How Hard Is It to Steal a Graph Neural Network, and Can We Stop It? An Analysis of GraphIP-Bench

Introduction and Motivation

Graph neural networks (GNNs) are widely deployed in commercial and industrial machine learning platforms for tasks such as item recommendation, fraud detection, molecular property prediction, and social network analysis. With the rise of cloud-based GNN services, the threat of model extraction—where an adversary learns a high-fidelity surrogate by querying the black-box model—has become a central intellectual property and security concern. While both extraction attacks and a burgeoning field of defenses (watermarking, query modification, fingerprinting) have seen significant research activity, the field has lacked rigorous, unified, empirical evaluation frameworks.

"GraphIP-Bench: How Hard Is It to Steal a Graph Neural Network, and Can We Stop It?" (2605.12827) addresses these gaps with a reproducible, unified benchmark that integrates twelve attacks, twelve defense methods, ten diverse graph datasets, three GNN backbones, and three core graph tasks under a standardized black-box protocol. This systematizes both the attack and defense evaluation, permits direct comparison, and exposes properties that prior, siloed empirical studies failed to capture.

Unified Benchmarking Protocol

GraphIP-Bench is designed around the principle of comparability: public splits, shared query sets, fixed query budgets, uniform endpoint semantics (label/confidence return), and explicit threat models. The protocol separates the evaluation into three tracks: (i) extraction (attacks on undefended models), (ii) ownership (defenses on clean models), and (iii) joint attack-defense evaluation (attack surrogates trained against defended models, testing watermark survival and verification).

GNN architectures are standardized to GCN, GAT, and GraphSAGE backbones (all with matched hidden dimensions), and experiments span a wide variety of graph-structural regimes including homophilic, heterophilic, and large-scale settings. Evaluation metrics include task utility (accuracy, macro F1), extraction fidelity (agreement between target and surrogate predictions), ownership verification (watermark/fingerprint detectability), and computational cost (wall-clock time and peak memory).

Model Extraction Attack Effectiveness

Across twelve attack methods—including six MEA-style baselines, adaptive query strategies (AdvMEA, CEGA), structure-reconstruction pipelines (Realistic), and three data-free (DFEA) variants—extraction success is principally driven by query budget and graph structural properties.

Figure 1: Budget-metric curves show that most attacks saturate in fidelity/accuracy at moderate budgets, especially for homophilic graphs.

Budget saturation is a universal phenomenon; on most datasets, especially homophilic graphs such as Cora and CoauthorPhysics, strong attacks achieve >90% fidelity with budgets of 0.25×–0.5× the test size. Removal of either features or structure only marginally decreases extraction performance for strong attacks, while only in the data-free regime do most baseline attacks collapse. However, the DFEA methods themselves remain competitive—data-free attacks nearly match data-driven ones on most datasets outside specific high-degree product graphs.

Notably, extraction is systematically harder on heterophilic graphs (e.g., RomanEmpire, AmazonRatings), evident from lower attainable fidelities and larger variance across attack types. Adaptive query attacks (e.g., AdvMEA), which rely on homophily or smooth decision boundaries, often underperform in these settings.

Defense Mechanisms: Performance and Trade-offs

Figure 2: Utility drop across datasets, showing that most defenses achieve only modest reductions in task accuracy except for aggressive misinformation defenses.

Defenses are categorized into information-limiting (output perturbation, rounding, query detection) and ownership-tracing (parameter/backdoor watermarking and query-based integrity). Among ownership defenses, backdoor watermarking consistently achieves perfect verification (100%) with minimal utility drop (~3pp loss), outperforming parameter-side watermarks (RandomWM, SurviveWM, ImperceptibleWM) that trade off between utility, stability, and verifiability. Query-based integrity schemes (e.g., Integrity) also yield high F1 and efficiency, but their binary verification signal can be dataset-dependent.

Information-limiting schemes display classic trade-offs: output perturbation and prediction rounding (especially top-1) are effective but can degrade accuracy on class-rich or heterophilic graphs; adaptive misinformation (AdaptMisinfo) sharply reduces fidelity but consistently loses 30–45pp of accuracy, precluding practical adoption. Query pattern detectors (PRADA, GradRedir) yield protection but are more sensitive to benign-versus-malicious query separation in graph data.

Figure 3: Radar plot capturing defense trade-offs across fidelity, verification, utility, and computational resources.

Efficiency and Practicality

Figure 4: Wall-clock costs display that most attacks and defenses are computationally lightweight except complex structure-reconstruction or representation-optimizing defenses such as Realistic and ImperceptibleWM.

Attack and defense efficiency is tracked across time and memory budgets. Most attacks finish within minutes and occupy sub-GB memory; only high-complexity approaches (structure recovery, deep backdoor/representation watermarking) exhibit significant resource demands, and in those cases, their marginal benefit over simple attacks diminishes at moderate query budgets.

Joint Attack-Defense Evaluation and Watermark Survival

A central, previously unexplored finding emerges when every attack is applied to every defended model: surrogate-fidelity remains high even when models are defended, and only certain defense types retain strong watermark signals in extracted surrogates.

Most parameter-side or training-time watermarks, despite verifying perfectly on the protected model, empirically lose almost all verification signal after extraction. In contrast, query-time integrity/fingerprinting mechanisms (e.g., Integrity) are robust: watermark verification rate remains 100% on both the protected model and the surrogate. This decisively demonstrates that verification on the protected model is insufficient for ownership tracing; defense evaluation must focus on the extracted surrogate.

Broader Implications and Research Directions

The systemic empirical insights of GraphIP-Bench challenge several prior assumptions:

• Model extraction is not a prohibitively expensive or limited process: High-fidelity surrogates are routinely learned at moderate query budgets, even by data-free methods for most graph structures.
• Parameter-side watermarks cannot be considered robust ownership verification in the black-box extraction regime: Future designs must prioritize watermark survival in surrogate models, motivating new schemes rooted in output-based or query-time mechanisms.
• Defenses trading accuracy for reduced extraction are currently not viable for most sensitive applications: Practical deployments require bounded utility loss—output perturbation and label-quantization defenses dominate this regime.

The benchmark also exposes open problems:

• Watermark/fingerprint mechanisms resilient to surrogate training are an immediate need.
• Query-pattern anomaly detection specifically optimized for graph query interfaces could improve the protection-utility frontier.
• A principled analysis connecting graph structure (homophily, degree, class count) and extraction/defense success would enable optimal method selection under deployment constraints.

Conclusion

GraphIP-Bench (2605.12827) establishes a unified, reproducible platform for empirical evaluation of GNN extraction attacks and defenses, revealing that GNNs are in most regimes easy to steal, and that many existing defenses, especially training-time, parameter-based watermarks, fail to provide robust post-extraction ownership verification. This advocates for the design of ownership-tracing techniques that anchor in output mechanisms and underlines the necessity for direct surrogate-side evaluation. The released benchmark stands as a reference point for objective, apples-to-apples comparison of future graph IP protection techniques and will shape research priorities in GNN security and trustworthiness.