ProvX: Counterfactual Explanations for Intrusion Detection

• The paper introduces ProvX, a framework that identifies the minimal necessary edge removals causing a GNN-based detector to flip its prediction.
• It employs differentiable mask optimization with a staged solidification strategy to ensure stable and precise counterfactual explanations on provenance graphs derived from host audit logs.
• Empirical evaluations on DARPA datasets demonstrate that ProvX significantly enhances explanation necessity and trust in provenance intrusion detection systems.

Searching arXiv for the cited papers and closely related work on provenance-graph explanation and adjacent provenance systems. ProvX is a counterfactual explanation framework for GNN-based provenance intrusion detection that aims to close the trust and usability gap between high-performing provenance graph detectors and the evidentiary requirements of security analysts. It is designed for provenance graphs reconstructed from host audit logs, where nodes denote system entities and directed edges encode causal relations such as process creation, file access, network communication, and memory interaction. Rather than highlighting merely contributory graph regions, ProvX seeks the minimal structural subset of a malicious provenance subgraph whose removal would flip the detector’s prediction, thereby producing explanations framed as necessary causes for the model’s judgment (Wu et al., 8 Aug 2025).

1. Definition, motivation, and analytical stance

ProvX was introduced to explain GNN-based security models on provenance graphs, a setting in which learning-based provenance-based intrusion detection systems can achieve strong detection performance but often provide no verifiable explanation for why a subgraph was judged malicious (Wu et al., 8 Aug 2025). The motivating problem is operational: analysts require explanations that map to real attack steps and that are necessary for the model’s decision, not merely correlated with it.

The framework adopts counterfactual explanation logic. For a provenance subgraph predicted as malicious, it asks which minimal set of edges, if absent, would cause the detector to change its output from malicious to benign. This differs from fact-based explanation methods such as GNNExplainer, PGExplainer, and SubgraphX, which identify contributory substructures but do not guarantee causal necessity. In this sense, ProvX is explicitly oriented toward root-cause analysis of model decisions rather than saliency alone (Wu et al., 8 Aug 2025).

The paper positions this requirement as especially important in provenance security. Provenance graphs encode the causal fabric of host execution from OS audit logs, and GNN-based detectors operate over subgraphs extracted from these logs. An alert without a verifiable explanation creates what the paper describes as a trust and actionability deficit. ProvX addresses that deficit by tying explanations to minimal graph perturbations that invalidate the detector’s original belief (Wu et al., 8 Aug 2025).

A plausible implication is that ProvX is best understood as an explanation layer over a frozen detector rather than as a standalone detector. The explanation is meaningful only with respect to the trained model and the subgraph partitioning used to present provenance evidence to that model.

2. Provenance graph setting and detector model

ProvX operates on provenance graphs $G = (V, E)$ derived from host audit logs. In this representation, nodes correspond to system entities such as processes, files, sockets or netflows, and memory objects, while directed edges encode heterogeneous causal relations. The paper groups these relations into four main interaction families: Process $\rightarrow$ Process, Process $\rightarrow$ File, Process $\rightarrow$ Netflow/Socket, and Process $\rightarrow$ Memory (Wu et al., 8 Aug 2025).

For downstream detection, the host-level provenance graph is partitioned into manageable subgraphs using an improved Louvain-based partitioning method with one-hop neighborhood expansion for disconnected partitions. Each subgraph $G_k$ is assigned a ground-truth label $Y_k \in \{0,1\}$, where $0$ denotes a benign subgraph and $1$ denotes a subgraph containing at least one stage of an APT attack. The detector then performs subgraph-level binary classification, with prediction defined as

$$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$

The detector architectures considered are standard two-layer GNNs with ReLU and Dropout, average pooling, and a two-layer MLP classifier, optimized with Adam for 50 epochs at learning rate $\rightarrow$0 (Wu et al., 8 Aug 2025).

The evaluation uses DARPA OpTC and DARPA TC E3 datasets. On OpTC, the reported detector performance includes GAT with AUC $\rightarrow$1 and FPR $\rightarrow$2, GraphSAGE with AUC $\rightarrow$3 and FPR $\rightarrow$4, and a weaker GCN with AUC $\rightarrow$5. On E3, the paper reports high detection quality across datasets, with FPRs near $\rightarrow$6 on Cadets and Trace (Wu et al., 8 Aug 2025).

This detector context is central to understanding ProvX. The framework does not explain arbitrary provenance reasoning; it explains the decisions of frozen GNN detectors trained on subgraph partitions. Accordingly, both the detector family and the graph partitioning granularity shape the resulting explanations.

3. Counterfactual formulation and differentiable mask optimization

The core ProvX problem is posed as a constrained minimization over perturbed subgraphs. Given a malicious subgraph $\rightarrow$7 with prediction $\rightarrow$8, the objective is to find a minimally perturbed subgraph $\rightarrow$9 such that the detector’s prediction flips:

$\rightarrow$0

This formulation encodes two simultaneous goals: prediction flipping and minimal structural deviation from the original graph (Wu et al., 8 Aug 2025).

Because direct search over discrete graph edits is difficult, ProvX relaxes the perturbation problem into continuous optimization over a differentiable edge mask. If $\rightarrow$1 is the adjacency matrix of the original subgraph, the perturbed adjacency is written first as

$\rightarrow$2

and then, under continuous relaxation,

$\rightarrow$3

where $\rightarrow$4 is the learnable continuous mask and $\rightarrow$5 is the sigmoid function (Wu et al., 8 Aug 2025).

The detector is held fixed during explanation. Predictions on the original and perturbed graphs are given by

$\rightarrow$6

Only the mask parameters are optimized (Wu et al., 8 Aug 2025).

ProvX combines a prediction-flip term and a distance term in a dual-objective loss. The paper defines

$\rightarrow$7

$\rightarrow$8

and

$\rightarrow$9

The first component reduces the detector’s confidence in the original malicious class under perturbation, while the second penalizes structural deviation from the original adjacency (Wu et al., 8 Aug 2025).

The final explanation is extracted by ranking edges according to their contribution to prediction flipping, operationalized in the paper as $\rightarrow$0 mask value, and selecting the top-$\rightarrow$1 removed edges. The explanation subgraph $\rightarrow$2 is then interpreted as follows: if these $\rightarrow$3 edges were removed from the original graph, the provenance graph would no longer be predicted as an APT attack (Wu et al., 8 Aug 2025).

A plausible implication is that ProvX explains malicious predictions in terms of edge-level necessity rather than node-level or temporal necessity. The paper explicitly notes that node-level or temporal explainability is future work.

4. Staged Solidification

A distinctive component of ProvX is the Staged Solidification strategy, introduced to improve the precision and stability of explanations. The stated purpose is to lock in clear trends among confident edges and penalize later deviations, thereby avoiding oscillation and exaggerated perturbation during mask optimization (Wu et al., 8 Aug 2025).

Solidification begins at epoch $\rightarrow$4. At that point, the current mask snapshot is recorded as

$\rightarrow$5

and two index sets are defined:

$\rightarrow$6

$\rightarrow$7

These correspond to edges already trending strongly toward removal or retention (Wu et al., 8 Aug 2025).

For later epochs, deviations from the snapshot on these confident edges are penalized through

$\rightarrow$8

The full objective becomes

$\rightarrow$9

The explainer is trained with Adam at learning rate $\rightarrow$0 for 200 epochs; default parameters are $\rightarrow$1, $\rightarrow$2, $\rightarrow$3, and $\rightarrow$4 (Wu et al., 8 Aug 2025).

The reported ablations indicate that solidification matters materially. The paper states that $\rightarrow$5 around $\rightarrow$6 of epochs works best, that without solidification $\rightarrow$7 the Probability of Necessity drops notably, and that the best $\rightarrow$8 lies around $\rightarrow$9–$G_k$0 depending on the detector (Wu et al., 8 Aug 2025). This makes Staged Solidification more than an optimization convenience; it is part of the explanation mechanism’s claim to stability.

5. Evaluation methodology and empirical results

ProvX is evaluated against gradient-based, decomposition-based, and perturbation-based baselines. The comparison set includes GradCam, DeepLIFT, GNN-LRP, GNNExplainer, PGExplainer, and SubgraphX (Wu et al., 8 Aug 2025). The experiments use DARPA OpTC and DARPA TC E3 datasets, with OpTC partitioned into 8339 benign and 1534 malicious subgraphs and a 7:1:2 train/validation/test split; random negative sampling is used to balance training splits (Wu et al., 8 Aug 2025).

Two explanation tasks are emphasized. The first is explanation hit degree, assessed by Accuracy, Precision, Recall, and F1 over malicious subgraphs by comparing the top-$G_k$1 explanation edges against ground-truth attack labels from prior works, with baseline $G_k$2. The second is explanation necessity, measured by the Probability of Necessity:

$G_k$3

Here $G_k$4 is the detector’s prediction after removing the explanation subgraph from the original graph (Wu et al., 8 Aug 2025).

The main quantitative findings are that ProvX achieves average explanation Accuracy of approximately $G_k$5 across datasets and an average PN of $G_k$6, with these necessity results outperforming current state-of-the-art explainers (Wu et al., 8 Aug 2025). On OpTC with $G_k$7, reported hit-degree values include, for GAT, Accuracy/Precision/Recall/F1 of $G_k$8, and for GraphSAGE, $G_k$9 (Wu et al., 8 Aug 2025).

For necessity, the paper reports PN values at $Y_k \in \{0,1\}$0 of $Y_k \in \{0,1\}$1 for GCN, $Y_k \in \{0,1\}$2 for GAT, and $Y_k \in \{0,1\}$3 for GraphSAGE on OpTC. On DARPA TC E3, the reported PN ranges are $Y_k \in \{0,1\}$4–$Y_k \in \{0,1\}$5 for Cadets, $Y_k \in \{0,1\}$6–$Y_k \in \{0,1\}$7 for Theia, and $Y_k \in \{0,1\}$8–$Y_k \in \{0,1\}$9 for Trace, depending on detector (Wu et al., 8 Aug 2025). The average necessity improvement over SOTA is reported as $0$0, $0$1, and $0$2 for GCN, GAT, and GraphSAGE detectors respectively (Wu et al., 8 Aug 2025).

The paper also reports characteristic trade-offs in hyperparameter tuning. As $0$3 increases, PN and Recall increase monotonically, while Precision tends to decrease. GraphSAGE reaches PN of approximately $0$4 at $0$5. The optimal $0$6 differs by detector—$0$7 for GCN, $0$8 for GAT, and $0$9 for GraphSAGE—and too large an $1$0 degrades both PN and F1 (Wu et al., 8 Aug 2025).

These results support the paper’s central claim that counterfactual, necessity-oriented explanations behave differently from fact-based explanation methods. High Recall and PN indicate that the selected edges are often genuinely decisive for the model, even when subgraph-level Precision is constrained by provenance hub structures and partition granularity.

6. Case studies, closed-loop feedback, and relation to adjacent systems

The paper includes case studies that tie ProvX explanations to concrete attack-relevant structures. In DARPA TC Trace, ProvX identifies the edges pass_mgr $1$1 sh and pass_mgr $1$2 gtcache as decisive, with their removal flipping the model’s prediction to benign. In DARPA E3 Theia, deleting interactions among payloads clean, profile, mail, and wdev flips the prediction, whereas other deletions such as gtcache $1$3 /proc/* do not, thereby exposing which structures the detector actually uses as root causes (Wu et al., 8 Aug 2025).

Beyond post hoc explanation, ProvX is also embedded in a closed-loop Detection–Explanation–Feedback enhancement framework. When an adversarial subgraph $1$4 is misclassified as benign, ProvX finds a minimal perturbation $1$5 that flips the prediction back to malicious. After analyst verification, both $1$6 and the retained subgraph $1$7, with corrected labels, are added to training for fine-tuning. The reported gains after feedback are Accuracy $1$8, Precision $1$9, Recall $$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$0, and F1 $$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$1 (Wu et al., 8 Aug 2025). This does not alter the ProvX loss itself; it uses explanations to curate hard training examples.

The framework’s limitations are explicit. It assumes that audit logging operates atop a Trusted Computing Base and yields complete, reliable provenance, and that ground-truth annotations are trusted. It explains frozen GNN detectors in a static subgraph-classification setting, so heterogeneous or temporal GNNs require adaptation. It is also computationally nontrivial: per-subgraph explainer training takes $$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$2–$$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$3s on CPU, with evaluation taking $$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$4–$$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$5s and peak memory around $$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$6–$$\hat{Y}_k = \operatorname*{arg\,max}_{c \in \{0,1\}} P(c \mid G_k).$$7 GB (Wu et al., 8 Aug 2025).

In the surrounding literature supplied here, the term “ProvX” is not used consistently. One briefing uses “ProvX” as implementation guidance inspired by PROVSEEK, an LLM-driven provenance forensics framework that fuses CTI retrieval, provenance databases, specialized agents, and verification-first summaries for threat investigation and detection (Mukherjee et al., 29 Aug 2025). Another paper states that “ProvX” is not used in the paper and that, in context, it likely refers to PRO-V, a multi-agent system for automatic RTL verification (Zhao et al., 13 Jun 2025). In the strict sense of the paper titled “ProvX: Generating Counterfactual-Driven Attack Explanations for Provenance-Based Detection,” however, ProvX denotes the counterfactual explanation framework for GNN-based provenance intrusion detection (Wu et al., 8 Aug 2025).

This distinction is significant. PROVSEEK concerns agentic forensic investigation over system provenance, and PRO-V concerns LLM-assisted RTL verification; ProvX, by contrast, is specifically an explainer for provenance-graph security models. A plausible implication is that these systems occupy complementary layers of a broader provenance-centered tooling landscape: detection and explanation in ProvX, investigation and verification in PROVSEEK, and unrelated hardware verification in PRO-V.