XInsight: Revealing Model Insights for GNNs with Flow-based Explanations

Abstract: Progress in graph neural networks has grown rapidly in recent years, with many new developments in drug discovery, medical diagnosis, and recommender systems. While this progress is significant, many networks are black boxes' with little understanding of thewhat' exactly the network is learning. Many high-stakes applications, such as drug discovery, require human-intelligible explanations from the models so that users can recognize errors and discover new knowledge. Therefore, the development of explainable AI algorithms is essential for us to reap the benefits of AI. We propose an explainability algorithm for GNNs called eXplainable Insight (XInsight) that generates a distribution of model explanations using GFlowNets. Since GFlowNets generate objects with probabilities proportional to a reward, XInsight can generate a diverse set of explanations, compared to previous methods that only learn the maximum reward sample. We demonstrate XInsight by generating explanations for GNNs trained on two graph classification tasks: classifying mutagenic compounds with the MUTAG dataset and classifying acyclic graphs with a synthetic dataset that we have open-sourced. We show the utility of XInsight's explanations by analyzing the generated compounds using QSAR modeling, and we find that XInsight generates compounds that cluster by lipophilicity, a known correlate of mutagenicity. Our results show that XInsight generates a distribution of explanations that uncovers the underlying relationships demonstrated by the model. They also highlight the importance of generating a diverse set of explanations, as it enables us to discover hidden relationships in the model and provides valuable guidance for further analysis.

• The paper introduces XInsight, which leverages GFlowNets to generate a diverse distribution of explanations for GNN predictions, enhancing model interpretability.
• It demonstrates the framework’s capability by generating class-specific graphs validated through experiments on synthetic acyclic graphs and the MUTAG dataset.
• The approach uncovers critical relationships, such as the link between lipophilicity and mutagenicity, confirmed via statistical validation and QSAR modeling.

XInsight: Revealing Model Insights for GNNs with Flow-based Explanations

Introduction

The paper "XInsight: Revealing Model Insights for GNNs with Flow-based Explanations" (2306.04791) presents a novel approach, XInsight, leveraging Generative Flow Networks (GFlowNets) to generate interpretable and diverse explanations for Graph Neural Networks (GNNs). It addresses the need for explainable AI in high-stakes applications such as drug discovery, where understanding model predictions is crucial for validation and new knowledge discovery.

GFlowNets and Their Role in Explainability

Generative Flow Networks (GFlowNets) are central to the XInsight framework. Unlike traditional explainability methods that optimize for a single reward-maximizing explanation, GFlowNets are designed to generate a diverse set of objects, with probabilities proportional to a reward function, facilitating a broader exploration of model behaviors. By implementing GFlowNets within the XInsight framework, the authors aim to provide users multiple perspectives into a GNN's learning patterns, crucial for deriving insights that may otherwise remain obscured.

Figure 1: Generated graphs (8 with cycles and 8 without cycles) to verify XInsight's ability to generate graphs of a specified target class.

XInsight Framework

XInsight produces a distribution of explanations, enabling deeper analysis of the underlying model's decision mechanism. By utilizing GFlowNets trained to generate graph structures aligned with a target class, XInsight can effectively highlight patterns the model associates with certain predictions. This capacity to generate multiple explanations represents a significant advancement over single-sample methods.

Experimental Evaluation

Acyclic Graph Generation

To validate the generative capabilities of XInsight, the authors conducted experiments using synthetically generated acyclic graphs. The GNN trained for this classification task demonstrated high accuracy, confirming that XInsight can produce class-specific graphs as directed by the model.

Figure 2: Distribution of explanations for the Mutagenic classifier generated by the trained XInsight model, with MUTAG class probabilities according to the trained proxy.

MUTAG Dataset Insights

Applying XInsight to the MUTAG dataset allowed the authors to discover meaningful relationships within mutagenic compound classifications. The visualization of system-generated compounds showed distinct clustering based on lipophilicity, verified through QSAR modeling. This indicates XInsight's effectiveness in revealing critical chemical properties related to mutagenicity, thereby confirming previously established scientific hypotheses.

Figure 3: Generated graph embeddings projected onto 2-dimensional plane using UMAP.

Knowledge Discovery and Verification

XInsight's application in knowledge discovery persisted through analyses revealing associations between mutagenicity and compound properties—specifically lipophilicity—a known determinant. Statistical tests confirmed the validity of these relationships within the MUTAG dataset. This underscores XInsight's potential for verifying GNN predictions against established scientific knowledge, a key requirement in domains like drug discovery.

Figure 4: Lipophilicity calculations for 10 of the clustered compounds generated by XInsight using the XLOGP3 method.

Conclusion

The XInsight model demonstrates a significant advancement in GNN explainability by producing diverse explanations through GFlowNets. It not only improves understanding of GNN predictions but also offers a practical tool for uncovering relationships within the represented data, which proves invaluable in scientific fields where model transparency is critical. Future directions involve harnessing XInsight's potential across real-world applications requiring high interpretability for safe and effective deployment.

This novel approach reiterates the necessity of distribution-based explanation frameworks, affirming XInsight’s value in contributing to explainable AI research and practical implementations. It sets a promising trajectory for similar endeavors seeking to bridge gaps between AI model outputs and human-intelligible insights.