Graph Representation Learning via Graphical Mutual Information Maximization (2002.01169v1)

Abstract: The richness in the content of various information networks such as social networks and communication networks provides the unprecedented potential for learning high-quality expressive representations without external supervision. This paper investigates how to preserve and extract the abundant information from graph-structured data into embedding space in an unsupervised manner. To this end, we propose a novel concept, Graphical Mutual Information (GMI), to measure the correlation between input graphs and high-level hidden representations. GMI generalizes the idea of conventional mutual information computations from vector space to the graph domain where measuring mutual information from two aspects of node features and topological structure is indispensable. GMI exhibits several benefits: First, it is invariant to the isomorphic transformation of input graphs---an inevitable constraint in many existing graph representation learning algorithms; Besides, it can be efficiently estimated and maximized by current mutual information estimation methods such as MINE; Finally, our theoretical analysis confirms its correctness and rationality. With the aid of GMI, we develop an unsupervised learning model trained by maximizing GMI between the input and output of a graph neural encoder. Considerable experiments on transductive as well as inductive node classification and link prediction demonstrate that our method outperforms state-of-the-art unsupervised counterparts, and even sometimes exceeds the performance of supervised ones.

• The paper introduces Graphical Mutual Information (GMI) to directly maximize mutual information between graph inputs and embeddings.
• It proposes a novel FMI decomposition that integrates feature and topological information for robust unsupervised learning.
• Experiments demonstrate improved node classification and link prediction, often outperforming state-of-the-art unsupervised and some supervised methods.

Graph Representation Learning via Graphical Mutual Information Maximization

This paper presents an innovative approach to unsupervised graph representation learning by introducing the concept of Graphical Mutual Information (GMI). The work focuses on the challenge of encoding rich graph-structured data into high-quality embeddings without relying on external supervision, which is particularly useful in applications like social networks, communication networks, and other domains where labeled data is scarce or costly.

Core Contributions

The authors propose GMI as a measure of the correlation between input graphs and their high-level hidden representations. This approach generalizes mutual information (MI) computations from vector spaces to graph domains, emphasizing node feature and topological structure aspects. GMI offers several key advantages:

1. Isomorphic Invariance: GMI remains invariant to the isomorphic transformations of input graphs, overcoming constraints in existing graph representation methods.
2. Efficient Estimation: It can be efficiently estimated and maximized using current MI estimation methods such as the Mutual Information Neural Estimation (MINE).
3. Theoretical Justification: The paper provides a solid theoretical foundation, confirming the correctness and rationality of GMI.

Methodology

The primary innovation lies in directly maximizing the GMI between the input and output of a graph neural encoder. This approach deviates from previous methods that rely on indirect correlations and potentially unreliable injective functions. The authors achieve this by:

• Feature Mutual Information (FMI) Decomposition: Proposing a decomposition theorem that breaks down global MI into a weighted sum of local MIs between hidden vectors and input features.
• Topology Consideration: Incorporating topological structure awareness into GMI by correlating node proximity in representation space with their inherent connectivity, thereby capturing both feature and structural information effectively.

Experiments and Findings

The effectiveness of GMI was demonstrated across various tasks:

• Node Classification: Both transductive and inductive experiments showed that GMI-based models often outperformed state-of-the-art unsupervised counterparts, and on some datasets, even surpassed supervised approaches.
• Link Prediction: The method showed significant improvements in link prediction accuracy compared to direct competitors, highlighting its ability to retain rich graph information.

Implications and Future Directions

The introduction of GMI opens up new pathways for unsupervised graph learning by demonstrating that a fine-grained, node-level MI maximization approach is capable of capturing the intricate details of input graphs. This has practical implications for large-scale graph-based applications where supervisory data is limited or unavailable.

In a theoretical context, the GMI framework provides a robust basis for revisiting graph embedding paradigms, potentially influencing future developments in graph neural architectures. Future work could explore extending this methodology to more complex graph types such as heterogeneous graphs and hypergraphs, adapting the GMI maximization strategy to accommodate their unique structures and relationships. Moreover, investigating task-specific adaptations could enhance the applicability and performance of GMI-based models in specialized domains.