Neural Embeddings of Graphs in Hyperbolic Space (1705.10359v1)

Abstract: Neural embeddings have been used with great success in NLP. They provide compact representations that encapsulate word similarity and attain state-of-the-art performance in a range of linguistic tasks. The success of neural embeddings has prompted significant amounts of research into applications in domains other than language. One such domain is graph-structured data, where embeddings of vertices can be learned that encapsulate vertex similarity and improve performance on tasks including edge prediction and vertex labelling. For both NLP and graph based tasks, embeddings have been learned in high-dimensional Euclidean spaces. However, recent work has shown that the appropriate isometric space for embedding complex networks is not the flat Euclidean space, but negatively curved, hyperbolic space. We present a new concept that exploits these recent insights and propose learning neural embeddings of graphs in hyperbolic space. We provide experimental evidence that embedding graphs in their natural geometry significantly improves performance on downstream tasks for several real-world public datasets.

• The paper demonstrates that hyperbolic embeddings capture hierarchical and power-law network structures more effectively than Euclidean methods.
• It adapts the classical word2vec Skipgram model using Poincaré disks and hyperbolic inner products to optimize graph embeddings.
• Empirical evaluations reveal superior performance over traditional DeepWalk, enhancing tasks such as community detection and vertex classification.

Neural Embeddings of Graphs in Hyperbolic Space: A Novel Approach to Complex Networks

Introduction

The usage of neural embeddings has become prevalent, particularly in the context of NLP, where compact and meaningful representations of words can capture linguistic relationships effectively. This approach's success has spurred exploration into other domains, notably graph-structured data, where vertex embeddings can potentially enhance performance in various tasks such as edge prediction and vertex labeling. Traditionally, these embeddings have been in Euclidean spaces. However, recent insights suggest that the structural complexity inherent in many networked systems might be better captured within hyperbolic space, a notion that this paper investigates by introducing neural embeddings in hyperbolic space for graphs.

Hyperbolic Geometry Foundation

The paper leverages the fundamental qualities of hyperbolic geometry, an area characterized by constant negative curvature. Unlike Euclidean geometry, hyperbolic space facilitates an exponential expansion of circles' area relative to radius, which is particularly beneficial for effectively embedding hierarchical structures and networks with power-law distributions and strong clustering, aspects often found in complex real-world datasets. Furthermore, hyperbolic space offers a continuum analog for tree-like structures, suggesting that such space is naturally fitted to accommodate graphs with similar characteristics.

Methodology

The authors propose adapting the well-known word2vec Skipgram model, originally developed for NLP, to operate within hyperbolic space for graph embeddings. This adaptation involves replacing the two Euclidean vector spaces with Poincaré disks, utilizing hyperbolic inner products as opposed to Euclidean ones. By leveraging negative sampling and Noise Contrastive Estimation (NCE), the model efficiently optimizes embeddings. The choice of using the Poincaré disk model allows for capturing complex geometric features while maintaining computational tractability.

Experimental Evaluation

The effectiveness of hyperbolic embeddings is empirically assessed through a series of experiments on well-known public graph datasets. These datasets include Zachary's Karate Club, Political Books, and others, each with distinct topological features and vertex classification tasks. In comparison to the traditional DeepWalk model operating in Euclidean space, hyperbolic embeddings exhibit superior performance across the board. Notably, in visualization tasks, hyperbolic embeddings afford a clearer visual distinction of community structures, as exemplified in the Karate Club network, where factions are more distinctly separable.

Furthermore, the quantitative assessments via macro F1 scores reveal a substantial advantage in using hyperbolic embeddings, as they consistently outperform varying dimensions of DeepWalk embeddings in Euclidean space. This suggests that the structure-preserving nature of hyperbolic space provides a more natural representation framework for these complex networks.

Conclusion and Implications

The introduction of neural embeddings in hyperbolic space represents a significant conceptual shift from traditional Euclidean embeddings for graph-structured data. The findings elucidate hyperbolic space's inherent suitability for capturing the underlying features of real-world networks with complex topologies. Practically, this work enables advancements in network analysis tasks such as community detection and node classification, with potential impacts extending to diverse fields such as biology, social network analysis, and information retrieval. Theoretically, it opens avenues for further exploration into the interaction between geometric properties of embedding spaces and the structural properties of the data being represented. Future research could explore optimizing such embeddings, possibly integrating them with dynamic or heterogeneous network models.