Symmetry-driven embedding of networks in hyperbolic space (2406.10711v2)

Abstract: Hyperbolic models are known to produce networks with properties observed empirically in most network datasets, including heavy-tailed degree distribution, high clustering, and hierarchical structures. As a result, several embeddings algorithms have been proposed to invert these models and assign hyperbolic coordinates to network data. Current algorithms for finding these coordinates, however, do not quantify uncertainty in the inferred coordinates. We present BIGUE, a Markov chain Monte Carlo (MCMC) algorithm that samples the posterior distribution of a Bayesian hyperbolic random graph model. We show that the samples are consistent with current algorithms while providing added credible intervals for the coordinates and all network properties. We also show that some networks admit two or more plausible embeddings, a feature that an optimization algorithm can easily overlook.

• The paper introduces BIGUE, a novel MCMC algorithm that samples from the Bayesian posterior to capture uncertainty in hyperbolic network embeddings.
• It employs cluster-based transformations to improve mixing and effective sample sizes, outperforming traditional dynamic HMC and random walk methods.
• Empirical results demonstrate BIGUE's ability to accurately quantify uncertainty, enhance link prediction, and preserve key network properties such as clustering and hierarchy.

Symmetry-Driven Embedding of Networks in Hyperbolic Space

Introduction

The paper "Symmetry-driven embedding of networks in hyperbolic space" addresses a prominent challenge in network science: embedding complex networks in hyperbolic spaces. Hyperbolic models capture several empirical network properties like heavy-tailed degree distributions, high clustering coefficients, and hierarchical structures. However, existing algorithms primarily focus on obtaining point estimates of these embeddings and largely ignore the uncertainty inherent in the process.

Key Contributions

The paper introduces a novel Markov chain Monte Carlo (MCMC) algorithm named BIGUE (Bayesian Inference of a Graph's Unknown Embedding) to sample the posterior distribution of a Bayesian hyperbolic random graph model (HRGM). The significant contributions discussed in the paper include:

1. Posterior Sampling in Hyperbolic Space: The authors emphasize the necessity of quantifying uncertainty in network embeddings by sampling from the posterior distribution rather than maximizing a likelihood function, which existing methods often do.
2. MCMC with Cluster Transformations: The introduction of cluster-based transformation techniques, which involve flipping, translating, and exchanging clusters, improves the mixing of the MCMC algorithm. This approach outperforms the commonly used dynamic Hamiltonian Monte Carlo (HMC) and naive random walk (RW) methods, especially for complex and multimodal posterior distributions.
3. Empirical Validation and Numerical Results: The authors validate their approach on synthetic and several empirical networks, showing that BIGUE can efficiently sample embeddings and provide credible intervals for the parameters, which reflect the true variability and uncertainty in the data.

Methodological Insights

The methodological framework of BIGUE stands on several significant pillars:

• Embedding in Hyperbolic Space: The authors utilize the $\mathbb{S}^1$ hyperbolic space model, which is conceptually close to the hyperbolic plane model ($\mathbb{H}^2$). The $\mathbb{S}^1$ model facilitates inference by simplifying the calculation of the likelihood function while retaining the network's structural properties.
• Bayesian Model and Posterior Distribution: By adopting a Bayesian framework, the authors model the likelihood of the observed network as a function of latent coordinates and parameters. The posterior distribution is then derived using Bayes' theorem, incorporating prior knowledge via independent priors for the model parameters.
• Cluster Transformations for Improved Mixing: Traditional MCMC methods often struggle with the complex and multimodal nature of the posterior distribution in high-dimensional latent spaces. The cluster transformation approach introduced here partitions the network into nearly independent clusters based on angular positions and applies transformations at the cluster level to explore the embedding space more effectively. This enhances the algorithm's ability to move between different modes of the posterior distribution.

Numerical Results and Implications

The paper reports several strong numerical results:

• Effective Sample Size and Mixing: The proposed BIGUE algorithm shows lower autocorrelation and better mixing properties compared to RW and HMC methods. The effective sample sizes are significantly higher, indicating the algorithm's efficacy in exploring the posterior space.
• Posterior Credible Intervals: The credible intervals for the embedding coordinates and the parameters, such as $\beta$ and $\kappa$, derived from BIGUE, provide reliable estimates that encompass the true values, demonstrating the importance of quantifying uncertainty.
• Network Properties and Link Prediction: BIGUE's ability to capture the network's structural properties is validated through measures such as density, clustering, greedy routing success rate, and the global hierarchy level. Additionally, the model's embeddings serve as effective classifiers for link prediction, with the AUC metric closely matching the ground truth.

Practical and Theoretical Implications

The implications of this research span both practical applications and theoretical advancements:

• Enhanced Algorithmic Techniques: The cluster transformation approach can be extended to other embedding tasks and used alongside existing optimization-based methods to enhance their accuracy and robustness.
• Improved Network Analysis: By offering a probabilistic framework for embedding, BIGUE allows researchers to account for the inherent uncertainty in network representations, leading to more reliable downstream tasks such as community detection, link prediction, and network navigation.
• Future Research Directions: The success of BIGUE suggests potential extensions to higher-dimensional hyperbolic spaces ($\mathbb{H}^{D}$) and other types of networks, including directed and weighted graphs. Additionally, combining MCMC-based sampling with optimization techniques for point estimation can further refine embedding algorithms.

Conclusion

The introduction of BIGUE marks a significant step forward in the probabilistic embedding of networks in hyperbolic space. By addressing the multimodality and uncertainty in network models, this research provides a comprehensive framework for more reliable and insightful network analysis. The methodological advancements and empirical validations presented pave the way for future innovations in the field of network science and complex systems.

Acknowledgments

The work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), Sentinelle Nord, and the Fonds de recherche du Québec. The authors acknowledge computing support from Calcul Québec and the Digital Research Alliance of Canada.

For those interested, a Python implementation of BIGUE is available on GitHub at https://github.com/DynamicaLab/bigue.