Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 167 tok/s
Gemini 2.5 Pro 50 tok/s Pro
GPT-5 Medium 16 tok/s Pro
GPT-5 High 17 tok/s Pro
GPT-4o 125 tok/s Pro
Kimi K2 191 tok/s Pro
GPT OSS 120B 430 tok/s Pro
Claude Sonnet 4.5 38 tok/s Pro
2000 character limit reached

Exact recovery in Gaussian weighted stochastic block model and planted dense subgraphs: Statistical and algorithmic thresholds (2402.12515v1)

Published 19 Feb 2024 in math.ST, math.PR, and stat.TH

Abstract: In this paper, we study the exact recovery problem in the Gaussian weighted version of the Stochastic block model with two symmetric communities. We provide the information-theoretic threshold in terms of the signal-to-noise ratio (SNR) of the model and prove that when SNR $<1$, no statistical estimator can exactly recover the community structure with probability bounded away from zero. On the other hand, we show that when SNR $>1$, the Maximum likelihood estimator itself succeeds in exactly recovering the community structure with probability approaching one. Then, we provide two algorithms for achieving exact recovery. The Semi-definite relaxation as well as the spectral relaxation of the Maximum likelihood estimator can recover the community structure down to the threshold value of 1, establishing the absence of an information-computation gap for this model. Next, we compare the problem of community detection with the problem of recovering a planted densely weighted community within a graph and prove that the exact recovery of two symmetric communities is a strictly easier problem than recovering a planted dense subgraph of size half the total number of nodes, by establishing that when the same SNR$< 3/2$, no statistical estimator can exactly recover the planted community with probability bounded away from zero. More precisely, when $1 <$ SNR $< 3/2$ exact recovery of community detection is possible, both statistically and algorithmically, but it is impossible to exactly recover the planted community, even statistically, in the Gaussian weighted model. Finally, we show that when SNR $>2$, the Maximum likelihood estimator itself succeeds in exactly recovering the planted community with probability approaching one. We also prove that the Semi-definite relaxation of the Maximum likelihood estimator can recover the planted community structure down to the threshold value of 2.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (21)
  1. Emmanuel Abbe. Community detection and stochastic block models: Recent developments. Journal of Machine Learning Research, 18(177):1–86, 2018.
  2. Exact recovery in the stochastic block model. IEEE Transactions on Information Theory, 62(1):471–487, 2016.
  3. Community detection in random networks. The Annals of Statistics, 42 Number 3:940 – 969, 2014.
  4. Entrywise eigenvector analysis of random matrices with low expected rank. Annals of statistics, 48(3):1452 – 1474, 2020.
  5. The Probabilistic Method. Wiley Publishing, 4th edition, 2016.
  6. Afonso S. Bandeira. Random laplacian matrices and convex relaxations. Foundations of Computational Mathematics, 18 Number 2 Pages 345–379, 2018.
  7. Notes on computational-to-statistical gaps: predictions using statistical physics. Port. Math., 75 no. 2, pp. 159–186, 2018.
  8. Improved graph clustering. IEEE Transactions on Information Theory, 60(10):6440–6455, oct 2014.
  9. Community detection: Exact recovery in weighted graphs. arXiv 2102.04439, 2021.
  10. Heuristics for semirandom graph problems. Journal of Computer and System Sciences, Volume 63, Number 4, Pages 639-671, 2001.
  11. A survey of statistical network models. Found. Trends Mach. Learn., 2(2):129–233, feb 2010.
  12. Stochastic blockmodels: First steps. Social Networks Volume 5 Number 2 Pages 109-137, 1983.
  13. Achieving exact cluster recovery threshold via semidefinite programming. 2015 IEEE International Symposium on Information Theory (ISIT), pages 1442–1446, 2015.
  14. Achieving exact cluster recovery threshold via semidefinite programming. IEEE Trans. Inf. Theor., 62(5):2788–2797, may 2016.
  15. Recovering communities in weighted stochastic block models. 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1308–1315, 2015.
  16. Information-theoretic bounds for exact recovery in weighted stochastic block models using the renyi divergence. CoRR, abs/1509.06418, 2015.
  17. A polylogarithmic approximation of the minimum bisection. SIAM Journal on Computing, 31(4):1090–1118, 2002.
  18. Consistency thresholds for the planted bisection model. Electronic Journal of Probability, 21(none), jan 2016.
  19. Cristopher Moore. The computer science and physics of community detection: Landscapes, phase transitions, and hardness. Bull. EATCS, 121, 2017.
  20. Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2018.
  21. Optimal rates for community estimation in the weighted stochastic block model. The Annals of Statistics, 48 Number 1 Pages 183 – 204, 2020.
Citations (1)

Summary

We haven't generated a summary for this paper yet.

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 1 tweet and received 0 likes.

Upgrade to Pro to view all of the tweets about this paper: