Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Robust Model Selection of Gaussian Graphical Models (2211.05690v2)

Published 10 Nov 2022 in stat.ML, cs.LG, math.ST, and stat.TH

Abstract: In Gaussian graphical model selection, noise-corrupted samples present significant challenges. It is known that even minimal amounts of noise can obscure the underlying structure, leading to fundamental identifiability issues. A recent line of work addressing this "robust model selection" problem narrows its focus to tree-structured graphical models. Even within this specific class of models, exact structure recovery is shown to be impossible. However, several algorithms have been developed that are known to provably recover the underlying tree-structure up to an (unavoidable) equivalence class. In this paper, we extend these results beyond tree-structured graphs. We first characterize the equivalence class up to which general graphs can be recovered in the presence of noise. Despite the inherent ambiguity (which we prove is unavoidable), the structure that can be recovered reveals local clustering information and global connectivity patterns in the underlying model. Such information is useful in a range of real-world problems, including power grids, social networks, protein-protein interactions, and neural structures. We then propose an algorithm which provably recovers the underlying graph up to the identified ambiguity. We further provide finite sample guarantees in the high-dimensional regime for our algorithm and validate our results through numerical simulations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. A. Anandkumar and R. Valluvan. Learning loopy graphical models with latent variables: Efficient methods and guarantees. The Annals of Statistics, pages 401–435, 2013.
  2. Grid topology identification with hidden nodes via structured norm minimization. IEEE Control Systems Letters, 6:1244–1249, 2022.
  3. Network reconfiguration in distribution systems for loss reduction and load balancing. IEEE Transactions on Power delivery, 4(2):1401–1407, 1989.
  4. Graph Theory. Oxford University Press, 1986.
  5. Brain graphs: graphical models of the human brain connectome. Annual review of clinical psychology, 7:113–140, 2011.
  6. Measurement error in nonlinear models, volume 105. CRC press, 1995.
  7. Robust estimation of tree structured models. arXiv preprint arXiv:2102.05472, 2021.
  8. Nonparanormal graph quilting with applications to calcium imaging. Stat, 12(1):e623, 2023.
  9. Robust covariance matrix estimation via matrix depth. arXiv preprint arXiv:1506.00691, 2015.
  10. Robust sparse regression under adversarial corruption. In International Conference on Machine Learning, pages 774–782. PMLR, 2013.
  11. Learning latent tree graphical models. J. of Machine Learning Research, 12:1771–1812, 2011.
  12. G. Dasarathy. Gaussian graphical model selection from size constrained measurements. In 2019 IEEE International Symposium on Information Theory (ISIT), pages 1302–1306. IEEE, 2019.
  13. Data requirement for phylogenetic inference from multiple loci: a new distance method. IEEE/ACM transactions on computational biology and bioinformatics, 12(2):422–432, 2014.
  14. A stochastic farris transform for genetic data under the multispecies coalescent with applications to data requirements. Journal of Mathematical Biology, 84(5):1–37, 2022.
  15. One breaker is enough: Hidden topology attacks on power grids. In 2015 IEEE Power & Energy Society General Meeting, pages 1–5. IEEE, 2015.
  16. Graphical models in meshed distribution grids: Topology estimation, change detection & limitations. IEEE Transactions on Smart Grid, 11(5):4299–4310, 2020.
  17. M. Drton and M. H. Maathuis. Structure learning in graphical modeling. Annual Review of Statistics and Its Application, 4:365–393, 2017.
  18. A few logs suffice to build (almost) all trees: Part ii. Theoretical Computer Science, 221(1-2):77–118, 1999.
  19. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3):432–441, 2008.
  20. F. Harary. Graph Theory. Addison Wesley series in mathematics. Addison-Wesley, 1971.
  21. S. Herbert et al. The architecture of complexity. Proceedings of the American Philosophical Society, 106(6):467–482, 1962.
  22. Matrix analysis. Cambridge university press, 2012.
  23. J. T. Hwang. Multiplicative errors-in-variables models with applications to recent data released by the us department of energy. Journal of the American Statistical Association, 81(395):680–688, 1986.
  24. Polynomial regression and estimating functions in the presence of multiplicative measurement error. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61(3):547–561, 1999.
  25. M. Kalisch and P. Bühlman. Estimating high-dimensional directed acyclic graphs with the pc-algorithm. Journal of Machine Learning Research, 8(3), 2007.
  26. Robust estimation of tree structured gaussian graphical models. In International Conference on Machine Learning, pages 3292–3300. PMLR, 2019.
  27. Robust estimation of tree structured ising models. arXiv preprint arXiv:2006.05601, 2020.
  28. M. Kim and P. Smaragdis. Single channel source separation using smooth nonnegative matrix factorization with markov random fields. In 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE, 2013.
  29. A. Krishnamurthy and A. Singh. Robust multi-source network tomography using selective probes. In 2012 Proceedings IEEE INFOCOM, pages 1629–1637. IEEE, 2012.
  30. S. L. Lauritzen. Graphical models, volume 17. Clarendon Press, 1996.
  31. High-dimensional robust precision matrix estimation: Cellwise corruption under e⁢p⁢s⁢i⁢l⁢o⁢n𝑒𝑝𝑠𝑖𝑙𝑜𝑛epsilonitalic_e italic_p italic_s italic_i italic_l italic_o italic_n-contamination. Electronic Journal of Statistics, 12(1):1429–1467, 2018.
  32. High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity. Advances in Neural Information Processing Systems, 24, 2011.
  33. K. Lounici. High-dimensional covariance matrix estimation with missing observations. Bernoulli, 20(3):1029–1058, 2014.
  34. Handbook of graphical models. CRC Press, 2018.
  35. Loopy belief propagation for approximate inference: An empirical study. arXiv preprint arXiv:1301.6725, 2013.
  36. Distributionally robust inverse covariance estimation: The wasserstein shrinkage estimator. Operations Research, 70(1):490–515, 2022.
  37. Learning tree structures from noisy data. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1771–1782. PMLR, 2019.
  38. V. Öllerer and C. Croux. Robust high-dimensional precision matrix estimation. In Modern nonparametric, robust and multivariate methods, pages 325–350. Springer, 2015.
  39. T. Ott and R. Stoop. The neurodynamics of belief propagation on binary markov random fields. Advances in neural information processing systems, 19, 2006.
  40. L. RJa and D. Rubin. Statistical analysis with missing data. 1987.
  41. N. Saitou and M. Nei. The neighbor-joining method: a new method for reconstructing phylogenetic trees. Molecular biology and evolution, 4(4):406–425, 1987.
  42. T. Schneider. Analysis of incomplete climate data: Estimation of mean values and covariance matrices and imputation of missing values. Journal of climate, 14(5):853–871, 2001.
  43. Phylogenetics, volume 24. Oxford University Press on Demand, 2003.
  44. Testing unfaithful gaussian graphical models. Advances in Neural Information Processing Systems, 27:2681–2689, 2014.
  45. O. Sporns. Graph theory methods: applications in brain networks. Dialogues in clinical neuroscience, 2018.
  46. H. Sun and H. Li. Robust gaussian graphical modeling via l1 penalization. Biometrics, 68(4):1197–1206, 2012.
  47. SGA: A robust algorithm for partial recovery of tree-structured graphical models with noisy samples. arXiv preprint arXiv:2101.08917, 2021.
  48. Geometry of the faithfulness assumption in causal inference. The Annals of Statistics, pages 436–463, 2013.
  49. Graph quilting: graphical model selection from partially observed covariances. arXiv preprint arXiv:1912.05573, 2019.
  50. J.-K. Wang et al. Robust inverse covariance estimation under noisy measurements. In International Conference on Machine Learning, pages 928–936. PMLR, 2014.
  51. Q. Xu and J. You. Covariate selection for linear errors-in-variables regression models. Communications in Statistics—Theory and Methods, 36(2):375–386, 2007.
  52. E. Yang and A. C. Lozano. Robust gaussian graphical modeling with the trimmed graphical lasso. Advances in Neural Information Processing Systems, 28, 2015.
  53. F. Zhang and V. Tan. Robustifying algorithms of learning latent trees with vector variables. Advances in Neural Information Processing Systems, 34, 2021.
  54. L. Zheng and G. I. Allen. Graphical model inference with erosely measured data. arXiv preprint arXiv:2210.11625, 2022.
  55. Incorporating prior biological knowledge for network-based differential gene expression analysis using differentially weighted graphical lasso. BMC bioinformatics, 18(1):1–14, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets