Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Large-Scale MTP$_2$ Gaussian Graphical Models via Bridge-Block Decomposition (2309.13405v3)

Published 23 Sep 2023 in cs.LG and eess.SP

Abstract: This paper studies the problem of learning the large-scale Gaussian graphical models that are multivariate totally positive of order two ($\text{MTP}_2$). By introducing the concept of bridge, which commonly exists in large-scale sparse graphs, we show that the entire problem can be equivalently optimized through (1) several smaller-scaled sub-problems induced by a \emph{bridge-block decomposition} on the thresholded sample covariance graph and (2) a set of explicit solutions on entries corresponding to bridges. From practical aspect, this simple and provable discipline can be applied to break down a large problem into small tractable ones, leading to enormous reduction on the computational complexity and substantial improvements for all existing algorithms. The synthetic and real-world experiments demonstrate that our proposed method presents a significant speed-up compared to the state-of-the-art benchmarks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Gaussian Markov random fields: theory and applications. Monographs on Statistics and Applied Probability, 104, 2005.
  2. Steffen L Lauritzen. Graphical models, volume 17. Clarendon Press, 1996.
  3. Joe Whittaker. Graphical models in applied multivariate statistics. Wiley Publishing, 2009.
  4. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. The Journal of Machine Learning Research, 9:485–516, 2008.
  5. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3):432–441, 2008.
  6. Ming Yuan and Yi Lin. Model selection and estimation in the Gaussian graphical model. Biometrika, 94(1):19–35, 2007.
  7. Stability approach to regularization selection for high dimensional graphical models. Advances in Neural Information Processing Systems, 23, 2010.
  8. The joint graphical lasso for inverse covariance estimation across multiple classes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(2):373–397, 2014.
  9. Erik Bølviken. Probability inequalities for the multivariate normal with non-negative partial correlations. Scandinavian Journal of Statistics, pages 49–58, 1982.
  10. Walk-sums and belief propagation in Gaussian graphical models. The Journal of Machine Learning Research, 7:2031–2064, 2006.
  11. Robert J Plemmons. M-matrix characterizations. I-nonsingular M-matrices. Linear Algebra and its Applications, 18(2):175–188, 1977.
  12. Total positivity in Markov structures. The Annals of Statistics, pages 1152–1184, 2017.
  13. Maximum likelihood estimation in Gaussian models under total positivity. The Annals of Statistics, 47(4):1835–1863, 2019.
  14. Total positivity in multivariate extremes. The Annals of Statistics, 51(3):962–1004, 2023.
  15. Dependence in elliptical partial correlation graphs. Electronic Journal of Statistics, 15(2):4236–4263, 2021.
  16. Archimedean copulae and positive dependence. Journal of Multivariate Analysis, 93(2):434–445, 2005.
  17. Covariance matrix estimation under total positivity for portfolio selection. Journal of Financial Econometrics, 20(2):367–389, 2022.
  18. Covariance matrix estimation under low-rank factor model with nonnegative correlations. IEEE Transactions on Signal Processing, 70:4020–4030, 2022.
  19. Estimation of positive definite M-matrices and structure learning for attractive Gaussian markov random fields. Linear Algebra and its Applications, 473:145–179, 2015.
  20. Learning graphs with monotone topology properties and multiple connected components. IEEE Transactions on Signal Processing, 66(9):2399–2413, 2018.
  21. A fast algorithm for graph learning under attractive Gaussian Markov random fields. In 2021 55th Asilomar Conference on Signals, Systems, and Computers, pages 1520–1524, 2021.
  22. Zengde Deng and Anthony Man-Cho So. A fast proximal point algorithm for generalized graph Laplacian learning. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5425–5429. IEEE, 2020.
  23. Does the ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-norm learn a sparse graph under Laplacian constrained graphical models? arXiv preprint arXiv:2006.14925, 2020.
  24. Nonconvex sparse graph learning under Laplacian constrained graphical model. In Advances in Neural Information Processing Systems, volume 33, pages 7101–7113, 2020.
  25. Minimax estimation of Laplacian constrained precision matrices. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130, pages 3736–3744, 2021.
  26. Graph learning from data under Laplacian and structural constraints. IEEE Journal of Selected Topics in Signal Processing, 11(6):825–841, 2017.
  27. Network topology inference with sparsity and Laplacian constraints. arXiv preprint arXiv:2309.00960, 2023.
  28. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine, 30(3):83–98, 2013.
  29. Kron reduction of graphs with applications to electrical networks. IEEE Transactions on Circuits and Systems I: Regular Papers, 60(1):150–163, 2012.
  30. GBST: Separable transforms based on line graphs for predictive video coding. In 2016 IEEE International Conference on Image Processing (ICIP), pages 2375–2379. IEEE, 2016.
  31. Graphical models in heavy-tailed markets. In Advances in Neural Information Processing Systems, volume 34, pages 19989–20001, 2021.
  32. Learning bipartite graphs: Heavy tails and multiple components. In Advances in Neural Information Processing Systems, volume 35, pages 14044–14057, 2022.
  33. A unified framework for structured graph learning via spectral constraints. Journal of Machine Learning Research, 21(22):1–60, 2020.
  34. Structured graph learning via Laplacian spectral constraints. In Advances in Neural Information Processing Systems, volume 32, pages 11647–11658, 2019.
  35. Generalized Laplacian precision matrix estimation for graph signal processing. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6350–6354. IEEE, 2016.
  36. Adaptive estimation of graphical models under total positivity. In International Conference on Machine Learning, pages 40054–40074, 2023.
  37. Fast projected Newton-like method for precision matrix estimation under total positivity. arXiv preprint arXiv:2112.01939, 2021.
  38. Somayeh Sojoudi. Equivalence of graphical lasso and thresholding for sparse graphs. The Journal of Machine Learning Research, 17(1):3943–3963, 2016.
  39. Graphical lasso and thresholding: Equivalence and closed-form solutions. Journal of Machine Learning Research, 2019.
  40. Jens M Schmidt. A simple test on 2-vertex-and 2-edge-connectivity. Information Processing Letters, 113(7):241–244, 2013.
  41. Graphical Gaussian models with edge and vertex symmetries. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(5):1005–1027, 2008.
  42. Michael L Stein. Space–time covariance functions. Journal of the American Statistical Association, 100(469):310–321, 2005.
  43. Network-constrained regularization and variable selection for analysis of genomic data. Bioinformatics, 24(9):1175–1182, 2008.
  44. R Endre Tarjan. A note on finding the bridges of a graph. Information Processing Letters, 2(6):160–161, 1974.
  45. Maintaining bridge-connected and biconnected components on-line. Algorithmica, 7(1):433–464, 1992.
  46. Efficiently computing k-edge connected components via graph decomposition. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pages 205–216, 2013.
  47. Finding maximal k-edge-connected subgraphs from a large graph. In Proceedings of the 15th International Conference on Extending Database Technology, pages 480–491, 2012.
  48. A survey of community search over big graphs. The VLDB Journal, 29(1):353–392, 2020.
  49. Mining newsgroups using networks arising from social behavior. In Proceedings of the 12th International Conference on World Wide Web, pages 529–535, 2003.
  50. Refining bridge-block decompositions through two-stage and recursive tree partitioning. arXiv preprint arXiv:2110.06998, 2021.
  51. Tackling box-constrained optimization via a new projected quasi-Newton approach. SIAM Journal on Scientific Computing, 32(6):3548–3563, 2010.
  52. Emergence of scaling in random networks. Science, 286(5439):509–512, 1999.
  53. Statistical mechanics of complex networks. Reviews of Modern Physics, 74(1):47, 2002.
  54. Stochastic blockmodels: First steps. Social Networks, 5(2):109–137, 1983.
  55. Santo Fortunato. Community detection in graphs. Physics Reports, 486(3-5):75–174, 2010.
  56. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6):1293–1305, 2019.
  57. Indexing and classifying gigabytes of time series under time warping. In SIAM International Conference on Data Mining, pages 1–10, 2017.
  58. Mark EJ Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23):8577–8582, 2006.
  59. The constrained Laplacian rank algorithm for graph-based clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.
Citations (1)

Summary

We haven't generated a summary for this paper yet.