Learning Large-Scale MTP$_2$ Gaussian Graphical Models via Bridge-Block Decomposition (2309.13405v3)
Abstract: This paper studies the problem of learning the large-scale Gaussian graphical models that are multivariate totally positive of order two ($\text{MTP}_2$). By introducing the concept of bridge, which commonly exists in large-scale sparse graphs, we show that the entire problem can be equivalently optimized through (1) several smaller-scaled sub-problems induced by a \emph{bridge-block decomposition} on the thresholded sample covariance graph and (2) a set of explicit solutions on entries corresponding to bridges. From practical aspect, this simple and provable discipline can be applied to break down a large problem into small tractable ones, leading to enormous reduction on the computational complexity and substantial improvements for all existing algorithms. The synthetic and real-world experiments demonstrate that our proposed method presents a significant speed-up compared to the state-of-the-art benchmarks.
- Gaussian Markov random fields: theory and applications. Monographs on Statistics and Applied Probability, 104, 2005.
- Steffen L Lauritzen. Graphical models, volume 17. Clarendon Press, 1996.
- Joe Whittaker. Graphical models in applied multivariate statistics. Wiley Publishing, 2009.
- Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. The Journal of Machine Learning Research, 9:485–516, 2008.
- Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3):432–441, 2008.
- Ming Yuan and Yi Lin. Model selection and estimation in the Gaussian graphical model. Biometrika, 94(1):19–35, 2007.
- Stability approach to regularization selection for high dimensional graphical models. Advances in Neural Information Processing Systems, 23, 2010.
- The joint graphical lasso for inverse covariance estimation across multiple classes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(2):373–397, 2014.
- Erik Bølviken. Probability inequalities for the multivariate normal with non-negative partial correlations. Scandinavian Journal of Statistics, pages 49–58, 1982.
- Walk-sums and belief propagation in Gaussian graphical models. The Journal of Machine Learning Research, 7:2031–2064, 2006.
- Robert J Plemmons. M-matrix characterizations. I-nonsingular M-matrices. Linear Algebra and its Applications, 18(2):175–188, 1977.
- Total positivity in Markov structures. The Annals of Statistics, pages 1152–1184, 2017.
- Maximum likelihood estimation in Gaussian models under total positivity. The Annals of Statistics, 47(4):1835–1863, 2019.
- Total positivity in multivariate extremes. The Annals of Statistics, 51(3):962–1004, 2023.
- Dependence in elliptical partial correlation graphs. Electronic Journal of Statistics, 15(2):4236–4263, 2021.
- Archimedean copulae and positive dependence. Journal of Multivariate Analysis, 93(2):434–445, 2005.
- Covariance matrix estimation under total positivity for portfolio selection. Journal of Financial Econometrics, 20(2):367–389, 2022.
- Covariance matrix estimation under low-rank factor model with nonnegative correlations. IEEE Transactions on Signal Processing, 70:4020–4030, 2022.
- Estimation of positive definite M-matrices and structure learning for attractive Gaussian markov random fields. Linear Algebra and its Applications, 473:145–179, 2015.
- Learning graphs with monotone topology properties and multiple connected components. IEEE Transactions on Signal Processing, 66(9):2399–2413, 2018.
- A fast algorithm for graph learning under attractive Gaussian Markov random fields. In 2021 55th Asilomar Conference on Signals, Systems, and Computers, pages 1520–1524, 2021.
- Zengde Deng and Anthony Man-Cho So. A fast proximal point algorithm for generalized graph Laplacian learning. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5425–5429. IEEE, 2020.
- Does the ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-norm learn a sparse graph under Laplacian constrained graphical models? arXiv preprint arXiv:2006.14925, 2020.
- Nonconvex sparse graph learning under Laplacian constrained graphical model. In Advances in Neural Information Processing Systems, volume 33, pages 7101–7113, 2020.
- Minimax estimation of Laplacian constrained precision matrices. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130, pages 3736–3744, 2021.
- Graph learning from data under Laplacian and structural constraints. IEEE Journal of Selected Topics in Signal Processing, 11(6):825–841, 2017.
- Network topology inference with sparsity and Laplacian constraints. arXiv preprint arXiv:2309.00960, 2023.
- The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine, 30(3):83–98, 2013.
- Kron reduction of graphs with applications to electrical networks. IEEE Transactions on Circuits and Systems I: Regular Papers, 60(1):150–163, 2012.
- GBST: Separable transforms based on line graphs for predictive video coding. In 2016 IEEE International Conference on Image Processing (ICIP), pages 2375–2379. IEEE, 2016.
- Graphical models in heavy-tailed markets. In Advances in Neural Information Processing Systems, volume 34, pages 19989–20001, 2021.
- Learning bipartite graphs: Heavy tails and multiple components. In Advances in Neural Information Processing Systems, volume 35, pages 14044–14057, 2022.
- A unified framework for structured graph learning via spectral constraints. Journal of Machine Learning Research, 21(22):1–60, 2020.
- Structured graph learning via Laplacian spectral constraints. In Advances in Neural Information Processing Systems, volume 32, pages 11647–11658, 2019.
- Generalized Laplacian precision matrix estimation for graph signal processing. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6350–6354. IEEE, 2016.
- Adaptive estimation of graphical models under total positivity. In International Conference on Machine Learning, pages 40054–40074, 2023.
- Fast projected Newton-like method for precision matrix estimation under total positivity. arXiv preprint arXiv:2112.01939, 2021.
- Somayeh Sojoudi. Equivalence of graphical lasso and thresholding for sparse graphs. The Journal of Machine Learning Research, 17(1):3943–3963, 2016.
- Graphical lasso and thresholding: Equivalence and closed-form solutions. Journal of Machine Learning Research, 2019.
- Jens M Schmidt. A simple test on 2-vertex-and 2-edge-connectivity. Information Processing Letters, 113(7):241–244, 2013.
- Graphical Gaussian models with edge and vertex symmetries. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(5):1005–1027, 2008.
- Michael L Stein. Space–time covariance functions. Journal of the American Statistical Association, 100(469):310–321, 2005.
- Network-constrained regularization and variable selection for analysis of genomic data. Bioinformatics, 24(9):1175–1182, 2008.
- R Endre Tarjan. A note on finding the bridges of a graph. Information Processing Letters, 2(6):160–161, 1974.
- Maintaining bridge-connected and biconnected components on-line. Algorithmica, 7(1):433–464, 1992.
- Efficiently computing k-edge connected components via graph decomposition. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pages 205–216, 2013.
- Finding maximal k-edge-connected subgraphs from a large graph. In Proceedings of the 15th International Conference on Extending Database Technology, pages 480–491, 2012.
- A survey of community search over big graphs. The VLDB Journal, 29(1):353–392, 2020.
- Mining newsgroups using networks arising from social behavior. In Proceedings of the 12th International Conference on World Wide Web, pages 529–535, 2003.
- Refining bridge-block decompositions through two-stage and recursive tree partitioning. arXiv preprint arXiv:2110.06998, 2021.
- Tackling box-constrained optimization via a new projected quasi-Newton approach. SIAM Journal on Scientific Computing, 32(6):3548–3563, 2010.
- Emergence of scaling in random networks. Science, 286(5439):509–512, 1999.
- Statistical mechanics of complex networks. Reviews of Modern Physics, 74(1):47, 2002.
- Stochastic blockmodels: First steps. Social Networks, 5(2):109–137, 1983.
- Santo Fortunato. Community detection in graphs. Physics Reports, 486(3-5):75–174, 2010.
- The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6):1293–1305, 2019.
- Indexing and classifying gigabytes of time series under time warping. In SIAM International Conference on Data Mining, pages 1–10, 2017.
- Mark EJ Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23):8577–8582, 2006.
- The constrained Laplacian rank algorithm for graph-based clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.