Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Sparse High-Dimensional Matrix-Valued Graphical Models From Dependent Data (2404.19073v1)

Published 29 Apr 2024 in stat.ML, cs.LG, and eess.SP

Abstract: We consider the problem of inferring the conditional independence graph (CIG) of a sparse, high-dimensional, stationary matrix-variate Gaussian time series. All past work on high-dimensional matrix graphical models assumes that independent and identically distributed (i.i.d.) observations of the matrix-variate are available. Here we allow dependent observations. We consider a sparse-group lasso-based frequency-domain formulation of the problem with a Kronecker-decomposable power spectral density (PSD), and solve it via an alternating direction method of multipliers (ADMM) approach. The problem is bi-convex which is solved via flip-flop optimization. We provide sufficient conditions for local convergence in the Frobenius norm of the inverse PSD estimators to the true value. This result also yields a rate of convergence. We illustrate our approach using numerical examples utilizing both synthetic and real data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. N. Meinshausen and P. Bühlmann, “High-dimensional graphs and variable selection with the Lasso,” Ann. Statist., vol. 34, no. 3, pp. 1436-1462, 2006.
  2. O. Banerjee, L.E. Ghaoui and A. d’Aspremont, “Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data,” J. Mach. Learn. Res., vol. 9, pp. 485-516, 2008.
  3. J. Friedman, T. Hastie and R. Tibshirani, “Sparse inverse covariance estimation with the graphical lasso,” Biostatistics, vol. 9, no. 3, pp. 432-441, July 2008.
  4. R. Dahlhaus, “Graphical interaction models for multivariate time series,” Metrika, vol. 51, pp. 157-172, 2000.
  5. A. Jung, G. Hannak and N. Goertz, “Graphical LASSO based model selection for time series,” IEEE Signal Process. Lett., vol. 22, no. 10, pp. 1781-1785, Oct. 2015.
  6. J.K. Tugnait, “Graphical modeling of high-dimensional time series,” in Proc. 52nd Asilomar Conf. Signals, Systems, Computers, pp. 840-844, Pacific Grove, CA, Oct. 29 - Oct. 31, 2018.
  7. J.K. Tugnait, “On sparse high-dimensional graphical model learning for dependent time series,” Signal Process., vol. 197, pp. 1-18, Aug. 2022, Article 108539.
  8. E. Avventi, A. Lindquist, and B. Wahlberg, “ARMA identification of graphical models,” IEEE Trans. Autom. Control, vol. 58, no. 5, pp. 1167-1178, 2013.
  9. M. Zorzi and R. Sepulchre, “AR identification of latent-variable graphical models,” IEEE Trans. Autom. Control, vol. 61, no. 9, pp. 2327-2340, 2016.
  10. D. Alpago, M. Zorzi and A. Ferrante, “Identification of sparse reciprocal graphical models,” IEEE Control Sys. Lett., vol. 22, no. 4, pp. 659-664, 2018.
  11. J. Songsiri and L. Vandenberghe, “Topology selection in graphical models of autoregressive processes,” J. Mach. Learn. Res., vol. 11, pp. 2671-2705, Oct. 2010.
  12. V. Ciccone, A. Ferrante and M. Zorzi, “Learning latent variable dynamic graphical models by confidence sets selection,” IEEE Trans. Autom. Control, vol. 65, no. 12, pp. 5130-5143, Dec. 2020.
  13. D. Alpago, M. Zorzi and A. Ferrante, “Data-driven link prediction over graphical models,” IEEE Trans. Autom. Control, vol. 68, no. 4, pp. 2215-2228, Apr. 2023.
  14. S. Basu and G. Michailidis, “Regularized estimation in sparse high-dimensional time series models,” Annals Statist., vol. 43, no. 4, pp. 1535-1567, 2015.
  15. C. Leng and C.Y. Tang, “Sparse matrix graphical models,” J. Amer. Statist. Assoc., vol. 107, pp. 1187-1200, Sep. 2012.
  16. T. Tsiligkaridis, A.O. Hero, III, and S. Zhou, “On convergence of Kronecker graphical lasso algorithms,” IEEE Trans. Signal Process., vol. 61, no. 7, pp. 1743-1755, April 2013.
  17. Y. Zhu and L. Li, “Multiple matrix Gaussian graphs estimation,” J. Royal Statistical Soc., Series B, vol. 80, pp. 927-950, 2018.
  18. X. Chen and W. Liu, “Graph estimation for matrix-variate Gaussian data,” Statistica Sinica, vol. 29, pp. 479-504, 2019.
  19. K. Greenewald, S. Zhou and A. Hero III, “Tensor graphical lasso (teralasso),” J. Royal Statistical Soc., Series B, vol. 81, no. 5, pp. 901-931, 2019.
  20. X. Lyu, W.W. Sun, Z. Wang, H. Liu, J. Yang and G. Cheng, “Tensor graphical model: Non-convex optimization and statistical inference," IEEE Trans. Pattern Analysis Mach. Intell., vol. 42, no. 8, pp. 2024-2037, 1 Aug. 2020.
  21. F. Huang and S. Chen, “Joint learning of multiple sparse matrix Gaussian graphical models," IEEE Trans. Neural Netw. Learning Sys., vol. 26, no. 11, pp. 2606-2620, Nov. 2015.
  22. S. Zhou, “Gemini: Graph estimation with matrix variate normal instances,” Annals Statist., vol. 42, no. 2, pp. 532-562, 2014.
  23. J. Yin and H. Li, “Model selection and estimation in the matrix normal graphical model,” J.  Multivariate Analysis, vol. 107, pp. 119-140, May 2012.
  24. S. He, J. Yin, H. Li and X. Wang, “Graphical model selection and estimation for high dimensional tensor data,” J.  Multivariate Analysis, vol. 128, pp. 165-185, 2014.
  25. K. Min, Q. Mai and X. Zhang, “Fast and separable estimation in high-dimensional tensor Gaussian graphical models,” J. Comp. Graphical Statistics, vol. 31, pp. 294-300, 2022.
  26. K. Werner, M. Jansson and P. Stoica, “On estimation of covariance matrices with Kronecker product structure,” IEEE Trans. Signal Process., vol. 56, no. 2, pp. 478-491, Feb. 2008.
  27. P.M. Weichsel, “The Kronecker product of graphs,” Proc. American Math. Soc., vol. 13, no. 1, pp. 37-52, 1962.
  28. J. Leskovec, D. Chakrabarti, J. Kleinberg, C. Faloutsos, and Z. Ghahramani, “Kronecker graphs: An approach to modeling networks,” J. Mach. Learn. Res., vol. 11, pp. 985-1042, Feb. 2010.
  29. C.M. Carvalho and M. West, “Dynamic matrix-variate graphical models,” Bayesian Analysis, vol. 2, no. 1, pp. 69-98, 2007.
  30. H. Wang and M. West, “Bayesian analysis of matrix normal graphical models,” Biometrika, vol. 96, no. 4, pp. 821-834, Dec. 2009.
  31. Y. Jiang, J. Bigot and S. Maabout, “Online graph topology learning from matrix-valued time series,” arXiv:2107.08020v1 [stat.ML], July 2021.
  32. M. Zorzi, “Nonparametric identification of Kronecker networks,” Automatica, vol. 145, no. 9, p. 110518, Nov. 2022.
  33. B. Sinquin and M. Verhaegen, “Quarks: Identification of large-scale Kronecker vector-autoregressive models,” IEEE Trans. Autom. Control, vol. 64, no. 3, pp. 448-463, 2019.
  34. M. Zorzi, “Autoregressive identification of Kronecker graphical models,” Automatica, vol. 119, no. 9, p. 109053, Sep. 2020.
  35. R. Chen, H. Xiao and D. Yang, “Autoregressive models for matrix-valued time series,” J. Econometrics, vol. 222, pp. 539-560, 2021.
  36. J.K. Tugnait, “Sparse high-dimensional matrix-valued graphical model learning from dependent data,” in Proc. 22nd IEEE Statistical Signal Proc. Workshop (SSP-2023), pp. 344-348, Hanoi, Vietnam, July 2-5, 2023.
  37. J.K. Tugnait, “Sparse-group lasso for graph learning from multi-attribute data,” IEEE Trans. Signal Process., vol. 69, pp. 1771-1786, 2021. (Corrections: vol. 69, p. 4758, 2021.)
  38. A.J. Rothman, P.J. Bickel, E. Levina and J. Zhu, “Sparse permutation invariant covariance estimation,” Elec. J. Statistics, vol. 2, pp. 494-515, 2008.
  39. J.K. Tugnait, “Edge exclusion tests for graphical model selection: Complex Gaussian vectors and time series,” IEEE Trans. Signal Process., vol. 67, no. 19, pp. 5062-5077, Oct. 1, 2019.
  40. J. Friedman, T. Hastie and R. Tibshirani, “A note on the group lasso and a sparse group lasso,” arXiv:1001.0736v1 [math.ST], 5 Jan 2010.
  41. N. Simon, J. Friedman, T. Hastie and R. Tibshirani, “A sparse-group lasso,” J. Comp. Graphical Statistics, vol. 22, pp. 231-245, 2013.
  42. M. Yuan and Y. Lin, “Model selection and estimation in regression with grouped variables,” J. Royal Statistical Soc., Series B, vol. 68, pp. 49-67, 2006.
  43. J. Gorski, F. Pfeuffer and K. Klamroth, “Biconvex sets and optimization with biconvex functions: A survey and extensions,” Math. Methods Operations Res., vol. 66, pp. 373-408, 2007.
  44. S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends in Machine Learning, vol. 3, no. 1, pp. 1-122, 2010.
  45. S. Zhang, B. Guo, A. Dong, J. He, Z.  Xu and S.X. Chen, “Cautionary tales on air-quality improvement in Beijing,” Proc. Royal Soc. A, vol. 473, p. 20170457, 2017.
  46. W. Chen, F. Wang, G. Xiao, J. Wu and S. Zhang, “Air quality of Beijing and impacts of the new ambient air quality standard,” Atmosphere, vol. 6, pp. 1243-1258, 2015.
  47. J. Fan, Y. Feng and Y. Wu, “Network exploration via the adaptive lasso and SCAD penalties,” Annals Applied Statistics, vol. 3, no. 2, pp. 521-541, 2009.
  48. J.K. Tugnait, “Sparse-group log-sum penalized graphical model learning for time series,” in Proc.  2022 IEEE Intern. Conf. Acoustics, Speech, Signal Process. (ICASSP 2022), pp. 5822-5826, Singapore, May 22-27, 2022.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now