Papers
Topics
Authors
Recent
Search
2000 character limit reached

Tensor Methods in High Dimensional Data Analysis: Opportunities and Challenges

Published 28 May 2024 in math.ST, cs.NA, math.NA, stat.ME, stat.ML, and stat.TH | (2405.18412v1)

Abstract: Large amount of multidimensional data represented by multiway arrays or tensors are prevalent in modern applications across various fields such as chemometrics, genomics, physics, psychology, and signal processing. The structural complexity of such data provides vast new opportunities for modeling and analysis, but efficiently extracting information content from them, both statistically and computationally, presents unique and fundamental challenges. Addressing these challenges requires an interdisciplinary approach that brings together tools and insights from statistics, optimization and numerical linear algebra among other fields. Despite these hurdles, significant progress has been made in the last decade. This review seeks to examine some of the key advancements and identify common threads among them, under eight different statistical settings.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (105)
  1. Genevera Allen. Sparse higher-order principal components analysis. In Artificial Intelligence and Statistics, pages 27–36. PMLR, 2012.
  2. Tensor decompositions for learning latent variable models. Journal of machine learning research, 15:2773–2832, 2014a.
  3. Guaranteed non-orthogonal tensor decomposition via alternating rank-1111 updates. arXiv preprint arXiv:1402.5180, 2014b.
  4. Learning overcomplete latent variable models through tensor methods. In Conference on Learning Theory, pages 36–112. PMLR, 2015.
  5. The landscape of the spiked tensor model. Communications on Pure and Applied Mathematics, 72(11):2282–2330, 2019.
  6. On estimating rank-one spiked tensors in the presence of heavy tailed errors. IEEE Transactions on Information Theory, 68(12):8053–8075, 2022.
  7. Perturbation bounds for (nearly) orthogonally decomposable tensors with statistical applications. Information and Inference: A Journal of the IMA, 2023a.
  8. Large dimensional independent component analysis: Statistical optimality and computational tractability. arXiv preprint arXiv:2303.18156, 2023b.
  9. Noisy tensor completion via the sum-of-squares hierarchy. In Conference on Learning Theory, pages 417–445. PMLR, 2016.
  10. Blind signal separation in the presence of gaussian noise. In Conference on Learning Theory, pages 270–287. PMLR, 2013.
  11. Eigenvectors of orthogonally decomposable functions. SIAM Journal on Computing, 47(2):547–615, 2018.
  12. Higher-order organization of complex networks. Science, 353(6295):163–166, 2016.
  13. Rajendra Bhatia. Matrix analysis, volume 169. Springer Science & Business Media, 2013.
  14. Tensors in statistics. Annual review of statistics and its application, 8:345–368, 2021.
  15. Nonconvex low-rank tensor completion from noisy data. Advances in neural information processing systems, 32, 2019.
  16. Uncertainty quantification for nonconvex tensor completion: Confidence intervals, heteroscedasticity and optimality. In International Conference on Machine Learning, pages 1271–1282. PMLR, 2020.
  17. Generalized low-rank plus sparse tensor estimation by fast riemannian optimization. Journal of the American Statistical Association, pages 1–17, 2022a.
  18. Provable tensor-train format tensor completion by riemannian optimization. Journal of Machine Learning Research, 23(123):1–77, 2022b.
  19. Analysis of individual differences in multidimensional scaling via an n-way generalization of Eckart-Young decomposition. Psychometrika, 35(3):283–319, 1970.
  20. Constrained factor models for high-dimensional matrix-variate time series. Journal of the American Statistical Association, 2019a.
  21. Non-convex projected gradient descent for generalized low-rank tensor regression. The Journal of Machine Learning Research, 20(1):172–208, 2019b.
  22. Autoregressive models for matrix-valued time series. Journal of Econometrics, 222(1):539–560, 2021a.
  23. Factor models for high-dimensional tensor time series. Journal of the American Statistical Association, 117(537):94–116, 2022.
  24. Learning mixtures of low-rank models. IEEE Transactions on Information Theory, 67(7):4613–4636, 2021b.
  25. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. IEEE signal processing magazine, 32(2):145–163, 2015.
  26. Pierre Comon. Independent component analysis, 1992.
  27. Handbook of Blind Source Separation: Independent component analysis and applications. Academic press, 2010.
  28. Multiway data analysis. North-Holland Publishing Co., 1989.
  29. On the best rank-1 and rank-(r1,r2,…,rn)subscript𝑟1subscript𝑟2…subscript𝑟𝑛(r_{1},r_{2},\dots,r_{n})( italic_r start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , italic_r start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT , … , italic_r start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT ) approximation of higher-order tensors. SIAM journal on Matrix Analysis and Applications, 21(4):1324–1342, 2000a.
  30. A multilinear singular value decomposition. SIAM journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000b.
  31. The geometry of algorithms with orthogonality constraints. SIAM journal on Matrix Analysis and Applications, 20(2):303–353, 1998.
  32. Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse problems, 27(2):025010, 2011.
  33. Escaping from saddle points—online stochastic gradient for tensor decomposition. In Conference on learning theory, pages 797–842. PMLR, 2015.
  34. Near-optimal sample complexity for convex tensor completion. Information and Inference: A Journal of the IMA, 8(3):577–619, 2019.
  35. Consistency of spectral partitioning of uniform hypergraphs under planted partition model. Advances in Neural Information Processing Systems, 27, 2014.
  36. Wolfgang Hackbusch. Tensor spaces and numerical tensor calculus, volume 42. Springer, 2012.
  37. Exact clustering in tensor block model: Statistical optimality and computational limit. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(5):1666–1698, 2022a.
  38. An optimal statistical and computational framework for generalized tensor estimation. The Annals of Statistics, 50(1):1–29, 2022b.
  39. Cp factor model for dynamic tensors. arXiv preprint arXiv:2110.15517, 2021.
  40. R Harshman. Foundations of the parafac procedure: Model and conditions for an explanatory factor analysis. Technical Report UCLA Working Papers in Phonetics 16, University of California, Los Angeles, Los Angeles, CA, 1970.
  41. Most tensor problems are np-hard. Journal of the ACM (JACM), 60(6):45, 2013.
  42. Frank L Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics, 6(1-4):164–189, 1927.
  43. Tensor principal component analysis via sum-of-square proofs. In Conference on Learning Theory, pages 956–1006, 2015.
  44. Learning mixtures of spherical gaussians: moment methods and spectral decompositions. In Proceedings of the 4th conference on Innovations in Theoretical Computer Science, pages 11–20, 2013.
  45. Aapo Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE transactions on Neural Networks, 10(3):626–634, 1999.
  46. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411–430, 2000.
  47. Independent component analysis. Studies in informatics and control, 11(2):205–207, 2002.
  48. Provable tensor factorization with missing data. Advances in Neural Information Processing Systems, 27, 2014.
  49. On iterative hard thresholding methods for high-dimensional m-estimation. Advances in neural information processing systems, 27, 2014.
  50. Spectral learning on matrices and tensors. Foundations and Trends® in Machine Learning, 12(5-6):393–536, 2019.
  51. Tensor principal component analysis via convex optimization. Mathematical Programming, 150(2):423–457, 2015.
  52. Community detection on mixture multilayer networks via regularized tensor decomposition. The Annals of Statistics, 49(6):3181–3205, 2021.
  53. Ian T Jolliffe. Principal component analysis for special types of data. Springer, 2002.
  54. Spatio-temporal transcriptome of the human brain. Nature, 478(7370):483–489, 2011.
  55. Community detection for hypergraph networks via regularized tensor power iteration. arXiv preprint arXiv:1909.06503, 2019.
  56. Multilayer networks. Journal of complex networks, 2(3):203–271, 2014.
  57. Low-rank tensor completion by riemannian optimization. BIT Numerical Mathematics, 54:447–468, 2014.
  58. Pieter M Kroonenberg. Applied multiway data analysis. John Wiley & Sons, 2008.
  59. Joseph B Kruskal. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear algebra and its applications, 18(2):95–138, 1977.
  60. Tensor completion for estimating missing values in visual data. IEEE transactions on pattern analysis and machine intelligence, 35(1):208–220, 2012.
  61. Characterizing spatiotemporal transcriptome of the human brain via low-rank tensor decomposition. Statistics in Biosciences, pages 1–29, 2022.
  62. A sharp blockwise tensor perturbation bound for orthogonal iteration. The Journal of Machine Learning Research, 22(1):8106–8153, 2021.
  63. Optimal clustering by lloyd algorithm for low-rank mixture model. arXiv preprint arXiv:2207.04600, 2022.
  64. Optimal estimation and computational limit of low-rank gaussian mixtures. The Annals of Statistics, 51(2):646–667, 2023.
  65. Optimal clustering of discrete mixtures: Binomial, poisson, block models, and multi-layer networks. arXiv preprint arXiv:2311.15598, 2023a.
  66. Latent space model for higher-order networks and generalized tensor decomposition. Journal of Computational and Graphical Statistics, pages 1–17, 2023b.
  67. Polynomial-time tensor decompositions with sum-of-squares. In 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), pages 438–446. IEEE, 2016.
  68. A doubly enhanced em algorithm for model-based tensor clustering. Journal of the American Statistical Association, 117(540):2120–2134, 2022.
  69. Peter McCullagh. Tensor methods in statistics. Courier Dover Publications, 2018.
  70. Spectral algorithms for tensor completion. Communications on Pure and Applied Mathematics, 71(11):2381–2425, 2018.
  71. Square deal: Lower bounds and improved relaxations for tensor recovery. In International conference on machine learning, pages 73–81. PMLR, 2014.
  72. Successive rank-one approximations for nearly orthogonally decomposable symmetric tensors. SIAM Journal on Matrix Analysis and Applications, 36(4):1638–1659, 2015.
  73. Greedy approaches to symmetric orthogonal tensor decomposition. SIAM Journal on Matrix Analysis and Applications, 38(4):1210–1226, 2017.
  74. Community structure in time-dependent, multiscale, and multiplex networks. Science, 328(5980):876–878, 2010.
  75. Esa Ollila. The deflation-based fastica estimator: Statistical analysis revisited. IEEE transactions on Signal Processing, 58(3):1527–1541, 2009.
  76. On the multiway principal component analysis. arXiv preprint arXiv:2302.07216, 2023.
  77. Exact tensor completion with sum-of-squares. In Conference on Learning Theory, pages 1619–1673. PMLR, 2017.
  78. Convex regularization for high-dimensional multiresponse tensor regression. The Annals of Statistics, 47(3):1554–1584, 2019.
  79. A statistical model for tensor pca. In Advances in Neural Information Processing Systems, pages 2897–2905, 2014.
  80. Multilinear dynamical systems for tensor time series. Advances in Neural Information Processing Systems, 26, 2013.
  81. Fast and robust tensor decomposition with applications to dictionary learning. In Conference on Learning Theory, pages 1760–1793. PMLR, 2017.
  82. Orthogonalized als: A theoretically principled tensor decomposition algorithm for practical use. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 3095–3104. JMLR. org, 2017.
  83. Tensor decomposition for signal processing and machine learning. IEEE Transactions on Signal Processing, 65(13):3551–3582, 2017.
  84. Matrix perturbation theory. Academic Press, 1990.
  85. Dynamic tensor clustering. Journal of the American Statistical Association, 114(528):1894–1907, 2019.
  86. Provable sparse tensor decomposition. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(3):899–916, 2017.
  87. Mode-wise principal subspace pursuit and matrix spiked covariance model. arXiv preprint arXiv:2307.00575, 2023.
  88. Convex tensor decomposition via structured schatten norm regularization. Advances in neural information processing systems, 26, 2013.
  89. Ledyard R Tucker. Implications of factor analysis of three-way matrices for measurement of change. Problems in measuring change, 15:122–137, 1963.
  90. Ledyard R Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279–311, 1966.
  91. High-dimensional low-rank tensor autoregressive time series modeling. Journal of Econometrics, 238(1):105544, 2024.
  92. Factor models for matrix-valued high-dimensional time series. Journal of econometrics, 208(1):231–248, 2019.
  93. Multiway clustering via tensor block models. Advances in neural information processing systems, 32, 2019.
  94. On polynomial time methods for exact low-rank tensor completion. Foundations of Computational Mathematics, 19(6):1265–1313, 2019.
  95. Effective tensor sketching via sparsification. IEEE Transactions on Information Theory, 67(2):1356–1369, 2021.
  96. The sup-norm perturbation of hosvd and low rank tensor denoising. The Journal of Machine Learning Research, 20(1):2206–2247, 2019.
  97. Statistically optimal and computationally efficient low rank tensor completion from noisy entries. The Annals of Statistics, 49(1):76–99, 2021.
  98. Learning from multiway data: Simple and efficient tensor regression. In International Conference on Machine Learning, pages 373–381. PMLR, 2016.
  99. On tensor completion via nuclear norm minimization. Foundations of Computational Mathematics, 16(4):1031–1068, 2016.
  100. Incoherent tensor norms and their applications in higher order tensor completion. IEEE Transactions on Information Theory, 63(10):6753–6766, 2017.
  101. Optimal sparse singular value decomposition for high-dimensional high-order data. Journal of the American Statistical Association, 114(528):1708–1725, 2019.
  102. Tensor svd: Statistical and computational limits. IEEE Transactions on Information Theory, 64(11):7311–7338, 2018.
  103. Islet: Fast and optimal low-rank tensor regression via importance sketching. SIAM journal on mathematics of data science, 2(2):444–479, 2020.
  104. Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association, 108(502):540–552, 2013.
  105. Heteroskedastic tensor clustering. arXiv preprint arXiv:2311.02306, 2023.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We found no open problems mentioned in this paper.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (3)

  1. Arnab Auddy 
  2. Dong Xia 
  3. Ming Yuan 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 3 tweets with 35 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free