Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Non-negative Tensor Mixture Learning for Discrete Density Estimation (2405.18220v1)

Published 28 May 2024 in stat.ML and cs.LG

Abstract: We present an expectation-maximization (EM) based unified framework for non-negative tensor decomposition that optimizes the Kullback-Leibler divergence. To avoid iterations in each M-step and learning rate tuning, we establish a general relationship between low-rank decomposition and many-body approximation. Using this connection, we exploit that the closed-form solution of the many-body approximation can be used to update all parameters simultaneously in the M-step. Our framework not only offers a unified methodology for a variety of low-rank structures, including CP, Tucker, and Train decompositions, but also their combinations forming mixtures of tensors as well as robust adaptive noise modeling. Empirically, we demonstrate that our framework provides superior generalization for discrete density estimation compared to conventional tensor-based approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. (1987). Congressional Voting Records. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5C01P.
  2. (1989). Solar Flare. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5530G.
  3. Amari, S.-i. (2016). Information geometry and its applications, volume 194. Springer.
  4. Chess (King-Rook vs. King). UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C57W2S.
  5. Tree tensor networks for generative modeling. Phys. Rev. B, 99:155131.
  6. On tensors, sparsity, and nonnegative factorizations. SIAM Journal on Matrix Analysis and Applications, 33(4):1272–1299.
  7. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1–22.
  8. Fast Tucker rank reduction for non-negative tensors using mean-field approximation. In Advances in Neural Information Processing Systems, volume 34, pages 443–454, Virtual Event.
  9. Many-body approximation for non-negative tensors. In Advances in Neural Information Processing Systems, volume 36, pages 257–292, New Orleans, US.
  10. Expressive power of tensor-network factorizations for probabilistic modeling. Advances in neural information processing systems, 32.
  11. Hayes-Roth. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5501T.
  12. Kullback-Leibler principal component for tensors is not NP-hard. In 2017 51st Asilomar Conference on Signals, Systems, and Computers, pages 693–697. IEEE.
  13. Matrix product states algorithms and continuous systems. Physical Review B, 75(10):104305.
  14. Jensen, J. L. W. V. (1906). Sur les fonctions convexes et les inégalités entre les valeurs moyennes. Acta mathematica, 30(1):175–193.
  15. Nonnegative Tucker decomposition with alpha-divergence. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1829–1832.
  16. Knowledge discovery approach to automated cardiac spect diagnosis. Artificial intelligence in medicine, 23(2):149–169.
  17. A multiresolution non-negative tensor factorization approach for single channel sound source separation. Signal Processing, 105:56–69.
  18. Image completion using low tensor tree rank and total variation minimization. IEEE Transactions on Multimedia, 21(2):338–350.
  19. Tensor-train density estimation. In Uncertainty in artificial intelligence, pages 1321–1331. PMLR.
  20. Orús, R. (2014). A practical introduction to tensor networks: Matrix product states and projected entangled pair states. Annals of Physics, 349:117–158.
  21. Fast and efficient algorithms for nonnegative Tucker decomposition. In Advances in Neural Networks-ISNN 2008: 5th International Symposium on Neural Networks, ISNN 2008, Beijing, China, September 24-28, 2008, Proceedings, Part II 5, pages 772–782. Springer.
  22. Silverman, B. W. (2018). Density estimation for statistics and data analysis. Routledge.
  23. Cluster ensembles—a knowledge reuse framework for combining multiple partitions. Journal of machine learning research, 3(Dec):583–617.
  24. Non-negative multiple tensor factorization. In 2013 IEEE 13th International Conference on Data Mining, pages 1199–1204. IEEE.
  25. Thrun, S. (1991). The monk’s problems: A performance comparison of different learning algorithems. Technical Report of Carnegie Mellon University.
  26. Tomczak, J. M. (2021). Deep Generative Modeling. Cham: Springer International Publishing.
  27. Wu, C. J. (1983). On the convergence properties of the em algorithm. The Annals of statistics, pages 95–103.
  28. Kullback-leibler divergence for nonnegative matrix factorization. In International Conference on Artificial Neural Networks, pages 250–257. Springer.
  29. Lymphography. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C54598.
  30. Primary Tumor. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5WK5Q.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now