Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning and Testing Latent-Tree Ising Models Efficiently (2211.13291v2)

Published 23 Nov 2022 in cs.LG, cs.DS, math.PR, math.ST, and stat.TH

Abstract: We provide time- and sample-efficient algorithms for learning and testing latent-tree Ising models, i.e. Ising models that may only be observed at their leaf nodes. On the learning side, we obtain efficient algorithms for learning a tree-structured Ising model whose leaf node distribution is close in Total Variation Distance, improving on the results of prior work. On the testing side, we provide an efficient algorithm with fewer samples for testing whether two latent-tree Ising models have leaf-node distributions that are close or far in Total Variation distance. We obtain our algorithms by showing novel localization results for the total variation distance between the leaf-node distributions of tree-structured Ising models, in terms of their marginals on pairs of leaves.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. Learning and testing causal models with interventions. Advances in Neural Information Processing Systems, 31, 2018.
  2. Latent variable models in econometrics. Handbook of econometrics, 2:1321–1393, 1984.
  3. Latent variable models and factor analysis: A unified approach, volume 904. John Wiley & Sons, 2011.
  4. Near-optimal learning of tree-structured distributions by chow-liu. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, 2021.
  5. Christopher M Bishop. Latent variable models. In Learning in graphical models, pages 371–403. Springer, 1998.
  6. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859–877, 2017.
  7. Chow-liu++: Optimal prediction-centric learning of tree ising models. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pages 417–426. IEEE, 2022.
  8. Guy Bresler. Efficiently learning ising models on arbitrary graphs. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 771–782, 2015.
  9. Learning a tree-structured ising model in order to make predictions. The Annals of Statistics, 48(2):713–737, 2020.
  10. Learning restricted boltzmann machines via influence maximization. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pages 828–839, 2019.
  11. Multi-item mechanisms without item-independence: Learnability via robustness. In Proceedings of the 21st ACM Conference on Economics and Computation, pages 715–761, 2020.
  12. Testing bayesian networks. In Conference on Learning Theory, pages 370–448. PMLR, 2017.
  13. Joseph T Chang. Full reconstruction of markov models on evolutionary trees: identifiability and consistency. Mathematical biosciences, 137(1):51–73, 1996.
  14. Maximum likelihood of evolutionary trees is hard. In Annual International Conference on Research in Computational Molecular Biology, pages 296–310. Springer, 2005.
  15. C Chow and T Wagner. Consistency of an estimate of tree-dependent probability distributions (corresp.). IEEE Transactions on Information Theory, 19(3):369–371, 1973.
  16. Approximating discrete probability distributions with dependence trees. IEEE transactions on Information Theory, 14(3):462–467, 1968.
  17. Evolutionary trees can be learned in polynomial time in the two-state general markov model. SIAM Journal on Computing, 31(2):375–397, 2001.
  18. Miklós Csurös. Fast recovery of evolutionary trees with thousands of nodes. Journal of Computational Biology, 9(2):277–297, 2002.
  19. Learning ising models from one or multiple samples. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 161–168, 2021.
  20. Square hellinger subadditivity for bayesian networks and its applications to identity testing. In Conference on Learning Theory, pages 697–703. PMLR, 2017.
  21. Sample-optimal and efficient learning of tree ising models. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, 2021.
  22. Optimal phylogenetic reconstruction. In Proceedings of the thirty-eighth annual ACM symposium on Theory of computing, pages 159–168, 2006.
  23. Phylogenies without branch bounds: Contracting the short, pruning the deep. In Annual International Conference on Research in Computational Molecular Biology, pages 451–465. Springer, 2009.
  24. Evolutionary trees and the ising model on the bethe lattice: a proof of steel’s conjecture. Probability Theory and Related Fields, 149(1):149–189, 2011.
  25. Testing ising models. IEEE Transactions on Information Theory, 65(11):6829–6852, 2019.
  26. Where does em converge in gaussian latent tree models? In Conference on Learning Theory (COLT), 2022.
  27. Bootstrapping em via power em and convergence in the naive bayes model. In International Conference on Artificial Intelligence and Statistics, pages 2056–2064. PMLR, 2018.
  28. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1–22, 1977.
  29. The minimax learning rates of normal and ising undirected graphical models. Electronic Journal of Statistics, 14(1):2338–2361, 2020.
  30. Gans with conditional independence graphs: On subadditivity of probability divergences. In International Conference on Artificial Intelligence and Statistics, pages 3709–3717. PMLR, 2021.
  31. A few logs suffice to build (almost) all trees (i). Random Structures & Algorithms, 14(2):153–184, 1999.
  32. B Everett. An introduction to latent variable models. Springer Science & Business Media, 2013.
  33. Joseph Felsenstein. Maximum-likelihood estimation of evolutionary trees from continuous characters. American journal of human genetics, 25(5):471, 1973.
  34. Joseph Felsenstein. Evolutionary trees from gene frequencies and quantitative characters: finding maximum likelihood estimates. Evolution, pages 1229–1242, 1981.
  35. Joseph Felsenstein. Inferring phylogenies, volume 2. Sinauer associates Sunderland, MA, 2004.
  36. Surbhi Goel. Learning restricted boltzmann machines with arbitrary external fields. arXiv preprint arXiv:1906.06595, 2019.
  37. From boltzmann machines to neural networks and back again. Advances in Neural Information Processing Systems, 33:6354–6365, 2020.
  38. Fast and reliable reconstruction of phylogenetic trees with very short edges. In SODA, volume 8, pages 379–388, 2008.
  39. Information theoretic properties of markov random fields, and their algorithmic applications. Advances in Neural Information Processing Systems, 30, 2017.
  40. Disk-covering, a fast-converging method for phylogenetic tree reconstruction. Journal of computational biology, 6(3-4):369–386, 1999.
  41. On learning discrete graphical models using group-sparse regularization. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 378–387. JMLR Workshop and Conference Proceedings, 2011.
  42. Michael I Jordan. Graphical models. Statistical science, 19(1):140–155, 2004.
  43. Statistical estimation from dependent data. In International Conference on Machine Learning, pages 5269–5278. PMLR, 2021.
  44. On the complexity of distance-based evolutionary tree. In Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms, page 444. SIAM, 2003.
  45. Learning graphical models using multiplicative weights. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pages 343–354. IEEE, 2017.
  46. Frederic Koehler. A note on minimax learning of tree models. 2020.
  47. Probabilistic graphical models: principles and techniques. MIT press, 2009.
  48. Steffen L Lauritzen. Graphical models, volume 17. Clarendon Press, 1996.
  49. Comet: A mesquite package for comparing models of continuous character evolution on phylogenies. Evolutionary Bioinformatics, 2:117693430600200022, 2006.
  50. Learning to sample from censored markov random fields. In Conference on Learning Theory, pages 3419–3451. PMLR, 2021.
  51. Elchanan Mossel. Distorted metrics on trees and phylogenetic forests. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 4(1):108–116, 2007.
  52. Learning nonsingular phylogenies and hidden markov models. In Proceedings of the thirty-seventh annual ACM symposium on Theory of computing, pages 366–375, 2005.
  53. Pac-learning bounded tree-width graphical models. In Proc. 20th Ann. Conf. on Uncertainty in Artificial Intelligence (UAI), 2004.
  54. Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan kaufmann, 1988.
  55. High-dimensional ising model selection using l 1-regularized logistic regression. The Annals of Statistics, 38(3):1287–1319, 2010.
  56. Sebastien Roch. A short proof that phylogenetic tree reconstruction by maximum likelihood is hard. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 3(1):92–94, 2006.
  57. Sebastien Roch. Toward extracting all phylogenetic information from matrices of evolutionary distances. Science, 327(5971):1376–1379, 2010.
  58. Phase transition in the sample complexity of likelihood-based phylogeny inference. Probability Theory and Related Fields, 169(1):3–62, 2017.
  59. Information-theoretic limits of selecting binary graphical models in high dimensions. IEEE Transactions on Information Theory, 58(7):4117–4134, 2012.
  60. Alexandros Stamatakis. Raxml-vi-hpc: maximum likelihood-based phylogenetic analyses with thousands of taxa and mixed models. Bioinformatics, 22(21):2688–2690, 2006.
  61. A large-deviation analysis of the maximum-likelihood learning of markov tree structures. IEEE Transactions on Information Theory, 57(3):1714–1735, 2011.
  62. Maximum likelihood estimation for brownian motion tree models based on one sample. arXiv preprint arXiv:2112.00816, 2021.
  63. Interaction screening: Efficient and sample-optimal learning of ising models. Advances in neural information processing systems, 29, 2016.
  64. Efficient learning of discrete graphical models. Journal of Statistical Mechanics: Theory and Experiment, 2021(12):124017, 2022.
  65. Yi Wang and Nevin Lianwen Zhang. Severity of local maxima for the em algorithm: Experiences with hierarchical latent class models. In Probabilistic Graphical Models, pages 301–308. Citeseer, 2006.
  66. Ziheng Yang. Paml: a program package for phylogenetic analysis by maximum likelihood. Computer applications in the biosciences, 13(5):555–556, 1997.
  67. Yannis G Yatracos. Rates of convergence of minimum distance estimators and kolmogorov’s entropy. The Annals of Statistics, 13(2):768–774, 1985.
  68. Maximum likelihood estimation for linear gaussian covariance models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(4):1269–1292, 2017.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now