Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning of Structurally Unambiguous Probabilistic Grammars (2203.09441v3)

Published 17 Mar 2022 in cs.LO and cs.AI

Abstract: The problem of identifying a probabilistic context free grammar has two aspects: the first is determining the grammar's topology (the rules of the grammar) and the second is estimating probabilistic weights for each rule. Given the hardness results for learning context-free grammars in general, and probabilistic grammars in particular, most of the literature has concentrated on the second problem. In this work we address the first problem. We restrict attention to structurally unambiguous weighted context-free grammars (SUWCFG) and provide a query learning algorithm for \structurally unambiguous probabilistic context-free grammars (SUPCFG). We show that SUWCFG can be represented using \emph{co-linear multiplicity tree automata} (CMTA), and provide a polynomial learning algorithm that learns CMTAs. We show that the learned CMTA can be converted into a probabilistic grammar, thus providing a complete algorithm for learning a structurally unambiguous probabilistic context free grammar (both the grammar topology and the probabilistic weights) using structured membership queries and structured equivalence queries. A summarized version of this work was published at AAAI 21.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. D. Angluin and M. Kharitonov. When won’t membership queries help? J. Comput. Syst. Sci., 50(2):336–355, 1995.
  2. Relating probabilistic grammars and automata. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 542–549, 1999.
  3. D. Angluin. Learning regular sets from queries and counterexamples. Inf. Comput., 75(2):87–106, 1987.
  4. D. Angluin. Negative results for equivalence queries. Machine Learning, 5:121–150, 1990.
  5. N. Abe and M. K. Warmuth. On the computational complexity of approximating distributions by probabilistic automata. Machine Learning, 9:205–260, 1992.
  6. J. K. Baker. Trainable grammars for speech recognition. In D. H. Klatt and J. J. Wolf, editors, Speech Communication Papers for the 97th Meeting of the Acoustical Society of America, pages 547–550, 1979.
  7. Learning functions represented as multiplicity automata. J. ACM, 47(3):506–530, 2000. doi:10.1145/337244.337257.
  8. F. Bergadano and S. Varricchio. Learning behaviors of automata from multiplicity and equivalence queries. SIAM J. Comput., 25(6):1268–1280, 1996.
  9. Consistent unsupervised estimators for anchored pcfgs. Trans. Assoc. Comput. Linguistics, 8:409–422, 2020. URL: https://transacl.org/ojs/index.php/tacl/article/view/1936.
  10. Noam Chomsky. Three models for the description of language. IRE Trans. Inf. Theory, 2(3):113–124, 1956. doi:10.1109/TIT.1956.1056813.
  11. Kenneth Ward Church. A stochastic parts program and noun phrase parser for unrestricted text. In Second Conference on Applied Natural Language Processing, pages 136–143, Austin, Texas, USA, February 1988. Association for Computational Linguistics. doi:10.3115/974235.974260.
  12. Alexander Clark. Beyond chomsky normal form: Extending strong learning algorithms for pcfgs. In Jane Chandlee, Rémi Eyraud, Jeff Heinz, Adam Jardine, and Menno van Zaanen, editors, Proceedings of the Fifteenth International Conference on Grammatical Inference, volume 153 of Proceedings of Machine Learning Research, pages 4–17. PMLR, 23–27 Aug 2021. URL: https://proceedings.mlr.press/v153/clark21a.html.
  13. Nonnegative ranks, decompositions, and factorizations of nonnegative matrices. Linear Algebra and its Applications, 190:149–168, 1993.
  14. Query learning of regular tree languages: How to avoid dead states. Theory of Computing Systems, 40(2):163–185, 2007.
  15. C. de la Higuera. Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, USA, 2010.
  16. E. Mark Gold. Complexity of automaton identification from given data. Information and Control, 37(3):302–320, 1978.
  17. Leslie Grate. Automatic RNA secondary structure determination with stochastic context-free grammars. In Christopher J. Rawlings, Dominic A. Clark, Russ B. Altman, Lawrence Hunter, Thomas Lengauer, and Shoshana J. Wodak, editors, Proceedings of the Third International Conference on Intelligent Systems for Molecular Biology, Cambridge, United Kingdom, July 16-19, 1995, pages 136–144. AAAI, 1995. URL: http://www.aaai.org/Library/ISMB/1995/ismb95-017.php.
  18. Learning multiplicity tree automata. In International Colloquium on Grammatical Inference, pages 268–280. Springer, 2006.
  19. Introduction to Automata Theory, Languages, and Computation. Addison-Wesley Publishing Company, 1979.
  20. Skeletal structural descriptions. Information and Control, 39(2):192 – 211, 1978. doi:10.1016/S0019-9958(78)90849-5.
  21. K. Lari and S. J. Young. The estimation of stochastic context-free grammars using the inside-outside algorithm. Computer Speech and Language, 4:35–56, 1990.
  22. Learning of structurally unambiguous probabilistic grammars. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 9170–9178, 2021. URL: https://ojs.aaai.org/index.php/AAAI/article/view/17107.
  23. L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. In PROCEEDINGS OF THE IEEE, pages 257–286, 1989.
  24. Rommel G. Regis. On the properties of positive spanning sets and positive bases. Optimization and Engineering, 17(1):229–262, Mar 2016. doi:10.1007/s11081-015-9286-x.
  25. Y. Sakakibara. Learning context-free grammars from structural data in polynomial time. In Proceedings of the First Annual Workshop on Computational Learning Theory, COLT ’88, Cambridge, MA, USA, August 3-5, 1988, pages 330–344, 1988. URL: http://dl.acm.org/citation.cfm?id=93109.
  26. Yasubumi Sakakibara. Efficient learning of context-free grammars from positive structural examples. Inform. Comput., 97:23–60, 1992.
  27. Weighted and probabilistic context-free grammars are equally expressive. Computational Linguistics, 33(4):477–491, 2007.
  28. L**{}^{\mbox{*}}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT-based learning of markov decision processes (extended version). Formal Aspects Comput., 33(4):575–615, 2021. doi:10.1007/s00165-021-00536-5.
  29. Learning weighted automata over principal ideal domains. In Foundations of Software Science and Computation Structures - 23rd International Conference, FOSSACS 2020, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020, Dublin, Ireland, April 25-30, 2020, Proceedings, pages 602–621, 2020. doi:10.1007/978-3-030-45231-5_31.
  30. Learning deterministic weighted automata with queries and counterexamples. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada, pages 8558–8569, 2019. URL: http://papers.nips.cc/paper/9062-learning-deterministic-weighted-automata-with-queries-and-counterexamples.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now