Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Semi-Autoregressive Graph Generative Model for Dependency Graph Parsing (2306.12018v1)

Published 21 Jun 2023 in cs.CL and cs.AI

Abstract: Recent years have witnessed the impressive progress in Neural Dependency Parsing. According to the different factorization approaches to the graph joint probabilities, existing parsers can be roughly divided into autoregressive and non-autoregressive patterns. The former means that the graph should be factorized into multiple sequentially dependent components, then it can be built up component by component. And the latter assumes these components to be independent so that they can be outputted in a one-shot manner. However, when treating the directed edge as an explicit dependency relationship, we discover that there is a mixture of independent and interdependent components in the dependency graph, signifying that both aforementioned models fail to precisely capture the explicit dependencies among nodes and edges. Based on this property, we design a Semi-Autoregressive Dependency Parser to generate dependency graphs via adding node groups and edge groups autoregressively while pouring out all group elements in parallel. The model gains a trade-off between non-autoregression and autoregression, which respectively suffer from the lack of target inter-dependencies and the uncertainty of graph generation orders. The experiments show the proposed parser outperforms strong baselines on Enhanced Universal Dependencies of multiple languages, especially achieving $4\%$ average promotion at graph-level accuracy. Also, the performances of model variations show the importance of specific parts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. ReZero is all you need: fast convergence at large depth. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence (UAI), pages 1352–1361, Virtual Event.
  2. One SPRING to rule them both: Symmetric AMR semantic parsing and generation without a complex pipeline. In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI), pages 12564–12573, Virtual Event.
  3. Overview of the IWPT 2020 shared task on parsing into enhanced universal dependencies. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), pages 151–161, Online.
  4. From raw text to enhanced Universal Dependencies: The parsing shared task at IWPT 2021. In Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), pages 146–157, Online.
  5. Deng Cai and Wai Lam. 2019. Core semantic first: A top-down approach for AMR parsing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3797–3807, Hong Kong, China.
  6. Deng Cai and Wai Lam. 2020. AMR parsing via graph-sequence iterative inference. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1290–1301, Online.
  7. Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 740–750, Doha, Qatar.
  8. Order matters: Probabilistic modeling of node sequence for graph generation. In Proceedings of the 38th International Conference on Machine Learning (ICML), pages 1630–1639, Virtual Event.
  9. Bi-directional attention with agreement for dependency parsing. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2204–2214, Austin, TX.
  10. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 8440–8451, Online.
  11. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In Proceedings of the 5th International Conference on Learning Representations (ICLR), Toulon, France.
  12. Daniel Fernández-González and Carlos Gómez-Rodríguez. 2019. Left-to-right dependency parsing with pointer networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 710–716, Minneapolis, MN.
  13. Stefan Grünewald and Annemarie Friedrich. 2020. Robertnlp at the IWPT 2020 shared task: Surprisingly simple enhanced UD parsing for english. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), pages 245–252, Online.
  14. RobertNLP at the IWPT 2021 shared task: Simple enhanced UD parsing for 17 languages. In Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), Online.
  15. Graph-based dependency parsing with graph neural networks. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), pages 2475–2485, Florence, Italy.
  16. Turku enhanced parser pipeline: From raw text to enhanced graphs in the IWPT 2020 shared task. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), pages 162–173, Online.
  17. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Easy-first dependency parsing with hierarchical tree LSTMs. Trans. Assoc. Comput. Linguistics, 4:445–461.
  18. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR), Toulon, France.
  19. Daniel Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing universal dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–2795, Hong Kong, China.
  20. Global greedy dependency parsing. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), pages 8319–8326, New York, NY.
  21. Extracting knowledge from web text with monte carlo tree search. In Proceedings of the Web Conference (WWW), pages 2585–2591, Taipei.
  22. Stack-pointer networks for dependency parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1403–1414, Melbourne, Australia.
  23. Global-aware beam search for neural abstractive summarization. In Advances in Neural Information Processing Systems (NeurIPS), pages 16545–16557, virtual.
  24. Sequence level training with recurrent neural networks. In Proceedings of the 4th International Conference on Learning Representations (ICLR), San Juan, Puerto Rico.
  25. Sebastian Schuster and Christopher D. Manning. 2016. Enhanced english universal dependencies: An improved representation for natural language understanding tasks. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC), Portorož, Slovenia.
  26. Tianze Shi and Lillian Lee. 2021. TGIF: Tree-graph integrated-format parser for enhanced UD with two-stage generic- to individual-language finetuning. In Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), pages 213–224, Online.
  27. A Predicate-Function-Argument Annotation of Natural Language for Open-Domain Information eXpression. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2140–2150, Online.
  28. Mingming Sun and Ping Li. 2019. Graph to graph: a topology aware approach for graph structures learning and generation. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), pages 2946–2955, Naha, Okinawa, Japan.
  29. Attention is all you need. In Advances in Neural Information Processing Systems (NIPS), pages 5998–6008, Long Beach, CA.
  30. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR), Vancouver, Canada.
  31. OIE@OIA: an adaptable and efficient open information extraction framework. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pages 6213–6226, Dublin, Ireland.
  32. Second-order semantic dependency parsing with end-to-end neural networks. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), pages 4609–4618, Florence, Italy.
  33. Enhanced Universal Dependency parsing with automated concatenation of embeddings. In Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), pages 189–195, Online.
  34. Enhanced universal dependency parsing with second-order inference and mixture of training data. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT), pages 215–220, Online.
  35. Structured training for neural network transition-based parsing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL), pages 323–333, Beijing, China.
  36. Do transformers really perform badly for graph representation? In Advances in Neural Information Processing Systems (NeurIPS), pages 28877–28888, virtual.
  37. AMR parsing as sequence-to-graph transduction. In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), pages 80–94, Florence, Italy.
  38. Broad-coverage semantic parsing as transduction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3784–3796, Hong Kong, China.
  39. Efficient second-order treecrf for neural dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3295–3305, Online.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now