Papers
Topics
Authors
Recent
2000 character limit reached

Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining

Published 1 Dec 2023 in cs.CL | (2312.00874v1)

Abstract: The knowledge graph is a structure to store and represent knowledge, and recent studies have discussed its capability to assist LLMs for various applications. Some variations of knowledge graphs aim to record arguments and their relations for computational argumentation tasks. However, many must simplify semantic types to fit specific schemas, thus losing flexibility and expression ability. In this paper, we propose the Hierarchical Argumentation Graph (Hi-ArG), a new structure to organize arguments. We also introduce two approaches to exploit Hi-ArG, including a text-graph multi-modal model GreaseArG and a new pre-training framework augmented with graph information. Experiments on two argumentation tasks have shown that after further pre-training and fine-tuning, GreaseArG supersedes same-scale LLMs on these tasks, while incorporating graph information during further pre-training can also improve the performance of vanilla LLMs. Code for this paper is available at https://github.com/ljcleo/Hi-ArG .

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Data acquisition for argument search: The args.me corpus. In KI 2019: Advances in Artificial Intelligence, pages 48–59, Cham. Springer International Publishing.
  2. End-to-end argumentation knowledge graph construction. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7367–7374.
  3. Employing argumentation knowledge graphs for neural argument generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4744–4754, Online. Association for Computational Linguistics.
  4. The moral debater: A study on the computational generation of morally framed arguments. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8782–8797, Dublin, Ireland. Association for Computational Linguistics.
  5. Key point analysis via contrastive learning and extractive argument summarization. In Proceedings of the 8th Workshop on Argument Mining, pages 184–189, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  6. Neural methods for logical reasoning over knowledge graphs. In International Conference on Learning Representations.
  7. Dbpedia: A nucleus for a web of open data. In The Semantic Web, pages 722–735, Berlin, Heidelberg. Springer Berlin Heidelberg.
  8. Stance detection with bidirectional conditional encoding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 876–885, Austin, Texas. Association for Computational Linguistics.
  9. Graph pre-training for AMR parsing and generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6001–6015, Dublin, Ireland. Association for Computational Linguistics.
  10. Abstract Meaning Representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 178–186, Sofia, Bulgaria. Association for Computational Linguistics.
  11. A neural transition-based model for argumentation mining. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6354–6364, Online. Association for Computational Linguistics.
  12. From arguments to key points: Towards automatic argument summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4029–4039, Online. Association for Computational Linguistics.
  13. Quantitative argument summarization and beyond: Cross-domain key point analysis. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 39–49, Online. Association for Computational Linguistics.
  14. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD ’08, pages 1247–1250, New York, NY, USA. Association for Computing Machinery.
  15. PropBank: Semantics of new predicate types. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 3013–3019, Reykjavik, Iceland. European Language Resources Association (ELRA).
  16. Katarzyna Budzynska and Chris Reed. 2011. Whence inference. Technical report, University of Dundee.
  17. IAM: A comprehensive and large-scale dataset for integrated argument mining tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2277–2287, Dublin, Ireland. Association for Computational Linguistics.
  18. Can unsupervised knowledge transfer from social discussions help argument mining? In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7774–7786, Dublin, Ireland. Association for Computational Linguistics.
  19. Vijay Prakash Dwivedi and Xavier Bresson. 2020. A generalization of transformer networks to graphs.
  20. Neural end-to-end learning for computational argumentation mining. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11–22, Vancouver, Canada. Association for Computational Linguistics.
  21. Analyzing the Persuasive Effect of Style in News Editorial Argumentation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3154–3160, Online. Association for Computational Linguistics.
  22. Overview of the 2021 key point analysis shared task. In Proceedings of the 8th Workshop on Argument Mining, pages 154–164, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  23. The workweek is the best time to start a family – a study of GPT-2 based claim generation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 528–544, Online. Association for Computational Linguistics.
  24. Ivan Habernal and Iryna Gurevych. 2017. Argumentation mining in user-generated web discourse. Computational Linguistics, 43(1):125–179.
  25. Causenet: Towards a causality graph extracted from the web. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, pages 3023–3030, New York, NY, USA. Association for Computing Machinery.
  26. Team enigma at ArgMining-EMNLP 2021: Leveraging pre-trained language models for key point matching. In Proceedings of the 8th Workshop on Argument Mining, pages 200–205, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  27. Unsupervised stance detection for arguments from consequences. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 50–60, Online. Association for Computational Linguistics.
  28. Towards an argumentative content search engine using weak supervision. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2066–2081, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  29. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.
  30. A unified syntax-aware framework for semantic role labeling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2401–2411, Brussels, Belgium. Association for Computational Linguistics.
  31. Marco Lippi and Paolo Torroni. 2016. Argument mining from speech: Detecting claims in political debates. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1).
  32. Roberta: A robustly optimized bert pretraining approach.
  33. OpenAI. 2023. Gpt-4 technical report.
  34. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106.
  35. Toward machine-assisted participation in erulemaking: An argumentation model of evaluability. In Proceedings of the 15th International Conference on Artificial Intelligence and Law, ICAIL ’15, pages 206–210, New York, NY, USA. Association for Computing Machinery.
  36. Isaac Persing and Vincent Ng. 2016. End-to-end argumentation mining in student essays. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1384–1394, San Diego, California. Association for Computational Linguistics.
  37. Matching the statements: A simple and accurate model for key point analysis. In Proceedings of the 8th Workshop on Argument Mining, pages 165–174, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  38. Aligning English strings with Abstract Meaning Representation graphs. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 425–429, Doha, Qatar. Association for Computational Linguistics.
  39. Improving language understanding by generative pre-training. OpenAI blog.
  40. Modern talking in key point analysis: Key point matching using pretrained encoders. In Proceedings of the 8th Workshop on Argument Mining, pages 175–183, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  41. João António Rodrigues and António Branco. 2022. Transferring confluent knowledge to argument mining. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6859–6874, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  42. Aspect-controlled neural argument generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 380–396, Online. Association for Computational Linguistics.
  43. Peng Shi and Jimmy Lin. 2019. Simple bert models for relation extraction and semantic role labeling.
  44. An autonomous debating system. Nature, 591(7850):379–384.
  45. Conceptnet 5.5: An open multilingual graph of general knowledge. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1).
  46. ArgumenText: Searching for arguments in heterogeneous sources. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 21–25, New Orleans, Louisiana. Association for Computational Linguistics.
  47. Christian Stab and Iryna Gurevych. 2014. Identifying argumentative discourse structures in persuasive essays. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 46–56, Doha, Qatar. Association for Computational Linguistics.
  48. Christian Stab and Iryna Gurevych. 2017. Parsing argumentation structures in persuasive essays. Computational Linguistics, 43(3):619–659.
  49. Stephen E. Toulmin. 2003. The Uses of Argument. Cambridge University Press.
  50. Computational argumentation quality assessment in natural language. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 176–187, Valencia, Spain. Association for Computational Linguistics.
  51. Neural network-based abstract generation for opinions and arguments. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 47–57, San Diego, California. Association for Computational Linguistics.
  52. A survey of pretraining on graphs: Taxonomy, methods, and applications.
  53. A survey of knowledge enhanced pre-trained models.
  54. GreaseLM: Graph REASoning enhanced language models. In International Conference on Learning Representations.
  55. Rethinking and scaling up graph contrastive learning: An extremely efficient approach with group discrimination. In Advances in Neural Information Processing Systems.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up