Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Facilitating Opinion Diversity through Hybrid NLP Approaches (2405.09439v1)

Published 15 May 2024 in cs.CL and cs.AI

Abstract: Modern democracies face a critical issue of declining citizen participation in decision-making. Online discussion forums are an important avenue for enhancing citizen participation. This thesis proposal 1) identifies the challenges involved in facilitating large-scale online discussions with NLP, 2) suggests solutions to these challenges by incorporating hybrid human-AI technologies, and 3) investigates what these technologies can reveal about individual perspectives in online discussions. We propose a three-layered hierarchy for representing perspectives that can be obtained by a mixture of human intelligence and LLMs. We illustrate how these representations can draw insights into the diversity of perspectives and allow us to investigate interactions in online discussions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (126)
  1. G. Adomavicius and A. Tuzhilin. 2005. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6):734–749.
  2. A research agenda for hybrid intelligence: Augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence. Computer, 53(8):18–28.
  3. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351.
  4. Lora Aroyo and Chris Welty. 2015. Truth is a lie: Crowd truth and the seven myths of human annotation. AI Magazine, 36(1):15–24.
  5. Fine-tuning language models to find agreement among humans with diverse preferences. In Advances in Neural Information Processing Systems, volume 35, pages 38176–38189. Curran Associates, Inc.
  6. Exposure to ideologically diverse news and opinion on facebook. Science, 348(6239):1130–1132.
  7. Every bite is an experience: Key Point Analysis of business reviews. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3376–3386, Online. Association for Computational Linguistics.
  8. Active semi-supervision for pairwise constrained clustering. In Proceedings of the 2004 SIAM International Conference on Data Mining (SDM), pages 333–344.
  9. Which examples should be multiply annotated? active learning when annotators may disagree. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10352–10371, Toronto, Canada. Association for Computational Linguistics.
  10. Toward a perspectivist turn in ground truthing for predictive computing. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6):6860–6868.
  11. Cost-effective crowdsourced entity resolution: A partial-order approach. In Proceedings of the 2016 International Conference on Management of Data, SIGMOD ’16, page 969–984, New York, NY, USA. Association for Computing Machinery.
  12. A survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol., 15(3).
  13. Seeing things from a different angle:discovering diverse perspectives about claims. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 542–557, Minneapolis, Minnesota. Association for Computational Linguistics.
  14. Design challenges for a multi-perspective search engine. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 293–303, Seattle, United States. Association for Computational Linguistics.
  15. The echo chamber effect on social media. Proceedings of the National Academy of Sciences, 118(9):e2023301118.
  16. Luke Collins and Brigitte Nerlich. 2019. Examining user comments for deliberative democracy: A corpus-driven analysis of the climate change debate online. In Climate Change Communication and the Internet, pages 41–59. Routledge.
  17. Combining click-stream data with nlp tools to better understand mooc completion. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, LAK ’16, page 6–14, New York, NY, USA. Association for Computing Machinery.
  18. Rowena Cullen. 2001. Addressing the digital divide. Online information review, 25(5):311–320.
  19. Lincoln Dahlberg. 2001. The internet and democratic discourse: Exploring the prospects of online deliberative forums extending the public sphere. Information, communication & society, 4(4):615–633.
  20. Performance in e-learning: online participation and student grades. British Journal of Educational Technology, 36(4):657–663.
  21. A contextual hybrid intelligent system design for diabetes lifestyle management. In International Workshop Modelling and Representing Context, ECAI, volume 23.
  22. Toward a quality model for hybrid intelligence teams. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, pages 1–10, Auckland. To appear.
  23. Hybrid intelligence. Business & Information Systems Engineering, 61:637–643.
  24. Harnessing the power of LLMs: Evaluating human-AI text co-creation through the lens of news headline generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3321–3339, Singapore. Association for Computational Linguistics.
  25. Mark Dingemanse and Andreas Liesenfeld. 2022. From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5614–5633, Dublin, Ireland. Association for Computational Linguistics.
  26. Comprehensive viewpoint representations for a deeper understanding of user interactions with debated topics. In Proceedings of the 2022 Conference on Human Information Interaction and Retrieval, CHIIR ’22, page 135–145, New York, NY, USA. Association for Computing Machinery.
  27. The crisis of democracy and the science of deliberation. Science, 363(6432):1144–1146.
  28. Towards measuring the representation of subjective global opinions in language models.
  29. Corpus wide argument mining—a working solution. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):7683–7691.
  30. Predicting moderation of deliberative arguments: Is argument quality the key? In Proceedings of the 8th Workshop on Argument Mining, pages 133–141, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  31. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11737–11762, Toronto, Canada. Association for Computational Linguistics.
  32. Evaluating superhuman models with consistency checks. In Socially Responsible Language Modelling Research.
  33. Dennis Friess and Christiane Eilders. 2015. A systematic review of online deliberation research. Policy & Internet, 7(3):319–339.
  34. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673.
  35. Chapter two - moral foundations theory: The pragmatic validity of moral pluralism. In Patricia Devine and Ashby Plant, editors, Advances in Experimental Social Psychology, volume 47 of Advances in Experimental Social Psychology, pages 55–130. Academic Press.
  36. Leo Groarke. 2024. Informal Logic. In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy, Spring 2024 edition. Metaphysics Research Lab, Stanford University.
  37. The argument reasoning comprehension task: Identification and reconstruction of implicit warrants. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1930–1940, New Orleans, Louisiana. Association for Computational Linguistics.
  38. The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation.
  39. Jeffrey Heer. 2019. Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences, 116(6):1844–1850.
  40. Aligning AI with shared human values. In International Conference on Learning Representations.
  41. Social media self-efficacy and information evaluation online. Computers in Human Behavior, 39:254–262.
  42. Moral foundations twitter corpus: A collection of 35k tweets annotated for moral sentiment. Social Psychological and Personality Science, 11(8):1057–1071.
  43. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.
  44. Socially augmented argumentation tools: Rationale, design and evaluation of a debate dashboard. International Journal of Human-Computer Studies, 72(3):298–319.
  45. Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural network approach. IEEE Journal of Biomedical and Health Informatics, 24(10):2733–2742.
  46. Mistral 7b.
  47. Yohan Jo and Alice H. Oh. 2011. Aspect and sentiment unification model for online review analysis. In Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, WSDM ’11, page 815–824, New York, NY, USA. Association for Computing Machinery.
  48. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Computational Linguistics.
  49. Dynabench: Rethinking benchmarking in NLP. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4110–4124, Online. Association for Computational Linguistics.
  50. Identifying the human values behind arguments. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4459–4471, Dublin, Ireland. Association for Computational Linguistics.
  51. SemEval-2023 task 4: ValueEval: Identification of human values behind arguments. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2287–2303, Toronto, Canada. Association for Computational Linguistics.
  52. Mark Klein. 2012. Enabling large-scale deliberation using attention-mediation metrics. Computer Supported Cooperative Work (CSCW), 21:449–473.
  53. Michele Knobel and Colin Lankshear. 2008. Digital literacy and participation in online social networking spaces. Digital literacies: Concepts, policies and practices, 11:249–278.
  54. Slipping to the extreme: A mixed method to explain how extreme opinions infiltrate online discussions. Proceedings of the International AAAI Conference on Web and Social Media, 16(1):524–535.
  55. Dilek Küçük and Fazli Can. 2020. Stance detection: A survey. ACM Comput. Surv., 53(1).
  56. Discord questions: A computational approach to diversity analysis in news coverage. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5180–5194, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  57. Improving diversity of demographic representation in large language models via collective-critiques and self-voting. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10383–10405, Singapore. Association for Computational Linguistics.
  58. Mining, assessing, and improving arguments in NLP and the social sciences. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts, pages 1–6, Dubrovnik, Croatia. Association for Computational Linguistics.
  59. John Lawrence and Chris Reed. 2020. Argument Mining: A Survey. Computational Linguistics, 45(4):765–818.
  60. Computational social science. Science, 323(5915):721–723.
  61. Piyawat Lertvittayakumjorn and Francesca Toni. 2021. Explanation-Based Human Debugging of NLP Models: A Survey. Transactions of the Association for Computational Linguistics, 9:1508–1528.
  62. Towards understanding and mitigating social biases in language models. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 6565–6576. PMLR.
  63. Discovering genres of online discussion threads via text mining. Computers & Education, 52(2):481–495.
  64. Value inference in sociotechnical systems. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’23, page 1774–1780, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems.
  65. Value preferences estimation and disambiguation in hybrid participatory systems.
  66. Bing Liu. 2012. Sentiment Analysis and Opinion Mining. Springer International Publishing.
  67. Ines Lörcher and Monika Taddicken. 2017. Discussing climate change online. topics and perceptions in online climate change communication in different online public arenas. Journal of Science Communication, 16(2):A03.
  68. How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences, 108(22):9020–9025.
  69. Human centered NLP with user-factor adaptation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1146–1155, Copenhagen, Denmark. Association for Computational Linguistics.
  70. Predict responsibly: Improving fairness and accuracy by learning to defer. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.
  71. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Comput. Surv., 56(2).
  72. Moralization in social networks and the emergence of violence during protests. Nature human behaviour, 2(6):389–396.
  73. Annotating perspectives on vaccination. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4964–4973, Marseille, France. European Language Resources Association.
  74. Public participation in crisis policymaking. how 30,000 dutch citizens advised their government on relaxing covid-19 lockdown measures. PLOS ONE, 16(5):1–42.
  75. Vocal minority versus silent majority: Discovering the opionions of the long tail. In 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, pages 103–110.
  76. German Neubaum and Nicole C. Krämer. 2017. Monitoring the opinion of the crowd: Psychological mechanisms underlying public opinion perceptions on social media. Media Psychology, 20(3):502–531.
  77. What can we learn from collective human opinions on natural language inference data? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9131–9143, Online. Association for Computational Linguistics.
  78. The shifted and the overlooked: A task-oriented investigation of user-GPT interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2375–2393, Singapore. Association for Computational Linguistics.
  79. Interactive concept learning for uncovering latent themes in large text collections. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5059–5080, Toronto, Canada. Association for Computational Linguistics.
  80. Barbara Plank. 2022. The “problem” of human label variation: On ground truth in data, modeling and evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  81. Development and validation of the personal values dictionary: A theory–driven tool for investigating references to basic human values in text. European Journal of Personality, 34(5):885–902.
  82. DEBAGREEMENT: A comment-reply dataset for (dis)agreement detection in online debates. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
  83. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  84. Automated configuration and usage of strategy portfolios for mixed-motive bargaining. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’22, page 1101–1109, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems.
  85. Milton Rokeach. 1967. Rokeach value survey. The nature of human values.
  86. Identifying morality frames in political tweets using relational learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9939–9958, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  87. Evaluating agent interactions through episodic knowledge graphs. In Proceedings of the 1st Workshop on Customized Chat Grounding Persona and Knowledge, pages 15–28, Gyeongju, Republic of Korea. Association for Computational Linguistics.
  88. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 29971–30004. PMLR.
  89. NLPositionality: Characterizing design biases of datasets and models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9080–9102, Toronto, Canada. Association for Computational Linguistics.
  90. Akrati Saxena and Harita Reddy. 2022. Users roles identification on online crowdsourced Q& platforms and encyclopedias: a survey. Journal of Computational Social Science, 5(1):285–317.
  91. Are emergent abilities of large language models a mirage? In Advances in Neural Information Processing Systems, volume 36, pages 55565–55581. Curran Associates, Inc.
  92. Shalom H Schwartz. 2012. An overview of the schwartz theory of basic values. Online readings in Psychology and Culture, 2(1):11.
  93. Reason against the machine? Future directions for mass online deliberation. Frontiers in Political Science.
  94. Using information-seeking argument mining to improve service. Journal of Service Research, 25(4):537–548.
  95. G. Smith. 2009. Democratic Innovations: Designing Institutions for Citizen Participation. Theories of Institutional Design. Cambridge University Press.
  96. The dynamics of message selection in online political discussion forums: Self-segregation or diverse exposure? Communication Research, 47(1):125–152.
  97. Measuring political deliberation: A discourse quality index. Comparative European Politics, 1:21–48.
  98. A review of natural language processing techniques for opinion mining systems. Information Fusion, 36:10–25.
  99. James Surowiecki. 2004. The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations. Doubleday.
  100. Indicative summarization of long discussions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2752–2788, Singapore. Association for Computational Linguistics.
  101. No language left behind: Scaling human-centered machine translation.
  102. Knowledge engineering for hybrid intelligence. In Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP ’23, page 75–82, Pensacola, FL, USA,. Association for Computing Machinery.
  103. AI for social good: unlocking the opportunity for positive impact. Nature Communications, 11(1):2468.
  104. S.E. Toulmin. 2003. The Uses of Argument. Cambridge University Press.
  105. Matthias Trénel. 2009. Facilitation and inclusive deliberation. Online deliberation: Design, research, and practice, pages 253–257.
  106. Learning from disagreement: A survey. Journal of Artificial Intelligence Research, 72:1385–1470.
  107. Three types of incremental learning. Nature Machine Intelligence, 4(12):1185–1197.
  108. Annotator-centric active learning for subjective NLP tasks.
  109. HyEnA: A Hybrid Method for Extracting Arguments from Opinions. In Proceedings of the first International Conference on Hybrid Human-Artificial Intelligence (HHAI 2022), pages 1–15, Amsterdam, the Netherlands. IOS Press.
  110. A hybrid intelligence method for argument mining. Journal of AI Research (JAIR, to appear).
  111. Do differences in values influence disagreements in online discussions? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15986–16008, Singapore. Association for Computational Linguistics.
  112. An empirical analysis of diversity in argument summarization. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2028–2045, St. Julian’s, Malta. Association for Computational Linguistics.
  113. Argumentation. Reasonableness and effectiveness in argumentative discourse: Fifty contributions to the development of Pragma-dialectics, pages 3–25.
  114. GRaSP: A multilayered annotation scheme for perspectives. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1177–1184, Portorož, Slovenia. European Language Resources Association (ELRA).
  115. Towards argument mining for social good: A survey. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1338–1352, Online. Association for Computational Linguistics.
  116. How far can camels go? exploring the state of instruction tuning on open resources. In Advances in Neural Information Processing Systems, volume 36, pages 74764–74786. Curran Associates, Inc.
  117. Putting humans in the natural language processing loop: A survey. In Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing, pages 47–52, Online. Association for Computational Linguistics.
  118. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7):5731–5780.
  119. Detecting minority arguments for mutual understanding: A moderation tool for the online climate change debate. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6715–6725, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  120. Emergent abilities of large language models. Transactions on Machine Learning Research. Survey Certification.
  121. What makes online communities ‘better’? measuring values, consensus, and conflict across thousands of subreddits. Proceedings of the International AAAI Conference on Web and Social Media, 16(1):1121–1132.
  122. Annotating expressions of opinions and emotions in language. Language resources and evaluation, 39:165–210.
  123. SentiRec: Sentiment diversity-aware neural news recommendation. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 44–53, Suzhou, China. Association for Computational Linguistics.
  124. Exploring antecedents and consequences of toxicity in online discussions: A case study on reddit. Proc. ACM Hum.-Comput. Interact., 4(CSCW2).
  125. Fei Xiong and Yun Liu. 2014. Opinion formation on social media: An empirical approach. Chaos: An Interdisciplinary Journal of Nonlinear Science, 24(1):013130.
  126. Humanly certifying superhuman classifiers. In The Eleventh International Conference on Learning Representations.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets