Papers
Topics
Authors
Recent
Search
2000 character limit reached

Agreeing and Disagreeing in Collaborative Knowledge Graph Construction: An Analysis of Wikidata

Published 20 Jun 2023 in cs.HC | (2306.11766v3)

Abstract: In this work, we study disagreements in discussions around Wikidata, an online knowledge community that builds the data backend of Wikipedia. Discussions are essential in collaborative work as they can increase contributor performance and encourage the emergence of shared norms and practices. While disagreements can play a productive role in discussions, they can also lead to conflicts and controversies, which impact contributor' well-being and their motivation to engage. We want to understand if and when such phenomena arise in Wikidata, using a mix of quantitative and qualitative analyses to identify the types of topics people disagree about, the most common patterns of interaction, and roles people play when arguing for or against an issue. We find that decisions to create Wikidata properties are much faster than those to delete properties and that more than half of controversial discussions do not lead to consensus. Our analysis suggests that Wikidata is an inclusive community, considering different opinions when making decisions, and that conflict and vandalism are rare in discussions. At the same time, while one-fourth of the editors participating in controversial discussions contribute legitimate and insightful opinions about Wikidata's emerging issues, they respond with one or two posts and do not remain engaged in the discussions to reach consensus. Our work contributes to the analysis of collaborative KG construction with insights about communication and decision-making in projects, as well as with methodological directions and open datasets. We hope our findings will help managers and designers support community decision-making and improve discussion tools and practices.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (85)
  1. Measuring user relevance in online debates through an argumentative model. Pattern Recognition Letters 133 (2020), 41–47.
  2. Information quality in Wikipedia: The effects of group composition and task conflict. Journal of management information systems 27, 4 (2011), 71–98.
  3. Stay on the Wikipedia task: When task-related disagreements slip into personal and procedural conflicts. Journal of the American Society for Information Science and Technology 64, 8 (2013), 1634–1648.
  4. Managing Conflict in Online Debate Communities: Foregrounding Moderators’ Beliefs and Values on Kialo. (2018).
  5. Peer production: A form of collective intelligence. Handbook of collective intelligence 175 (2015).
  6. Detecting influencers in written online conversations. (2012).
  7. Societal controversies in Wikipedia articles. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 193–196.
  8. Network analysis of collaboration structure in Wikipedia. In Proceedings of the 18th international conference on World wide web. 731–740.
  9. Fine-grained controversy detection in Wikipedia. In 2015 IEEE 31st International Conference on Data Engineering. IEEE, 1573–1584.
  10. Echoes of power: Language effects and power differences in social interaction. In Proceedings of the 21st international conference on World Wide Web. 699–708.
  11. Carsten KW De Dreu and Laurie R Weingart. 2003. Task Versus Relationship Conflict, Team Performance, and Team Member Satisfaction: A Meta-Analysis. Journal of Applied Psychology 88, 4 (2003), 741–749.
  12. Christine De Kock and Andreas Vlachos. 2021. I Beg to Differ: A study of constructive disagreement in online conversations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2017–2027.
  13. Esin Durmus and Claire Cardie. 2019. A corpus for modeling user and language effects in argumentation on online debating. arXiv preprint arXiv:1906.11310 (2019).
  14. Steve M Easterbrook. 1993. Cooperation or Conflict?
  15. Lisa Ehrlinger and Wolfram Wöß. 2016. Towards a Definition of Knowledge Graphs. SEMANTiCS (Posters, Demos, SuCCESS) 48 (2016), 1–4.
  16. M Elliott and Walt Scacchi. 2002. Communicating and mitigating conflict in open source software development projects. Projects & Profits (2002), 25–41.
  17. Knowledge collaboration in online communities. Organization science 22, 5 (2011), 1224–1239.
  18. Anna Filippova and Hichang Cho. 2015. Mudslinging and manners: Unpacking conflict in free and open source software. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 1393–1403.
  19. Anna Filippova and Hichang Cho. 2016. The effects and antecedents of conflict in free and open source software development. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. 705–716.
  20. Anatomy of a flame: conflict and community building on the Internet. IEEE Technology and Society Magazine 14, 2 (1995), 12–21.
  21. The role of argumentation in online epistemic communities: the anatomy of a conflict in Wikipedia. In Proceedings of the 28th Annual European conference on cognitive ergonomics. 91–98.
  22. Quantifying controversy on social media. ACM Transactions on Social Computing 1, 1 (2018), 1–27.
  23. Statistical analysis of the social network and discussion threads in slashdot. In Proceedings of the 17th international conference on World Wide Web. 645–654.
  24. Alejandro Gonzalez-Hevia and Daniel Gayo-Avello. 2022. Leveraging Wikidata’s edit history in knowledge graph refinement tasks. arXiv preprint arXiv:2210.15495 (2022).
  25. Ivan Habernal and Iryna Gurevych. 2016. What makes a convincing argument? empirical analysis and detecting attributes of convincingness in web argumentation. In Proceedings of the 2016 conference on empirical methods in natural language processing. 1214–1223.
  26. Noriko Hara and Madelyn Rose Sanfilippo. 2016. Co-constructing controversy: Content analysis of collaborative knowledge negotiation in online communities. Information, Communication & Society 19, 11 (2016), 1587–1604.
  27. Noriko Hara and Madelyn Rose Sanfilippo. 2017. Analysis of roles in engaging contentious online discussions in science. Journal of the Association for Information Science and Technology 68, 8 (2017), 1953–1966.
  28. Cross-cultural analysis of the Wikipedia community. Journal of the American Society for Information Science and Technology 61, 10 (2010), 2097–2108.
  29. Detecting subgroups in online discussions by modeling positive and negative relations among participants. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. 59–70.
  30. Sarada Herke and Christina M Mynhardt. 2009. Radial trees. Discrete mathematics 309, 20 (2009), 5950–5962.
  31. Jorge E Hirsch. 2005. An index to quantify an individual’s scientific research output. Proceedings of the National academy of Sciences 102, 46 (2005), 16569–16572.
  32. French Wikipedia talk pages: Profiling and conflict detection. In 4th Conference on CMC and Social Media Corpora for the Humanities.
  33. Knowledge graphs. Synthesis Lectures on Data, Semantics, and Knowledge 12, 2 (2021), 1–257.
  34. Glenn D Israel. 1992. Determining sample size. Technical Report. University of Florida, Cooperative Extension Service, Institute of Food and Agricultural Sciences.
  35. A corpus of participant roles in contentious discussions. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14). 1751–1756.
  36. Predicting controversy of Wikipedia articles using the article feedback tool. In Proceedings of the 2014 international conference on social computing. 1–7.
  37. Controversial information spreads faster and further than non-controversial information in Reddit. Journal of Computational Social Science 5, 1 (2022), 111–122.
  38. Karen A Jehn. 1995. A multimethod examination of the benefits and detriments of intragroup conflict. Administrative science quarterly (1995), 256–282.
  39. Emergent life cycle: The tension between knowledge change and knowledge retention in open online coproduction communities. Management Science 60, 12 (2014), 3026–3048.
  40. Aniket Kittur and Robert E Kraut. 2010. Beyond Wikipedia: coordination and conflict in online production groups. In Proceedings of the 2010 ACM conference on Computer supported cooperative work. 215–224.
  41. He says, she says: conflict and coordination in Wikipedia. In Proceedings of the SIGCHI conference on Human factors in computing systems. 453–462.
  42. Emotions and dialogue in a peer-production community: the case of Wikipedia. In proceedings of the eighth annual international symposium on wikis and open collaboration. 1–10.
  43. When the Wikipedians talk: Network and tree structure of Wikipedia discussion pages. In Fifth international AAAI conference on weblogs and social media.
  44. Jure Leskovec and Eric Horvitz. 2008. Planetary-scale views on a large instant-messaging network. In Proceedings of the 17th international conference on World Wide Web. 915–924.
  45. Marco Lippi and Paolo Torroni. 2016. Argumentation mining: State of the art and emerging trends. ACM Transactions on Internet Technology (TOIT) 16, 2 (2016), 1–25.
  46. David F Marks and Lucy Yardley. 2004. Research methods for clinical and health psychology. Sage.
  47. Ernan McMullin. 1987. Scientific controversy and its termination. Scientific Controversies, Cambridge University Press, Cambridge 1987 (1987), 49–91.
  48. Peer-production system or collaborative ontology engineering effort: What is Wikidata?. In Proceedings of the 11th International Symposium on Open Collaboration. 1–10.
  49. Martin C Nwadiugwu and Cynthia C Nwadiugwu. 2021. Influencing Followership: Understanding the Perspective of Those Leading Active Discussions on Quora. Frontiers in Computer Science 3 (2021), 582242.
  50. Egon S Pearson. 1931. The test of significance for the correlation coefficient. J. Amer. Statist. Assoc. 26, 174 (1931), 128–134.
  51. DILIGENT: Towards a fine-grained methodology for DIstributed, Loosely-controlled and evolvInG Engineering of oNTologies. In ECAI, Vol. 16. 393.
  52. Wikidatians Are Born: Paths to Full Participation in a Collaborative Structured Knowledge Base. In 50th Hawaii International Conference on System Sciences. University of Hawaii, 4354–4363.
  53. Alessandro Piscopo and Elena Simperl. 2018. Who Models the World? Collaborative Ontology Creation and User Roles in Wikidata. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–18.
  54. Hoda Sepehri Rad and Denilson Barbosa. 2012. Identifying controversial articles in Wikipedia: A comparative study. In Proceedings of the eighth annual international symposium on wikis and open collaboration. 1–10.
  55. Discovering, explaining and summarizing controversial discussions in community q&a sites. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 151–162.
  56. Jodi Schneider. 2014. Automated argumentation mining to the rescue? Envisioning argumentation and decision-making support for debates in open online collaboration communities. In Proceedings of the First Workshop on Argumentation Mining.
  57. A content analysis: How Wikipedia talk pages are used. In The Web Science Conference 2010 (WebSci ’10). Raleigh, North Carolina, USA.
  58. Arguments about deletion: How experience improves the acceptability of arguments in ad-hoc online task groups. In Proceedings of the 2013 conference on Computer supported cooperative work. 1069–1080.
  59. Why did they post that argument? Communicative Intentions of Web 2.0 Arguments. Arguing on the Web 2 (2014).
  60. Hoda Sepehri Rad and Denilson Barbosa. 2011. Towards identifying arguments in Wikipedia pages. In Proceedings of the 20th international conference companion on World wide web. 117–118.
  61. Leveraging editor collaboration patterns in Wikipedia. In Proceedings of the 23rd ACM conference on Hypertext and social media. 13–22.
  62. Elena Simperl and Markus Luczak-Rösch. 2014. Collaborative ontology engineering: a survey. The Knowledge Engineering Review 29, 1 (2014), 101–131.
  63. Argumentation structures that integrate dialectical and non-dialectical reasoning. The Knowledge Engineering Review 16, 4 (2001), 331–348.
  64. Knowledge engineering: Principles and methods. Data & knowledge engineering 25, 1-2 (1998), 161–197.
  65. Us vs. them: Understanding social dynamics in Wikipedia with revert graph visualizations. In 2007 IEEE symposium on visual analytics science and technology. IEEE, 163–170.
  66. Edit wars in Wikipedia. In 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing. IEEE, 724–727.
  67. Diego Sáez Trumper. 2021. Research:Identifying Controversial Content in Wikidata. https://meta.wikimedia.org/wiki/Research:Identifying_Controversial_Content_in_Wikidata. [Online; accessed 10-January-2023].
  68. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th international conference on world wide web. 613–624.
  69. An argumentation ontology for distributed, loosely-controlled and evolving engineering processes of ontologies (DILIGENT). In The Semantic Web: Research and Applications: Second European Semantic Web Conference, ESWC 2005, Heraklion, Crete, Greece, May 29–June 1, 2005. Proceedings 2. Springer, 241–256.
  70. Argumentation-based ontology engineering. IEEE Intelligent Systems 22, 6 (2007), 52–59.
  71. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & health sciences 15, 3 (2013), 398–405.
  72. Linked Open Vocabularies (LOV): a gateway to reusable semantic vocabularies on the Web. Semantic Web 8, 3 (2017), 437–452.
  73. Studying cooperation and conflict between authors with history flow visualizations. In Proceedings of the SIGCHI conference on Human factors in computing systems. 575–582.
  74. Talk before you type: Coordination in Wikipedia. In 2007 40th Annual Hawaii International Conference on System Sciences (HICSS’07). IEEE, 78–78.
  75. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: A Free Collaborative Knowledge Base. Commun. ACM 57, 10 (2014), 78–85.
  76. The DILIGENT knowledge processes. Journal of Knowledge Management 9, 5 (2005), 85–96.
  77. On ranking controversies in wikipedia: models and evaluation. In Proceedings of the 2008 international conference on Web search and data mining. 171–182.
  78. Christian Wagner. 2004. Wiki: A technology for conversational knowledge management and group collaboration. Communications of the association for information systems 13, 1 (2004), 19.
  79. Argumentation schemes. Cambridge University Press.
  80. Lu Wang and Claire Cardie. 2016. A piece of my mind: A sentiment analysis approach for online dispute detection. arXiv preprint arXiv:1606.05704 (2016).
  81. Argulens: Anatomy of community opinions on usability issues using argumentation models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.
  82. Is this post persuasive? Ranking argumentative comments in online forum. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 195–200.
  83. The most controversial topics in Wikipedia. Global Wikipedia: International and cross-cultural issues in online collaboration 25 (2014), 25–48.
  84. Characterizing online discussion using coarse discourse sequences. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 11. 357–366.
  85. Computing controversy: Formal model and algorithms for detecting controversy on Wikipedia and in search queries. Information Processing & Management 54, 1 (2018), 14–36.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up