Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Help Me Reflect: Leveraging Self-Reflection Interface Nudges to Enhance Deliberativeness on Online Deliberation Platforms (2401.10820v1)

Published 19 Jan 2024 in cs.HC

Abstract: The deliberative potential of online platforms has been widely examined. However, little is known about how various interface-based reflection nudges impact the quality of deliberation. This paper presents two user studies with 12 and 120 participants, respectively, to investigate the impacts of different reflective nudges on the quality of deliberation. In the first study, we examined five distinct reflective nudges: persona, temporal prompts, analogies and metaphors, cultural prompts and storytelling. Persona, temporal prompts, and storytelling emerged as the preferred nudges for implementation on online deliberation platforms. In the second study, we assess the impacts of these preferred reflectors more thoroughly. Results revealed a significant positive impact of these reflectors on deliberative quality. Specifically, persona promotes a deliberative environment for balanced and opinionated viewpoints while temporal prompts promote more individualised viewpoints. Our findings suggest that the choice of reflectors can significantly influence the dynamics and shape the nature of online discussions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (141)
  1. 1989. Oxford English Dictionary. Oxford University Press. Deliberation.
  2. 1989. Webster’s Encyclopedic Unabridged Dictionary of the English Language. New York: Portland House.
  3. Matthew Adams. 2003. The reflexive self and culture: a critique. The British journal of sociology 54, 2 (2003), 221–238.
  4. Embodying, enacting and entangling design: A phenomenological view to co-designing services. Swedish Design Research Journal 1, 1 (2013), 29–41.
  5. Lucas Anastasiou and Anna De Libbo. 2023. BCause: Reducing group bias and promoting cohesive discussion in online deliberation processes through a simple and engaging online deliberation tool. In Proceedings of the First Workshop on Social Influence in Conversations (SICon 2023). 39–49.
  6. Are all biases bad? Collaborative grounded theory in developmental evaluation of education policy. Journal of Multidisciplinary Evaluation 12, 27 (2016), 44–57.
  7. Storytelling as an instructional method: Descriptions and research questions. (2009).
  8. Kevin Arceneaux and Ryan J Vander Wielen. 2017. Taming intuition: How reflection minimizes partisan reasoning and promotes democratic accountability. Cambridge University Press.
  9. Jonathan Barnes Aristotle et al. 1984. The complete works of Aristotle. Vol. 2. Princeton University Press Princeton, NJ.
  10. Social Media Technology and Government Transparency. Computer 43, 11 (2010), 53–59. https://doi.org/10.1109/MC.2010.325
  11. Diane M Billings. 2016. Storytelling: A strategy for providing context for learning. The Journal of Continuing Education in Nursing 47, 3 (2016), 109–110.
  12. Self-governance through group discussion in Wikipedia: Measuring deliberation in online groups. Small Group Research 42, 5 (2011), 595–634.
  13. Stefan Blomkvist. 2002. Persona–an overview. Retrieved November 22 (2002), 2004.
  14. James Bohman. 2000. Public deliberation: Pluralism, complexity, and democracy. MIT press.
  15. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021).
  16. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  17. The impact of multiple parallel phrase suggestions on email input and composition behaviour of native and non-native english writers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
  18. Kaya Cahit. 2015. Internal validity: A must in research designs. Educational Research and Reviews 10, 2 (2015), 111–118.
  19. Kelly Caine. 2016. Local standards for sample size at CHI. In Proceedings of the 2016 CHI conference on human factors in computing systems. 981–992.
  20. Donald T Campbell and Julian C Stanley. 2015. Experimental and quasi-experimental designs for research. Ravenio books.
  21. Argument repertoire as a reliable and valid measure of opinion quality: Electronic dialogue during campaign 2000. Political Communication 19, 1 (2002), 73–93.
  22. Simone Chambers. 2003. Deliberative democratic theory. Annual review of political science 6, 1 (2003), 307–326.
  23. Personas: from theory to practices. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges. 439–442.
  24. Thomas Christiano and John Christman. 2009. Debates in Political Philosophy. Wiley Online Library.
  25. The interplay between culturally responsive teaching, cultural diversity beliefs, and self-reflection: A multiple case study. Teaching and Teacher Education 77 (2019), 341–351.
  26. Creative writing with a machine in the loop: Case studies on slogans and stories. In 23rd International Conference on Intelligent User Interfaces. 329–340.
  27. Quasi-experimentation: Design & analysis issues for field settings. Vol. 351. Houghton Mifflin Boston.
  28. Alan Cooper. 1999. The inmates are running the asylum. Indianapolis, IA: SAMS. Macmillan (1999).
  29. About face 3: the essentials of interaction design. John Wiley & Sons.
  30. Institutional aesthetics: Embodied ways of encountering, evaluating, and enacting institutions. Organization Studies 41, 3 (2020), 415–435.
  31. Justcorrect: Intelligent post hoc text correction techniques on smartphones. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 487–499.
  32. Beyond text generation: Supporting writers with continuous automatic text summaries. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–13.
  33. Empowering the collective reflection-based argumentation mapping strategy to enhance students’ argumentative speaking. Computers & Education 184 (2022), 104516.
  34. Todd Davies and Reid Chandler. 2013. Online deliberation design: Choices, criteria, and evidence. arXiv preprint arXiv:1302.5177 (2013).
  35. Patrick Dawson. 2014. Reflections: On time, temporality and change in organizations. Journal of Change Management 14, 3 (2014), 285–308.
  36. Robert Dawson. 2011. How significant is a boxplot outlier? Journal of Statistics Education 19, 2 (2011).
  37. Rianne Dekker and Victor Bekkers. 2015. The contingency of governments’ responsiveness to the virtual public sphere: A systematic literature review and meta-synthesis. Government Information Quarterly 32, 4 (2015), 496–505.
  38. Melissa Desjarlais and Peter Smith. 2011. A comparative analysis of reflection and self-assessment. International Journal of Process Education 3, 1 (2011), 3–18.
  39. Pierpaolo Donati. 2011. Modernization and relational reflexivity. International Review of Sociology 21, 1 (2011), 21–39.
  40. John S Dryzek. 2002. Deliberative democracy and beyond: Liberals, critics, contestations. Oxford University Press, USA.
  41. John S Dryzek. 2009. Democratization as deliberative capacity building. Comparative political studies 42, 11 (2009), 1379–1402.
  42. Janet E Dyment and Timothy S O’Connell. 2011. Assessing the quality of reflection in student journals: A review of the research. Teaching in Higher Education 16, 1 (2011), 81–97.
  43. Public deliberation in an era of communicative plenty. Policy & politics 47, 1 (2019), 19–35.
  44. Designing an online civic engagement platform: Balancing “more” vs.“better” participation in complex public policymaking. International Journal of E-Politics (IJEP) 5, 1 (2014), 16–40.
  45. Barbara Glesner Fines. 2014. Assessing reflection. Retrieved on December 12 (2014), 2021.
  46. Brian J Fogg. 2002. Persuasive technology: using computers to change what we think and do. Ubiquity 2002, December (2002), 2.
  47. Natasha Freidus and Michelle Hlubinka. 2002. Digital storytelling for reflective practice in communities of learners. ACM SIGGROUP Bulletin 23, 2 (2002), 24–26.
  48. CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysis. ACM Transactions on Computer-Human Interaction (2023).
  49. John Gastil and Laura Black. 2007. Public deliberation as the organizing principle of political communication research. Journal of Public Deliberation 4, 1 (2007).
  50. Dedre Gentner. 1982. Are scientific analogies metaphors. Metaphor: Problems and perspectives 7, 7 (1982).
  51. Katy Ilonka Gero and Lydia B Chilton. 2019. Metaphoria: An algorithmic companion for metaphor creation. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–12.
  52. Alex Goddard and Alex Gillespie. 2023. Textual Indicators of Deliberative Dialogue: A Systematic Review of Methods for Studying the Quality of Online Dialogues. Social Science Computer Review (2023), 08944393231156629.
  53. Robert E Goodin. 2000. Democratic deliberation within. Philosophy & Public Affairs 29, 1 (2000), 81–109.
  54. Robert E Goodin. 2003. Reflective democracy. OUP Oxford.
  55. Robert E Goodin and Simon J Niemeyer. 2003. When does deliberation begin? Internal reflection versus public discussion in deliberative democracy. Political Studies 51, 4 (2003), 627–649.
  56. Todd Graham and Tamara Witschge. 2003. In search of online deliberation: Towards a new method for examining the quality of online discussions. (2003).
  57. Anthony M Grant. 2001. Rethinking psychological mindedness: Metacognition, self-reflection, and insight. Behaviour Change 18, 1 (2001), 8–17.
  58. Anthony M Grant. 2003. The impact of life coaching on goal attainment, metacognition and mental health. Social Behavior and Personality: an international journal 31, 3 (2003), 253–263.
  59. The self-reflection and insight scale: A new measure of private self-consciousness. Social Behavior and Personality: an international journal 30, 8 (2002), 821–835.
  60. Jürgen Habermas. 1984. The theory of communicative action. Volume two: Lifeworld and system. Trans. Thomas McCarthy. Boston: Beacon Press (1984).
  61. Digital storytelling as a tool for fostering reflection. Frontiers: The Interdisciplinary Journal of Study Abroad 31, 1 (2019), 59–73.
  62. AI-mediated communication: Definition, research agenda, and ethical considerations. Journal of Computer-Mediated Communication 25, 1 (2020), 89–100.
  63. Markus Holdo. 2020. Meta-deliberation: Everyday acts of critical reflection in deliberative systems. Politics 40, 1 (2020), 106–119.
  64. Linguistic appropriateness and pedagogic usefulness of reading comprehension questions. In Proceedings of the Twelfth Language Resources and Evaluation Conference. 1753–1762.
  65. Identifying the social signals that drive online discussions: A case study of reddit communities. In 2017 26th International Conference on Computer Communication and Networks (ICCCN). IEEE, 1–9.
  66. Reducing sentiment bias in language models via counterfactual evaluation. arXiv preprint arXiv:1911.03064 (2019).
  67. Arbinger Institute et al. 2016. The outward mindset: Seeing beyond ourselves. Berrett-Koehler Publishers.
  68. Talking together: Public deliberation and political participation in America. University of Chicago Press.
  69. Co-writing with opinionated language models affects users’ views. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
  70. Human heuristics for AI-generated language are flawed. Proceedings of the National Academy of Sciences 120, 11 (2023), e2208839120.
  71. The Ghost in the Machine has an American accent: value conflict in GPT-3. arXiv preprint arXiv:2203.07785 (2022).
  72. Lucas A Keefer and Mark J Landau. 2016. Metaphor and analogy in everyday problem solving. Wiley Interdisciplinary Reviews: Cognitive Science 7, 6 (2016), 394–405.
  73. Ronald T Kellogg. 1999. The psychology of writing. Oxford University Press.
  74. StarryThoughts: Facilitating Diverse Opinion Exploration on Social Issues. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–29.
  75. Crowdsourcing perspectives on public policy from stakeholders. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.
  76. Factful: Engaging taxpayers in the public discussion of a government budget. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2843–2852.
  77. Alison King. 1994. Inquiry as a tool in critical thinking. Changing college classrooms: New teaching and learning strategies in an increasing complex world (1994).
  78. Mark Klein. 2011. How to harvest collective wisdom on complex problems: An introduction to the mit deliberatorium. Center for Collective Intelligence working paper (2011).
  79. Svetlana Koltovskaia. 2020. Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing 44 (2020), 100450.
  80. Factors confounding the assessment of reflection: a critical review. BMC medical education 11, 1 (2011), 1–9.
  81. Supporting reflective public thought with considerit. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. 265–274.
  82. Will Kymlicka. 2002. Contemporary political philosophy: An introduction. oxford: oxford University Press.
  83. Nancy Lampert. 2006. Enhancing critical thinking with aesthetic, critical, and creative inquiry. Art Education 59, 5 (2006), 46–50.
  84. Hee Seung Lee and Keith J Holyoak. 2007. Causal models guide analogical inference. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 29.
  85. Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1–19.
  86. Thomas C Leonard. 2008. Richard H. Thaler, Cass R. Sunstein, Nudge: Improving decisions about health, wealth, and happiness: Yale University Press, New Haven, CT, 2008, 293 pp, 26.00.
  87. The persona effect: affective impact of animated pedagogical agents. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. 359–366.
  88. Affective diary: designing for bodily expressiveness and self-reflection. In CHI’06 extended abstracts on Human factors in computing systems. 1037–1042.
  89. Li Lucy and David Bamman. 2021. Gender and representation bias in GPT-3 generated stories. In Proceedings of the Third Workshop on Narrative Understanding. 48–55.
  90. Stephen Macedo. 1999. Deliberative politics: Essays on democracy and disagreement. Oxford University Press.
  91. Kate McDowell. 2021. Storytelling wisdom: Story, information, and DIKW. Journal of the Association for Information Science and Technology 72, 10 (2021), 1223–1233.
  92. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica 22, 3 (2012), 276–282.
  93. The anatomy of Reddit: An overview of academic research. Dynamics On and Of Complex Networks III: Machine Learning and Statistical Physics Approaches 10 (2019), 183–204.
  94. Nudge for deliberativeness: How interface features influence online discourse. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
  95. Assessing the quality of student-generated short answer questions using GPT-3. In European conference on technology enhanced learning. Springer, 243–257.
  96. Alain Morin and James Everett. 1990. Inner speech as a mediator of self-awareness, self-consciousness, and self-knowledge: An hypothesis. New ideas in psychology 8, 3 (1990), 337–356.
  97. Nicos Mouzelis. 2010. Self and self—other reflexivity: The apophatic dimension. European Journal of Social Theory 13, 2 (2010), 271–284.
  98. Lala Muradova. 2021. Seeing the other side? Perspective-taking and reflective political judgements in interpersonal deliberation. Political Studies 69, 3 (2021), 644–664.
  99. Kyoko Murakami and Rachel L Jacobs. 2017. Connecting dots: Family reminiscence. (2017).
  100. Shinichi Nakagawa. 2004. A farewell to Bonferroni: the problems of low statistical power and publication bias. Behavioral ecology 15, 6 (2004), 1044–1045.
  101. Vlad Niculae and Cristian Danescu-Niculescu-Mizil. 2016. Conversational markers of constructive discussions. arXiv preprint arXiv:1604.07407 (2016).
  102. Understanding needs embodiment: A theory-guided reanalysis of the role of metaphors and analogies in understanding science. Science Education 96, 5 (2012), 849–877.
  103. HONEST: Measuring hurtful sentence completion in language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics.
  104. Pauline Phemister. 2017. Leibniz’s mirrors: Reflecting the past. In X Internationaler Leibniz Kongress. Georg Olms Verlag, 93–108.
  105. Does disagreement contribute to more deliberative opinion? Political communication 19, 1 (2002), 95–112.
  106. John Pruitt and Tamara Adlin. 2010. The persona lifecycle: keeping people in mind throughout product design. Elsevier.
  107. K Andrew R Richards and Michael A Hemphill. 2018. A practical guide to collaborative qualitative data analysis. Journal of Teaching in Physical education 37, 2 (2018), 225–231.
  108. Clare Rigg. 2018. Somatic learning: Bringing the body into critical reflection. Management Learning 49, 2 (2018), 150–167.
  109. Civic technology for social innovation: A systematic literature review. Computer Supported Cooperative Work (CSCW) 28 (2019), 169–207.
  110. A simple more general boxplot method for identifying outliers. Computational statistics & data analysis 47, 1 (2004), 165–174.
  111. Jasna K Schwind. 2009. Metaphor-reflection in my healthcare experience. Aporia 1, 1 (2009).
  112. Olivier Serrat. 2008. Storytelling. (2008).
  113. Paul J Silvia. 2022. The self-reflection and insight scale: Applying item response theory to craft an efficient short form. Current Psychology 41, 12 (2022), 8635–8645.
  114. Where to hide a stolen elephant: Leaps in creative writing with multimodal machine intelligence. ACM Transactions on Computer-Human Interaction (2022).
  115. Marion K Slack and Jolaine R Draugalis Jr. 2001. Establishing the internal and external validity of experimental studies. American journal of health-system pharmacy 58, 22 (2001), 2173–2181.
  116. Self-reflection and the temporal focus of the wandering mind. Consciousness and cognition 20, 4 (2011), 1120–1126.
  117. Measuring political deliberation: A discourse quality index. Comparative European Politics 1 (2003), 21–48.
  118. On the linguistic and pedagogical quality of automatic question generation via neural machine translation. In Technology-Enhanced Learning for a Free, Safe, and Sustainable World: 16th European Conference on Technology Enhanced Learning, EC-TEL 2021, Bolzano, Italy, September 20-24, 2021, Proceedings 16. Springer, 289–294.
  119. Jennifer Stromer-Galley. 2007. Measuring deliberation’s content: A coding scheme. Journal of Deliberative Democracy 3, 1 (2007).
  120. John L Sullivan and John E Transue. 1999. The psychological underpinnings of democracy: A selective review of research on political tolerance, interpersonal trust, and social capital. Annual review of psychology 50, 1 (1999), 625–650.
  121. Richard Thaler and Cass Sunstein. 2008. Nudge: The gentle power of choice architecture. New Haven, Conn.: Yale (2008).
  122. Quick and easy implementation of the Benjamini-Hochberg procedure for controlling the false positive rate in multiple comparisons. Journal of educational and behavioral statistics 27, 1 (2002), 77–83.
  123. Hobbes Thomas. 1946. Leviathan. London: Basil Blackwell.
  124. Mathias Trénel. 2004. Measuring the deliberativeness of online discussions. Coding scheme 2.4. report, Berlin: Social Science Research Centrex (2004).
  125. Thomas Daniel Ullmann. 2011. An architecture for the automated detection of textual indicators of reflection. (2011).
  126. Understanding interobserver agreement: the kappa statistic. Fam med 37, 5 (2005), 360–363.
  127. Josina Vink and Kaisa Koskela-Huotari. 2022. Building reflexivity using service design methods. Journal of Service Research 25, 3 (2022), 371–389.
  128. Douglas Walton. 1999. One-sided arguments: A dialectical analysis of bias. SUNY Press.
  129. A field trial of privacy nudges for facebook. In Proceedings of the SIGCHI conference on human factors in computing systems. 2367–2376.
  130. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023).
  131. Claes Wohlin. 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering. 1–10.
  132. Assessing the level of student reflection from reflective journals. Journal of advanced nursing 22, 1 (1995), 48–57.
  133. Design for online deliberative processes and technologies: Towards a multidisciplinary research agenda. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. 865–868.
  134. AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing. In Joint Proceedings of the ACM IUI Workshops, Vol. 10. CEUR-WS Team.
  135. Effects of knowledge and reflection in intrapersonal deliberation. Journal of Deliberative Democracy 17, 1 (2021).
  136. Wordcraft: story writing with large language models. In 27th International Conference on Intelligent User Interfaces. 841–852.
  137. Type, then correct: Intelligent text correction techniques for mobile text entry using neural networks. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 843–855.
  138. Weiyu Zhang. 2005. Are online discussions deliberate. A Case Study of a Chinese Online Discussion Board. Vortrag auf der III. Interna (2005).
  139. The structural features and the deliberative quality of online discussions. Telematics and informatics 30, 2 (2013), 74–86.
  140. Weiyu Zhang and Leanne Chang. 2014. Perceived speech conditions and disagreement of everyday talk: A proceduralist perspective of citizen deliberation. Communication Theory 24, 2 (2014), 124–145.
  141. Nudge for reflection: more than just a channel to political knowledge. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–10.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets