Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Value-Sensitive Conversational Agent Co-Design Framework (2310.11848v1)

Published 18 Oct 2023 in cs.HC and cs.AI

Abstract: Conversational agents (CAs) are gaining traction in both industry and academia, especially with the advent of generative AI and LLMs. As these agents are used more broadly by members of the general public and take on a number of critical use cases and social roles, it becomes important to consider the values embedded in these systems. This consideration includes answering questions such as 'whose values get embedded in these agents?' and 'how do those values manifest in the agents being designed?' Accordingly, the aim of this paper is to present the Value-Sensitive Conversational Agent (VSCA) Framework for enabling the collaborative design (co-design) of value-sensitive CAs with relevant stakeholders. Firstly, requirements for co-designing value-sensitive CAs which were identified in previous works are summarised here. Secondly, the practical framework is presented and discussed, including its operationalisation into a design toolkit. The framework facilitates the co-design of three artefacts that elicit stakeholder values and have a technical utility to CA teams to guide CA implementation, enabling the creation of value-embodied CA prototypes. Finally, an evaluation protocol for the framework is proposed where the effects of the framework and toolkit are explored in a design workshop setting to evaluate both the process followed and the outcomes produced.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (120)
  1. A. Abaza. 2021. MLOps: Why is it the Most Important Technology in the Age of AI? https://www.synapse-analytics.io/post/mlops-why-is-it-the-most-important-technology-in-the-age-of-ai
  2. Co-Designing for Privacy, Transparency, and Trust in K-12 Learning Analytics. Proceedings of the International Learning Analytics and Knowledge Conference (LAK). https://doi.org/10.1145/3448139.3448145
  3. Collection of Metaphors for Human-Robot Interaction. In Proceedings of the Designing Interactive Systems Conference (DIS).
  4. PA. Arrighi and C. Mougenot. 2019. Towards user empowerment in product design: a mixed reality tool for interactive virtual prototyping. J Intell Manuf 30 (2019), 743–754.
  5. Social Robot Co-Design Canvases: A Participatory Design Framework. J. Hum.-Robot Interact. 11, 1, Article 3 (oct 2021), 39 pages. https://doi.org/10.1145/3472225
  6. What to Discuss? — A Blueprint Topic Model for Health Coaching Dialogues With Conversational Agents. International Journal of Human–Computer Interaction (2022).
  7. Informing Public Deliberation: Value Sensitive Design of Indicators for a Large-Scale Urban Simulation. ECSCW 2005.
  8. J.F. Boujut and E. Blanco. 2003. Intermediary Objects as a Means to Foster Co-operation in Engineering Design. Computer Supported Cooperative Work (CSCW) 12 (2003), 205–219.
  9. L. Byrd. 2020. Tech Inclusion: How culture may enhance conversational AI technology. https://uxmag.com/articles/tech-inclusion-how-culture-may-enhance-conversational-ai-technology
  10. F. Cabitza and C. Simone. 2012. Affording Mechanisms: An Integrated View of Coordination and Knowledge Management. Computer Supported Cooperative Work (CSCW) 21 (2012), 227–260.
  11. R. Calvo and D. Peters. 2017. Positive Computing. MIT Press.
  12. Co-Designing a Conversational Interactive Exhibit for Children. Proceedings of the ACM Interaction Design and Children Conference: Extended Abstracts.
  13. Next Steps for Human-Centered Generative AI: A Technical Perspective. Computing Research Repository (CoRR) (2023). https://arxiv.org/abs/2306.15774
  14. Creating a Chatbot for and with Migrants: Chatbot Personality Drives Co-Design Activities. In Proceedings of the ACM Designing Interactive Systems Conference.
  15. M. Cherubini. 2017. Ethical Autonomous Algorithms. https://medium.com/@mchrbn/ethical-autonomous-algorithms-5ad07c311bcc
  16. B. Christian. 2020. The Alignment Problem – Machine Learning and Human Values. W. W. Norton & Company.
  17. Gilbert Cockton. 2005. A Development Framework for Value-Centred Design. CHI Extended Abstracts on Human Factors in Computing Systems.
  18. Gilbert Cockton. 2006. Designing Worth is Worth Designing. Proceedings of the Nordic Conference on Human-Computer Interaction.
  19. K. Crawford and R. Calo. 2016. There is a blind spot in AI research. Nature 538 (2016), 311–313.
  20. C. Custis. 2021. Operationalizing AI Ethics through Documentation: ABOUT ML in 2020 and Beyond. https://partnershiponai.org/about-ml-2021/
  21. Parenting from the Pocket: Value Tensions and Technical Directions for Secure and Private Parent-Teen Mobile Safety. Proceedings of the Sixth Symposium on Usable Privacy and Security.
  22. Values As lived experience: evolving value sensitive design in support of value discovery. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
  23. Patients, Pacemakers, and Implantable Defibrillators: Human Values and Security for Wireless Implantable Medical Devices. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
  24. Deloitte Digital. 2019. Conversational AI: The next wave of customer and employee experience.
  25. DigitalCatapult. 2020. Lessons in Practical AI Ethics.
  26. C. DÍgnazio and L. Klein. 2020. Data Feminism. MIT Press.
  27. Engagements and articulations of ethics in design practice. International Journal of Design 16, 2 (2022), 47–56. https://doi.org/10.57698/v16i2.04
  28. A Virtual Agent to Support Individuals Living With Physical and Mental Comorbidities: Co-Design and Acceptability Testing. Journal of Medical Internet Research (JMIR) 21, 5 (2019).
  29. Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI. Proceedings of the ACM on Human-Computer Interaction 34 (2023).
  30. Assessing the Reusability of Design Principles in the Realm of Conversational Agents. Proceedings of the International Conference on Design Science Research in Information Systems and Technology (DESRIST).
  31. V. Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.
  32. Go FAIR. 2021. Data Based Science: FAIR Becomes the New Normal. https://www.go-fair.org/2021/01/21/data-based-science-fair-becomes-the-new-normal/
  33. Embodying values in technology: Theory and practice. Information technology and moral philosophy (2008), 322–346.
  34. Future directions for chatbot research: an interdisciplinary research agenda. Computing (2021), 2915––2942.
  35. SIG: Chatbots for Social Good. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems.
  36. Centre for AI Safety. 2023. Statement on AI Risk: AI experts and public figures express their concern about AI risk. https://www.safe.ai/statement-on-ai-risk
  37. N. Freier. 2008. Children Attribute Moral Standing to a Personified Agent. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
  38. Eight grand challenges for value sensitive design from the 2016 Lorentz workshop. Ethics and Information Technology 23 (2021), 5–16.
  39. B. Friedman and D. Hendry. 2019. Value Sensitive Design: Shaping Technology with Moral Imagination. MIT Press.
  40. The Watcher and the Watched: Social Judgments About Privacy in a Public Place. Human-Computer Interaction 21 (2006), 235–272.
  41. Values in Action (ViA): Combining Usability, User Experience and User Acceptance. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI).
  42. R. Garg and S. Sengupta. 2020. Conversational Technologies for In-home Learning: Using Co-Design to Understand Children’s and Parents’ Perspectives. Proceedings of the CHI Conference on Human Factors in Computing Systems.
  43. E. Görnemann and S. Spiekermann. 2022. Emotional responses to human values in technology: The case of conversational agents. Human–Computer Interaction (2022), 1–28.
  44. T. Hagendorff. 2020. The ethics of AI ethics: an evaluation of guidelines. Minds and Machines 30 (2020), 99–120.
  45. S. Hallensleben and C. Hustedt. 2020. From Principles to Practice; An interdisciplinary framework to operationalise AI ethics.
  46. M. Harbers and A. Overdiek. 2022. Towards a living lab for responsible applied AI. In Proceedings of the Design Research Society Conference (DRS).
  47. G. Hayes. 2020. Inclusive and Engaged HCI. Interactions 27, 2 (2020), 26–31. https://doi.org/10.1145/3378561
  48. J. Hedman and G. Gimpel. 2010. The adoption of hyped technologies: a qualitative study. Information and Technology Management 11, 4 (2010), 161–175.
  49. J. Heo and U. Lee. 2023. Form to Flow: Exploring Challenges and Roles of Conversational UX Designers in Real-world, Multi-channel Service Environments. Proceedings of the ACM on Human-Computer Interaction.
  50. J. Hockey. 2007. United Kingdom art and design practice PhDs: Evidence from students and their supervisors. Studies in Art Education 48, 2 (2007), 155–170.
  51. IEEE 7000-2021 2021. IEEE Standard Model Process for Addressing Ethical Concerns during System Design. Standard. The Institute of Electrical and Electronics Engineers.
  52. How Different Groups Prioritize Ethical Values for Responsible AI. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT).
  53. The global landscape of AI ethics guidelines. Natural Machine Intelligence 1 (2019), 389–399.
  54. W. Jonas. 2018. A Cybernetic Model of Design Research: Towards a trans-domain of knowing. Routledge.
  55. Collaborative Practices with Structured Data: Do Tools Support What Users Need? Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems.
  56. Human-centred mechanism design with Democratic AI. Natural Human Behaviour 6 (2022), 1398–1407.
  57. M.T. Kot and G. Leszczyński. 2020. The concept of intelligent agent in business interactions: is virtual assistant an actor or a boundary object? Journal of Business & Industrial Marketing 35, 7 (2020), 1155–1164.
  58. P. Kroes and I. Van de Poel. 2020. Design for Values and the Definition, Specification, and Operationalization of Values. Springer Reference.
  59. Designing Intelligent Assistant through User Participations. Proceedings of the Conference on Designing Interactive Systems.
  60. State-of-the-Art Analysis of Adopting AI-based Conversational Agents in Organizations: A Systematic Literature Review. Proceedings of Pacific Asia Conference on Information Systems (PACIS).
  61. Y. Lim and B. Kim. 2022. Value-drive design approach to envision speculative futures. In Proceedings of the Design Research Society Conference (DRS).
  62. Capturing Teens’ Voice in Designing Supportive Agents. Proceedings of the 4th Conference on Conversational User Interfaces. https://doi.org/10.1145/3543829.3543838
  63. M. Luria. 2018. Designing Robot Personality Based on Fictional Sidekick Characters. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction.
  64. Robotic Futures: Learning about Personally-Owned Agents through Performance. Proceedings of the ACM Designing Interactive Systems Conference (DIS). https://doi.org/10.1145/3357236.3395488
  65. N. Manders-Huits. 2011. What Values in Design? The Challenge of Incorporating Moral Values into Design. Science and Engineering Ethics 17, 2 (2011), 271––287.
  66. N. Manders-Huits and M. Zimmer. 2009. Values and pragmatic action: The challenges of introducing ethical intelligence in technical design communities. The International Review of Information Ethics 10 (2009), 37–44.
  67. Does ACM’s Code of Ethics Change Ethical Decision Making in Software Development?. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Lake Buena Vista, FL, USA) (ESEC/FSE 2018). Association for Computing Machinery, New York, NY, USA, 729–733. https://doi.org/10.1145/3236024.3264833
  68. Ethical Matrix Manual. Agricultural and Forest Meteorology. (2006).
  69. Value tensions in design: the value sensitive design, development, and appropriation of a corporation’s groupware system. Proceedings of the 2007 International ACM Conference on Supporting Group Work.
  70. R. Moore. 2018. A natural conversation framework for conversational UX design. Springer International Publishing, 181–204.
  71. R. Moore and R. Arar. 2019. Conversational UX Design: A Practitioner’s Guide to the Natural Conversation Framework. ACM Books.
  72. The machinery of natural conversation and the design of conversational machines. American Sociological Association Annual Meeting.
  73. Operationalising AI ethics: barriers, enablers and next steps. AI & Society (2021).
  74. What’s The Talk on VUI Guidelines? A Meta-Analysis of Guidelines for Voice User Interface Design. Proceedings of the ACM conference on Conversational User Interfaces (CUI).
  75. Improvement of the Industrial Design Process by the Creation and Usage of Intermediate Representations of Technology, ”TechCards”. Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces.
  76. S. Paavola and R. Miettinen. 2019. Dynamics of Design Collaboration: BIM Models as Intermediary Digital Objects. Computer Supported Cooperative Work (CSCW) 28 (2019), 1–23.
  77. A. Palmer and D. Schwan. 2023. More Process, Less Principles: The Ethics of Deploying AI and Robotics in Medicine. Cambridge Quarterly of Healthcare Ethics (2023), 1–14.
  78. Power Dynamics and Value Conflicts in Designing and Maintaining Socio-Technical Algorithmic Processes. 6 (2022).
  79. D. Pennington. 2010. The Dynamics of Material Artifacts in Collaborative Research Teams. Computer Supported Cooperative Work (CSCW) 19 (2010), 175–199.
  80. How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study. (2021).
  81. Elicitation of situated values: need for tools to help stakeholders and designers to reflect and communicate. Ethics and Information Technology 14 (2012), 285–303.
  82. I. Rahwan. 2018. Society-in-the-loop: programming the algorithmic social contract. Ethics and Information Technology 20, 1 (2018), 5–14.
  83. I. Redaelli and A. Carassa. 2018. New Perspectives on Plans: Studying Planning as an Instance of Instructed Action. Computer Supported Cooperative Work (CSCW) 27 (2018), 107–148.
  84. S. Russell. 2019. Human Compatible: AI and the Problem of Control. Allen Lane.
  85. Trends, Challenges and Processes in Conversational Agent Design: Exploring Practitioners’ Views through Semi-Structured Interviews. In Proceedings of the 5th International Conference on Conversational User Interfaces (Eindhoven, Netherlands) (CUI ’23). Association for Computing Machinery, New York, NY, USA, Article 13, 10 pages. https://doi.org/10.1145/3571884.3597143
  86. E. Sanders and P. Stappers. 2008. Co-creation and the new landscapes of design. CoDesign 4, 1 (2008), 5–18.
  87. K. Shilton. 2013. Values Levers: Building Ethics into Design. Science, Technology, & Human Values 38, 3 (2013), 374–397.
  88. D. Showkat and E. Baumer. 2022. “It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation. Proceedings of the Designing Interactive Systems (DIS) Conference.
  89. V. Sleeswijk. 2018. Structuring Roles in Research through Design Collaboration. Proceedings of the Design Research Society, DRS 2018: Design as a catalyst for change.
  90. J. Spatz. 2018. Skeuomorphism in VUI design: Jennifer Spatz from Rhode Island School of Design. https://www.youtube.com/watch?v=VxIrn5AV2Eo
  91. S. Spiekermann. 2021. From value-lists to value-based engineering with IEEE 7000™. In IEEE International Symposium on Technology and Society (ISTAS).
  92. M. Steen. 2013. Co-Design as a Process of Joint Inquiry and Imagination. Design Issues 29 (2013), 16–28.
  93. What are you optimizing for? Aligning Recommender Systems with Human Values. Proceedings of the ICML 2020 Participatory Approaches to Machine Learning workshop.
  94. The value sensitive design of a preventive health check app. Ethics and Information Technology 24, 38 (2022).
  95. S. Umbrello. 2018. The moral psychology of value sensitive design: the methodological issues of moral intuitions for responsible innovation. Journal of Responsible Innovation 5, 2 (2018), 186–200. https://doi.org/10.1080/23299460.2018.1457401
  96. Value Sensitive Design to Achieve the UN SDGs with AI: A Case of Elderly Care Robots. Mind and Machines 31 (2021), 395–419.
  97. S. Umbrello and I. Van de Poel. 2021. Mapping value sensitive design onto AI for social good principles. AI and Ethics 1, 3 (2021), 283–296.
  98. I. van de Poel. 2013. Translating Values into Design Requirements. Springer.
  99. I. van de Poel. 2020. Embedding Values in Artificial Intelligence (AI) Systems. Mind and Machines 30 (2020), 385–409.
  100. I. Van de Poel and P. Kroes. 2014. Can Technology Embody Values? Springer, 103–124.
  101. Rama Adithya Varanasi and Nitesh Goyal. 2023. “It is Currently Hodgepodge”: Examining AI/ML Practitioners’ Challenges during Co-Production of Responsible AI Values. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3580903
  102. Q. Vera and M. Muller. 2019. Enabling Value Sensitive AI Systems through Participatory Design Fictions. CoRR (2019).
  103. I. Verdiesen and V. Dignum. 2022. Value elicitation on a scenario of autonomous weapon system deployment: a qualitative study based on the value deliberation process. AI and Ethics (2022).
  104. Design Methods in Design for Values. Springer.
  105. Configuring Participation: On How We Involve People in Design. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
  106. S. Wachter and B. Mittelstadt. 2019. A Right to Reasonable Inferences: Re-Thinking Data Protection Law in the Age of Big Data and AI. Columbia Business Law Review 2 (2019).
  107. M. Wahde and M. Virgolin. 2022. Conversational Agents: Theory and Applications. World Scientific Publishing Company.
  108. Ethical Design of Conversational Agents: Towards Principles for a Value-Sensitive Design. Lecture Notes in Information Systems and Organisation (2021).
  109. FoodChattAR: Exploring the Design Space of Edible Virtual Agents for Human-Food Interaction. Proceedings of the ACM Designing Interactive Systems Conference (DIS). https://doi.org/10.1145/3461778.3461998
  110. Discriminating Systems: Gender, Race, and Power.
  111. The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society.
  112. The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society.
  113. A. Winecoff and E. Watkins. 2022. Artificial Concepts of Artificial Intelligence: Institutional Compliance and Resistance in AI Startups. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3514094.3534138
  114. J. Woelfer and D. Hendry. 2009. Stabilizing homeless young people with information and place. Journal of the American Society for Information Science and Technology 60, 11 (2009), 2300–2312.
  115. Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics. Computing Research Repository (CoRR) (2022). https://arxiv.org/abs/2202.08792
  116. Investigating How Experienced UX Designers Effectively Work with Machine Learning. Proceedings of the CHI Conference on Designing Interactive Systems (DIS).
  117. Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook. In Proceedings of the CHI Conference on Human Factors in Computing Systems,.
  118. A value sensitive action-reflection model: evolving a co-design space with stakeholder and designer prompts. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI).
  119. Towards a multi-stakeholder value-based assessment framework for algorithmic systems. Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT).
  120. Heads-Up Computing Moving Beyond the Device-Centered Paradigm. Commun. ACM 66, 9 (aug 2023), 56–63. https://doi.org/10.1145/3571722
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Malak Sadek (5 papers)
  2. Rafael A. Calvo (5 papers)
  3. Celine Mougenot (1 paper)
Citations (1)
View on Semantic Scholar