Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cross-Cultural Validation of Partner Models for Voice User Interfaces (2405.09002v1)

Published 15 May 2024 in cs.HC and cs.CY

Abstract: Recent research has begun to assess people's perceptions of voice user interfaces (VUIs) as dialogue partners, termed partner models. Current self-report measures are only available in English, limiting research to English-speaking users. To improve the diversity of user samples and contexts that inform partner modelling research, we translated, localized, and evaluated the Partner Modelling Questionnaire (PMQ) for non-English speaking Western (German, n=185) and East Asian (Japanese, n=198) cohorts where VUI use is popular. Through confirmatory factor analysis (CFA), we find that the scale produces equivalent levels of goodness-to-fit for both our German and Japanese translations, confirming its cross-cultural validity. Still, the structure of the communicative flexibility factor did not replicate directly across Western and East Asian cohorts. We discuss how our translations can open up critical research on cultural similarities and differences in partner model use and design, whilst highlighting the challenges for ensuring accurate translation across cultural contexts.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Glenn Adams and Eric L. Stocks. 2008. A Cultural Analysis of the Experiment and an Experimental Analysis of Culture. Social and Personality Psychology Compass 2, 5 (2008), 1895–1912. https://doi.org/10.1111/j.1751-9004.2008.00137.x _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1751-9004.2008.00137.x.
  2. Decolonial Pathways: Our Manifesto for a Decolonizing Agenda in HCI Research and Design. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3411763.3450365
  3. User representations of computer systems in human-computer speech interaction. International Journal of Man-Machine Studies 38, 4 (April 1993), 547–566. https://doi.org/10.1006/imms.1993.1026
  4. Robotic Coaches Delivering Group Mindfulness Practice at a Public Cafe. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23). Association for Computing Machinery, New York, NY, USA, 86–90. https://doi.org/10.1145/3568294.3580048
  5. Shaowen Bardzell. 2010. Feminist HCI: Taking stock and outlining an agenda for design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). Association for Computing Machinery, New York, NY, USA, 1301–1310. https://doi.org/10.1145/1753326.1753521
  6. Shaowen Bardzell and Jeffrey Bardzell. 2011. Towards a feminist HCI methodology: Social science, feminism, and HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11). Association for Computing Machinery, New York, NY, USA, 675–684. https://doi.org/10.1145/1978942.1979041
  7. Orlando Behling and Kenneth S. Law. 2000. Translating Questionnaires and Other Research Instruments: Problems and Solutions. Vol. 133. Sage, London, UK.
  8. Allan Bell. 1984. Language style as audience design. Language in Society 13, 2 (June 1984), 145–204. https://doi.org/10.1017/S004740450001037X Publisher: Cambridge University Press.
  9. Linda Bell and Joakim Gustafson. 1999. Interaction with an animated agent in a spoken dialogue system. In 6th European Conference on Speech Communication and Technology (Eurospeech 1999). ISCA, 1143–1146. https://doi.org/10.21437/Eurospeech.1999-266
  10. Peter M. Bentler and Chih-Ping Chou. 1987. Practical issues in structural modeling. Sociological methods & research 16, 1 (Aug. 1987), 78–117. https://doi.org/10.1177/0049124187016001004 ISBN: 0049-1241 Publisher: Sage Publications.
  11. Paula Bourges-Waldegg and Stephen A. R. Scrivener. 1998. Meaning, the central issue in cross-cultural HCI design. Interacting with Computers 9, 3 (Jan. 1998), 287–309. https://doi.org/10.1016/S0953-5438(97)00032-5
  12. Linguistic alignment between people and computers. Journal of Pragmatics 42, 9 (Sept. 2010), 2355–2368. https://doi.org/10.1016/j.pragma.2009.12.012
  13. The role of beliefs in lexical alignment: Evidence from dialogs with humans and computers. Cognition 121, 1 (Oct. 2011), 41–57. https://doi.org/10.1016/j.cognition.2011.05.011
  14. Chapter 8 - Two Minds, One Dialog: Coordinating Speaking and Understanding. In Psychology of Learning and Motivation, Brian H. Ross (Ed.). The Psychology of Learning and Motivation: Advances in Research and Theory, Vol. 53. Academic Press, 301–344. https://doi.org/10.1016/S0079-7421(10)53008-1
  15. Timothy A. Brown. 2015. Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press, New York, NY, US. Pages: xvii, 462.
  16. Barry R. Chiswick and Paul W. Miller. 2005. Linguistic distance: A quantitative measure of the distance between English and other languages. Journal of Multilingual and Multicultural Development 26, 1 (Jan. 2005), 1–11. https://doi.org/10.1080/14790710508668395 Publisher: Routledge _eprint: https://doi.org/10.1080/14790710508668395.
  17. The state of speech in HCI: Trends, themes and challenges. Interacting with Computers 31, 4 (June 2019), 349–371. https://doi.org/10.1093/iwc/iwz016
  18. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300705
  19. Lee Anna Clark and David Watson. 2016. Constructing validity: Basic issues in objective scale development. American Psychological Association, Washington, DC, US. https://doi.org/10.1037/14805-012 Pages: 203.
  20. They know as much as we do: Knowledge estimation and partner modelling of artificial partners. In Proceedings of the Annual Meeting of the Cognitive Science Society. Cognitive Science Society, London, UK.
  21. What’s in an accent? The impact of accented synthetic speech on lexical choice in human-machine dialogue. In Proceedings of the 1st International Conference on Conversational User Interfaces (CUI ’19). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3342775.3342786
  22. Norman K. Denzin. 2009. The elephant in the living room: or extending the conversation about the politics of evidence. Qualitative Research 9, 2 (April 2009), 139–160. https://doi.org/10.1177/1468794108098034 Publisher: SAGE Publications.
  23. Norman K. Denzin and Yvonna S. Lincoln. 2005. Introduction: The Discipline and Practice of Qualitative Research. In The SAGE Handbook of Qualitative Research (3rd ed.). Sage Publications Ltd, Thousand Oaks, CA, 1–32.
  24. What Do We See in Them? Identifying Dimensions of Partner Models for Speech Interfaces Using a Psycholexical Approach. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3411764.3445206
  25. Mapping Perceptions of Humanness in Intelligent Personal Assistant Interaction. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3338286.3340116
  26. The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners. https://doi.org/10.48550/arXiv.2308.07164 Submitted to ACM Transactions on Computer-Human Interaction.
  27. Mar Díaz-Millón and María Dolores Olvera-Lobo. 2023. Towards a definition of transcreation: A systematic literature review. Perspectives 31, 2 (March 2023), 347–364. https://doi.org/10.1080/0907676X.2021.2004177 Publisher: Routledge _eprint: https://doi.org/10.1080/0907676X.2021.2004177.
  28. Andy Field. 2013. Discovering Statistics Using IBM SPSS Statistics (4th ed.). SAGE Publications, London, UK.
  29. Vive la différence! A survey of cultural-aware issues in HCI. In Proceedings of the 10th Brazilian Symposium on Human Factors in Computing Systems and the 5th Latin American Conference on Human-Computer Interaction (IHC+CLIHC ’11). Brazilian Computer Society, Porto Alegre, BRA, 13–22.
  30. Edward T. Hall. 1976. Beyond Culture. Anchor, Hamburg, Germany.
  31. Most people are not WEIRD. Nature 466, 7302 (July 2010), 29–29. https://doi.org/10.1038/466029a Number: 7302 Publisher: Nature Publishing Group.
  32. Challenges and Strategies in Cross-Cultural NLP. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Dublin, Ireland, 6997–7013. https://doi.org/10.18653/v1/2022.acl-long.482
  33. Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations With a Chatbot. Journal of Communication 68, 4 (Aug. 2018), 712–733. https://doi.org/10.1093/joc/jqy026
  34. Geert H. Hofstede. 2001. Culture’s Consequences: Comparing Values, Behaviors, Institutions and Organizations Across Nations. SAGE, Thousand Oaks, CA, USA. Google-Books-ID: w6z18LJ_1VsC.
  35. Juliane House and Gabriele Kasper. 1981. Politeness markers in English and German. Rasmus Rask Studies in Pragmatic Linguistics, Volume 2, Conversational Routine (1981), 157–186. Publisher: De Gruyter Mouton.
  36. Li‐tze Hu and Peter M. Bentler. 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6, 1 (Jan. 1999), 1–55. https://doi.org/10.1080/10705519909540118 Publisher: Routledge _eprint: https://doi.org/10.1080/10705519909540118.
  37. Ping-Hsuan Huang and Yu-Shan Zhang. 2020. Media richness and adoption intention of voice assistants: a cross-cultural study. International Journal of Multinational Corporation Strategy 3, 2 (Jan. 2020), 108–129. https://doi.org/10.1504/IJMCS.2020.114688 Publisher: Inderscience Publishers.
  38. International Test Commission. 2018. ITC guidelines for translating and adapting tests. International Journal of Testing 18, 2 (2018), 101–134. https://doi.org/10.1080/15305058.2017.1398166 ISBN: 1530-5058 Publisher: ROUTLEDGE JOURNALS, TAYLOR & FRANCIS LTD 2-4 PARK SQUARE, MILTON PARK ….
  39. Wenying Jiang. 2000. The relationship between culture and language. ELT Journal 54, 4 (Oct. 2000), 328–334. https://doi.org/10.1093/elt/54.4.328
  40. P. N. Johnson-Laird. 1980. Mental models in cognitive science. Cognitive Science 4, 1 (Jan. 1980), 71–115. https://doi.org/10.1016/S0364-0213(81)80005-5
  41. Philip N. Johnson-Laird. 2010. Mental models and human reasoning. Proceedings of the National Academy of Sciences 107, 43 (Oct. 2010), 18243–18250. https://doi.org/10.1073/pnas.1012933107 Publisher: Proceedings of the National Academy of Sciences.
  42. Henry F. Kaiser. 1974. An index of factorial simplicity. Psychometrika 39, 1 (March 1974), 31–36. https://doi.org/10.1007/BF02291575 ISBN: 0033-3123 Publisher: Springer.
  43. Paul Kline. 2000. A Psychometrics Primer. Free Association Books, London, UK.
  44. Çiğdem Kâğıtçıbaşı. 1996. Family and Human Development Across Cultures: A View from the Other Side. Psychology Press.
  45. Cultural Dimensions for User Experience: Cross-Country and Cross-Product Analysis of Users’ Cultural Characteristics. In BCS-HCI ’08, Vol. 1. BCS Learning & Development, Liverpool, UK, 3 –12. https://doi.org/10.14236/ewic/HCI2008.1
  46. Carsten Levisen. 2019. Biases we live by: Anglocentrism in linguistics and cognitive sciences. Language Sciences 76 (Nov. 2019), 101173. https://doi.org/10.1016/j.langsci.2018.05.010
  47. How WEIRD is CHI?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3411764.3445488
  48. Ewa Luger and Abigail Sellen. 2016. ”Like Having a Really Bad PA”: The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). Association for Computing Machinery, New York, NY, USA, 5286–5297. https://doi.org/10.1145/2858036.2858288
  49. Enthusiasts, pragmatists, and skeptics: Investigating users’ attitudes towards emotion- and personality-aware voice assistants across cultures. In Mensch und Computer 2022. Darmstadt, Germany, 23. https://doi.org/10.1145/3543758.3543776
  50. Selfways: Diversity in modes of cultural participation. In The conceptual self in context: Culture, experience, self-understanding. Cambridge University Press, New York, NY, US, 13–61.
  51. Ignacio Martín-Baró. 1994. Writings for a Liberation Psychology. Harvard University Press. Google-Books-ID: EqMdtWL32K0C.
  52. Donald A. Norman. 1983. Some Observations on Mental Models. In Mental Models. Psychology Press, 7–14. Num Pages: 8.
  53. Jum C. Nunnally. 1967. Psychometric Theory (first edition ed.). McGraw-Hill, New York.
  54. Critical Race Theory for HCI. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3313831.3376392
  55. Should speech dialogue system use honorific expression?. In Proceedings of the 7th International Conference on Human-Agent Interaction (HAI ’19). ACM, New York, NY, USA, 232–233. https://doi.org/10.1145/3349537.3352775
  56. Linguistic Adaptations During Spoken and Multimodal Error Resolution. Language and Speech 41, 3-4 (July 1998), 419–442. https://doi.org/10.1177/002383099804100409 Publisher: SAGE Publications Ltd.
  57. Nancy Bonvillain Rock, Bard College at Simon’s. 2019. Language, Culture, and Communication: The Meaning of Messages. Rowman & Littlefield, Lanham, MD, USA. Google-Books-ID: cyyKDwAAQBAJ.
  58. Jennifer A. Rode. 2011. A theoretical agenda for feminist HCI. Interacting with Computers 23, 5 (Sept. 2011), 393–400. https://doi.org/10.1016/j.intcom.2011.04.005 Conference Name: Interacting with Computers.
  59. Katja Rogers and Katie Seaborn. 2023. The systematic review-lution: A manifesto to promote rigour and inclusivity in research synthesis. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3544549.3582733
  60. Language and the Bicultural Self. Personality and Social Psychology Bulletin 28, 8 (Aug. 2002), 1040–1050. https://doi.org/10.1177/01461672022811003 Publisher: SAGE Publications Inc.
  61. Intersectional HCI: Engaging Identity through Gender, Race, and Class. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5412–5427. https://doi.org/10.1145/3025453.3025766
  62. Not Only WEIRD but “Uncanny”? A Systematic Review of Diversity in Human–Robot Interaction Research. International Journal of Social Robotics (March 2023). https://doi.org/10.1007/s12369-023-00968-4
  63. Voice in human-agent interaction: A survey. ACM Computing Surveys (CSUR) 54, 4 (2022), Article No. 81. https://doi.org/10.1145/3386867
  64. Coimagining the Future of Voice Assistants with Cultural Sensitivity. Human Behavior and Emerging Technologies 2024 (March 2024), e3238737. https://doi.org/10.1155/2024/3238737 Publisher: Hindawi.
  65. Katie Seaborn and Jacqueline Urakami. 2021. Measuring voice UX quantitatively: A rapid review. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama, Japan, No. 416. https://doi.org/10.1145/3411763.3451712
  66. Secretariat of the Headquarters for Digital Market Competition, Cabinet Secretariat. 2022. Competition Assessment of New Customer Contact Points (Voice Assistants and Wearables) Interim Report Summary. Technical Report Document 3. Headquarters for Digital Market Competition, Tokyo, Japan. 27 pages. https://www.kantei.go.jp/jp/singi/digitalmarket/pdf_e/documents_220426.pdf
  67. Jeffrey Steedle. 2018. Detecting Inattentive Responding on a Psychosocial Measure of College Readiness. Research Report 2018-5. Research ED592756. ACT, Inc, Iowa City, IA, USA. 26 pages. https://eric.ed.gov/?id=ED592756 Publication Title: ACT, Inc. ERIC Number: ED592756.
  68. Dreaming Disability Justice in HCI. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/3491101.3503731
  69. Fons Trompenaars and Charles Hampden-Turner. 2011. Riding the waves of culture: Understanding diversity in global business. Nicholas Brealey International, Boston, MA, USA.
  70. Trust and Reliance in Consensus-Based Explanations from an Anti-Misinformation Agent. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3544549.3585713
  71. How culture influences perspective taking: differences in correction, not integration. Frontiers in Human Neuroscience 7 (Dec. 2013), 822. https://doi.org/10.3389/fnhum.2013.00822
  72. Shali Wu and Boaz Keysar. 2007. The Effect of Culture on Perspective Taking. Psychological Science 18, 7 (July 2007), 600–606. https://doi.org/10.1111/j.1467-9280.2007.01946.x Publisher: SAGE Publications Inc.
  73. Ching Yun Yu and B.O. Muthén. 2002. Evaluation of model fit indices for latent variable models with categorical and continuous outcomes. In Paper presented at the annual conference of the American Educational Research Association, April 4, 2002, New Orleans. New Orleans, LI, USA.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Katie Seaborn (41 papers)
  2. Iona Gessinger (3 papers)
  3. Suzuka Yoshida (2 papers)
  4. Philip R. Doyle (13 papers)
  5. Benjamin R. Cowan (18 papers)
Citations (1)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets