To Trust or Distrust Trust Measures: Validating Questionnaires for Trust in AI (2403.00582v2)
Abstract: Despite the importance of trust in human-AI interactions, researchers must adopt questionnaires from other disciplines that lack validation in the AI context. Motivated by the need for reliable and valid measures, we investigated the psychometric quality of two trust questionnaires, the Trust between People and Automation scale (TPA) by Jian et al. (2000) and the Trust Scale for the AI Context (TAI) by Hoffman et al. (2023). In a pre-registered online experiment (N = 1485), participants observed interactions with trustworthy and untrustworthy AI (autonomous vehicle and chatbot). Results support the psychometric quality of the TAI while revealing opportunities to improve the TPA, which we outline in our recommendations for using the two questionnaires. Furthermore, our findings provide additional empirical evidence of trust and distrust as two distinct constructs that may coexist independently. Building on our findings, we highlight the opportunities and added value of measuring both trust and distrust in human-AI research and advocate for further work on both constructs.
- “Transparency in Measurement Reporting: A Systematic Literature Review of CHI PLAY” In Proc. ACM Hum.-Comput. Interact. 5.CHI PLAY New York, NY, USA: Association for Computing Machinery, 2021 DOI: 10.1145/3474660
- “Trusting intelligent machines: Deepening trust within socio-technical systems” In IEEE Technology and Society Magazine 37.4 IEEE, 2018, pp. 76–83 DOI: 10.1109/MTS.2018.2876107
- John Benamati, Mark A Serva and Mark A Fuller “Are trust and distrust distinct constructs? An empirical study of the effects of trust and distrust among online banking users” In Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS’06) 6 IEEE, 2006, pp. 121b–121b DOI: 10.1109/HICSS.2006.63
- “The Value of Measuring Trust in AI-A Socio-Technical System Perspective” Retrieved from https://arxiv.org/abs/2204.13480 In arXiv Preprint arXiv:2204.13480, 2022
- Joyce Berg, John Dickhaut and Kevin McCabe “Trust, reciprocity, and social history” In Games and economic behavior 10.1 Elsevier, 1995, pp. 122–142 DOI: 10.1006/game.1995.1027
- “’It’s Reducing a Human Being to a Percentage”’ In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18 New York, NY: ACM, 2018, pp. 1–14 DOI: 10.1145/3173574.3173951
- “The Quality of Data Collected Online: An Investigation of Careless Responding in a Crowdsourced Sample” In Methods in Psychology 2, 2020, pp. 100022 DOI: 10.1016/j.metip.2020.100022
- “Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems” In Proceedings of the 25th International Conference on Intelligent User Interfaces, IUI ’20 Cagliari, Italy: ACM, 2020, pp. 454–464 DOI: 10.1145/3377325.3377498
- Michael Buhrmester, Tracy Kwang and Samuel D. Gosling “Amazon’s Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?” In Perspectives on Psychological Science 6.1, 2011, pp. 3–5 DOI: 10.1177/1745691610393980
- John T Cacioppo and Gary G Berntson “Relationship Between Attitudes and Evaluative Space: A Critical Review, with Emphasis on the Separability of Positive and Negative Substrates.” In Psychological Bulletin 115.3 American Psychological Association, 1994, pp. 401–423 DOI: 10.1037/0033-2909.115.3.401
- “Trust theory: A socio-cognitive and computational model” Hoboken, NJ: John Wiley & Sons, 2010 DOI: 10.1002/9780470519851
- “Antecedents and distinctions between online trust and distrust: Predicting high-and low-risk internet behaviors” In Journal of Electronic Commerce Research 14.2 Journal of Electronic Commerce Research, 2013, pp. 149
- Lee J Cronbach “Coefficient alpha and the internal structure of tests” In Psychometrika 16.3 Springer, 1951, pp. 297–334 DOI: 10.1007/BF02310555
- Paul G. Curran “Methods for the detection of carelessly invalid responses in survey data” In Journal of Experimental Social Psychology 66, 2016, pp. 4–19 DOI: 10.1016/j.jesp.2015.07.006
- Fred D. Davis “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology” In MIS Quarterly 13.3 Management Information Systems Research Center, University of Minnesota, 1989, pp. 319–340 DOI: 10.2307/249008
- Robert F. DeVellis “Scale Development: Theory and Applications” Thousand Oaks, CA: SAGE publications, Inc., 2017
- “Towards a Generalized Scale to Measure Situational Trust in AI Systems” In CHI 2022 TRAIT Workshop on Trust and Reliance in AI-Human Teams Virtual/New Orleans, LA, USA: Association for Computing Machinery, 2022
- Benjamin D Douglas, Patrick J Ewell and Markus Brauer “Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA” In PLoS ONE 18.3, 2023, pp. e0279720 DOI: 10.1371/journal.pone.0279720
- Thomas J. Dunn, Thom Baguley and Vivienne Brunsden “From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation” In British Journal of Psychology 105.3, 2014, pp. 399–412 DOI: 10.1111/bjop.12046
- Fredrick Ekman, Mikael Johansson and Jana Sochor “Creating appropriate trust in automated vehicle systems: A framework for HMI design” In IEEE Transactions on Human-Machine Systems 48.1 IEEE, 2017, pp. 95–101 DOI: 10.1109/THMS.2017.2776209
- Connor Esterwood and Lionel P Robert Jr “Three Strikes and you are out!: The impacts of multiple human–robot trust violations and repairs on robot trustworthiness” In Computers in Human Behavior 142, 2023, pp. 107658 DOI: 10.1016/j.chb.2023.107658
- Jessica K. Flake and Eiko I. Fried “Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them” In Advances in Methods and Practices in Psychological Science 3.4, 2020, pp. 456–465 DOI: 10.1177/2515245920952393
- Eiko I. Fried “Theories and Models: What They Are, What They Are for, and What They Are About” In Psychological Inquiry 31.4 Routledge, 2020, pp. 336–344 DOI: 10.1080/1047840X.2020.1854011
- Francis Fukuyama “Trust: The social virtues and the creation of prosperity” New York, NY: Free Press, 1996
- Mike Furr “Scale Construction and Psychometrics for Social and Personality Psychology” London: SAGE publications, Ltd., 2011
- Diego Gambetta “Can We Trust Trust?” In Trust: Making and Breaking Cooperative Relations, electronic edition Oxford: Department of Sociology, University of Oxford, 2000, pp. 213–237 URL: http://www.sociology.ox.ac.uk/papers/gambetta213-237.pdf
- “IBM SPSS statistics 26 step by step: A simple guide and reference” New York, NY: Routledge, 2019 DOI: 10.4324/9780429056765
- “Multivariate Data Analysis” Hoboken, NJ: Prentice Hall, 2010
- D Harrison McKnight and Norman Chervany “While trust is cool and collected, distrust is fiery and frenzied: A model of distrust concepts” In 7th Americas Conference on Information Systems Boston, MA: AMCIS 2001 Proceedings, 2001, pp. 883–888
- “A Class of Invariant Consistent Tests for Multivariate Normality” In Communications in Statistics-Theory and Methods 19.10 Taylor & Francis, 1990, pp. 3595–3617 DOI: 10.1080/03610929008830400
- Kevin A. Hoff and Masooda Bashir “Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust” In Human Factors 57.3 Los Angeles, CA, USA: Sage Publications, 2015, pp. 407–434 DOI: 10.1177/0018720814547570
- “The dynamics of trust in cyberdomains” In IEEE Intelligent Systems 24.6 IEEE, 2009, pp. 5–11 DOI: 10.1109/MIS.2009.124
- “Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance” In Frontiers in Computer Science 5, 2023 DOI: 10.3389/fcomp.2023.1096257
- “Situational Trust Scale for Automated Driving (STS-AD): Development and Initial Validation” In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’20 Virtual Event, DC, USA: Association for Computing Machinery, 2020, pp. 40–47 DOI: 10.1145/3409120.3410637
- Kenneth D Hopkins “Educational and Psychological Measurement and Evaluation” London: Pearson, 1998
- Matt C. Howard “A Review of Exploratory Factor Analysis Decisions and Overview of Current Practices: What We Are Doing and How Can We Improve?” In International Journal of Human–Computer Interaction 32.1 Taylor & Francis, 2016, pp. 51–62 DOI: 10.1080/10447318.2015.1087664
- Li-tze Hu and Peter M. Bentler “Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria Versus new Alternatives” In Structural Equation Modeling: A Multidisciplinary Journal 6.1 Routledge, 1999, pp. 1–55 DOI: 10.1080/10705519909540118
- “Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21 Virtual Event, Canada: ACM, 2021, pp. 624–635 DOI: 10.1145/3442188.3445923
- “How Different Groups Prioritize Ethical Values for Responsible AI” In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22 Seoul, Republic of Korea: Association for Computing Machinery, 2022, pp. 310–323 DOI: 10.1145/3531146.3533097
- Jiun-Yin Jian, Ann M. Bisantz and Colin G. Drury “Foundations for an Empirically Determined Scale of Trust in Automated Systems” In International Journal of Cognitive Ergonomics 4.1 London, UK, GBR: Routledge, 2000, pp. 53–71 DOI: 10.1207/S15327566IJCE0401_04
- Elizabeth F. Juniper “Validated Questionnaires Should not be Modified” In European Respiratory Journal 34.5 European Respiratory Society, 2009, pp. 1015–1017 DOI: 10.1183/09031936.00110209
- ““Because AI is 100% Right and Safe”: User Attitudes and Sources of AI Authority in India” In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22 New Orleans, LA, USA: Association for Computing Machinery, 2022 DOI: 10.1145/3491102.3517533
- “Trust in artificial intelligence: Meta-analytic findings” In Human factors 65.2 SAGE Publications Sage CA: Los Angeles, CA, 2023, pp. 337–359 DOI: 10.1177/00187208211013988
- Rex B. Kline “Principles and practice of structural equation modeling” New York, NY: The Guilford Press, 2016
- “Measurement of trust in automation: A narrative review and reference guide” In Frontiers in Psychology 12 Frontiers Media SA, 2021, pp. 604977 DOI: 10.3389/fpsyg.2021.604977
- Frens Kroeger “Unlocking the treasure trove: How can Luhmann’s theory of trust enrich trust research?” In Journal of Trust Research 9.1 Taylor & Francis, 2019, pp. 110–124 DOI: 10.1080/21515581.2018.1552592
- “Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies” In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’23 Chicago, IL, USA: Association for Computing Machinery, 2023, pp. 1369–1385 DOI: 10.1145/3593013.3594087
- “What Do We Want From Explainable Artificial Intelligence (XAI)? A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research” In Artificial Intelligence 296 Amsterdam, NLD: Elsevier, 2021, pp. 103473 DOI: 10.1016/j.artint.2021.103473
- ““Look! It’s a Computer Program! It’s an Algorithm! It’s AI!”: Does Terminology Affect Human Perceptions and Evaluations of Algorithmic Decision-Making Systems?” In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22 New Orleans, LA, USA: Association for Computing Machinery, 2022 DOI: 10.1145/3491102.3517527
- John D. Lee and Katrina A. See “Trust in Automation: Designing for Appropriate Reliance” In Human factors 46.1 London, UK, GBR: SAGE Publications, 2004, pp. 50–80 DOI: 10.1518/hfes.46.1.50_30392
- Roy J Lewicki, Daniel J McAllister and Robert J Bies “Trust and Distrust: New Relationships and Realities” In The Academy of Management Review 23.3 Academy of Management Briarcliff Manor, NY 10510, 1998, pp. 438–458 DOI: 10.2307/259288
- Roy J Lewicki, Edward C Tomlinson and Nicole Gillespie “Models of interpersonal trust development: Theoretical approaches, empirical evidence, and future directions” In Journal of management 32.6 Sage Publications Sage CA: Thousand Oaks, CA, 2006, pp. 991–1022 DOI: 10.1177/0149206306294405
- James R. Lewis and Jeff Sauro “Revisiting the Factor Structure of the System Usability Scale” In Journal of Usability Studies 12.4, 2017, pp. 183–192
- James R. Lewis, Brian S. Utesch and Deborah E. Maher “UMUX-LITE: When There’s No Time for the SUS” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13 Paris, France: Association for Computing Machinery, 2013, pp. 2099–2102 DOI: 10.1145/2470654.2481287
- “Designing for Responsible Trust in AI Systems: A Communication Perspective” In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22 Seoul, Republic of Korea: Association for Computing Machinery, 2022, pp. 1257–1268 DOI: 10.1145/3531146.3533182
- Zachary C. Lipton “The Mythos of Model Interpretability: In Machine Learning, the Concept of Interpretability is Both Important and Slippery.” In Queue 16.3 New York, NY, USA: ACM, 2018, pp. 31–57 DOI: 10.1145/3236386.3241340
- Robert Loo “A caveat on using single-item versus multiple-item scales” In Journal of Managerial Psychology 17.1 MCB UP Ltd, 2002, pp. 68–75 DOI: 10.1108/02683940210415933
- “Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks” In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21 Yokohama, Japan: Association for Computing Machinery, 2021 DOI: 10.1145/3411764.3445562
- Niklas Luhmann “Trust and Power” Hoboken, NJ: Wiley, 1979
- Robert C MacCallum, Michael W Browne and Hazuki M Sugawara “Power analysis and determination of sample size for covariance structure modeling” In Psychological methods 1.2 American Psychological Association, 1996, pp. 130–149 DOI: 10.1037/1082-989X.1.2.130
- Kantilal Vardichand Mardia “Measures of Multivariate Skewness and Kurtosis with Applications” In Biometrika 57.3, 1970, pp. 519–530 DOI: 10.1093/biomet/57.3.519
- Roger C. Mayer, James H. Davis and F.David Schoorman “An Integrative Model of Organizational Trust” In Academy of Management Review 20.3 Briarcliff Manor, NY, USA: Academy of Management, 1995, pp. 709–734 DOI: 10.2307/258792
- Roderick P McDonald “Test theory: A unified treatment” New York, NY: Psychology Press, 1999 DOI: 10.4324/9781410601087
- D.Harrison McKnight and Norman L. Chervany “What trust means in e-commerce customer relationships: An interdisciplinary conceptual typology” In International journal of electronic commerce 6.2 Taylor & Francis, 2001, pp. 35–59 DOI: 10.1080/10864415.2001.11044235
- Adam W Meade and S Bartholomew Craig “Identifying careless responses in survey data.” In Psychological methods 17.3 American Psychological Association, 2012, pp. 437–455 DOI: 10.1037/a0028085
- Stephanie M. Merritt “Affective Processes in Human–Automation Interactions” In Human Factors 53.4 Los Angeles, CA, USA: Sage Publications, 2011, pp. 356–370 DOI: 10.1177/0018720811411912
- Sina Mohseni, Niloofar Zarei and Eric D. Ragan “A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems” Retrieved from https://arxiv.org/abs/1811.11839 In arXiv: Human-Computer Interaction, 2020 eprint: 1811.11839
- “Testtheorie und Fragebogenkonstruktion [Test Theory and Questionnaire Construction]” Berlin, Germany: Springer Berlin, Heidelberg, 2020 DOI: 10.1007/978-3-662-61532-4
- Bonnie M Muir “Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems” In Ergonomics 37.11 Taylor & Francis, 1994, pp. 1905–1922 DOI: 10.1080/00140139408964957
- Carol Xiaojuan Ou and Choon Ling Sia “To Trust or to Distrust, That is the Question: Investigating the Trust-Distrust Paradox” In Commun. ACM 52.5 New York, NY, USA: Association for Computing Machinery, 2009, pp. 135–139 DOI: 10.1145/1506409.1506442
- “Humans and Automation: Use, Misuse, Disuse, Abuse” In Human Factors 39.2 Los Angeles, CA, USA: Sage Publications, 1997, pp. 230–253 DOI: 10.1518/001872097778543886
- “Data quality of platforms and panels for online behavioral research” In Behavior Research Methods 54 Springer, 2022, pp. 1643–1662 DOI: 10.3758/s13428-021-01694-3
- Sebastian A.C. Perrig, Nicolas Scharowski and Florian Brühlmann “Trust Issues with Trust Scales: Examining the Psychometric Quality of Trust Measures in the Context of AI” In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA ’23 Hamburg, Germany: Association for Computing Machinery, 2023 DOI: 10.1145/3544549.3585808
- “Development and Validation of a Positive-Item Version of the Visual Aesthetics of Websites Inventory: The VisAWI-Pos” In International Journal of Human–Computer Interaction 0.0 Taylor & Francis, 2023, pp. 1–25 DOI: 10.1080/10447318.2023.2258634
- Tobias M. Peters and Roel W. Visser “The Importance of Distrust in AI” In Explainable Artificial Intelligence Cham: Springer Nature Switzerland, 2023, pp. 301–317 DOI: 10.1007/978-3-031-44070-0_15
- “Manipulating and Measuring Model Interpretability” In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI ’21 Yokohama, Japan: ACM, 2021 DOI: 10.1145/3411764.3445315
- R Core Team “R: A Language and Environment for Statistical Computing”, 2022 R Foundation for Statistical Computing URL: https://www.R-project.org/
- “Not So Different After All: A Cross-Discipline View Of Trust” In Academy of Management Review 23.3 Briarcliff Manor, NY, USA: Academy of Management Briarcliff Manor, NY 10510, 1998, pp. 393–404 DOI: 10.5465/AMR.1998.926617
- Mark NK Saunders, Graham Dietz and Adrian Thornhill “Trust and distrust: Polar opposites, or independent but co-existing?” In Human Relations 67.6 Sage Publications Sage UK: London, England, 2014, pp. 639–665 URL: https://doi.org/10.1177/0018726713500831
- Jeff Sauro and James R. Lewis “When Designing Usability Questionnaires, Does It Hurt to Be Positive?” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11 Vancouver, BC, Canada: Association for Computing Machinery, 2011, pp. 2215–2224 DOI: 10.1145/1978942.1979266
- Kristin E. Schaefer “Measuring Trust in Human Robot Interactions: Development of the "Trust Perception Scale-HRI"” In Robust Intelligence and Trust in Autonomous Systems Boston, MA: Springer US, 2016, pp. 191–218 DOI: 10.1007/978-1-4899-7668-0_10
- Nicolas Scharowski and Sebastian A.C. Perrig “Distrust in (X)AI – Measurement Artifact or Distinct Construct?” In CHI 2023 TRAIT Workshop on Trust and Reliance in AI-Human Teams Hamburg, Germany: Association for Computing Machinery, 2023 DOI: 10.48550/arXiv.2303.16495
- “Trust and Reliance in XAI–Distinguishing Between Attitudinal and Behavioral Measures” In CHI TRAIT Workshop Virtual/New Orleans, LA, USA: Association for Computing Machinery, 2022 DOI: 10.48550/arXiv.2203.12318
- “Certification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study” In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’23 Chicago, IL, USA: Association for Computing Machinery, 2023, pp. 248–260 DOI: 10.1145/3593013.3593994
- “A Micro and Macro Perspective on Trustworthiness: Theoretical Underpinnings of the Trustworthiness Assessment Model (TrAM)” PsyArXiv, 2022 DOI: 10.31234/osf.io/qhwvx
- “Trust and distrust on the web: User experiences and website characteristics” In Computers in human behavior 45 Elsevier, 2015, pp. 39–50 DOI: doi.org/10.1016/j.chb.2014.11.064
- Sim B Sitkin and Nancy L Roth “Explaining the limited effectiveness of legalistic “remedies” for trust/distrust” In Organization science 4.3 INFORMS, 1993, pp. 367–392 URL: https://doi.org/10.1287/orsc.4.3.367
- Randall D. Spain, Ernesto A. Bustamante and James P. Bliss “Towards an Empirically Developed Scale for System Trust: Take Two” In Proceedings of the Human Factors and Ergonomics Society Annual Meeting 52.19 Los Angeles, USA: SAGE Publications, 2008, pp. 1335–1339 DOI: 10.1177/154193120805201907
- Lauren Thornton, Bran Knowles and Gordon Blair “Fifty Shades of Grey: In Praise of a Nuanced Approach Towards Trustworthy Design” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21 Virtual Event, Canada: Association for Computing Machinery, 2021, pp. 64–76 DOI: 10.1145/3442188.3445871
- “The Relationship Between Trust in AI and Trustworthy Machine Learning Technologies” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency New York, NY: ACM, 2020, pp. 272–283 DOI: 10.1145/3351095.3372834
- “Trust in Human-AI Interaction: Scoping Out Models, Measures, and Methods” In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI EA ’22 New Orleans, LA, USA: Association for Computing Machinery, 2022 DOI: 10.1145/3491101.3519772
- Oleksandra Vereschak, Gilles Bailly and Baptiste Caramiaux “How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies” In Proc. ACM Hum.-Comput. Interact. 5.CSCW2 New York, NY, USA: Association for Computing Machinery, 2021 DOI: 10.1145/3476068
- “Gradual trust and distrust in recommender systems” In Fuzzy Sets and Systems 160.10 Elsevier, 2009, pp. 1367–1382 DOI: 10.1016/j.fss.2008.11.014
- David Watson, Lee Anna Clark and Auke Tellegen “Development and validation of brief measures of positive and negative affect: the PANAS scales.” In Journal of personality and social psychology 54.6 American Psychological Association, 1988, pp. 1063–1070 DOI: 10.1037/0022-3514.54.6.1063
- ““Let me explain!”: exploring the potential of virtual agents in explainable AI interaction design” In Journal on Multimodal User Interfaces 15.2 Springer, 2021, pp. 87–98 DOI: 10.1007/s12193-020-00332-0
- Tiffany A. Whittaker and Randall E. Schumacker “A beginner’s guide to structural equation modeling” New York, NY: Routledge, 2022 DOI: 10.4324/9781003044017
- Magdalena Wischnewski, Nicole Krämer and Emmanuel Müller “Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions” In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23 Hamburg, Germany: Association for Computing Machinery, 2023 DOI: 10.1145/3544548.3581197
- Ming Yin, Jennifer Wortman Vaughan and Hanna Wallach “Understanding The Effect of Accuracy on Trust in Machine Learning Models” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19 Glasgow, Scotland Uk: Association for Computing Machinery, 2019, pp. 1–12 DOI: 10.1145/3290605.3300509
- “User Trust Dynamics: An Investigation Driven by Differences in System Performance” In Proceedings of the 22nd international conference on intelligent user interfaces, IUI ’17 Limassol, Cyprus: Association for Computing Machinery, 2017, pp. 307–317 DOI: 10.1145/3025171.3025219