Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

User Characteristics in Explainable AI: The Rabbit Hole of Personalization? (2403.00137v1)

Published 29 Feb 2024 in cs.HC

Abstract: As AI becomes ubiquitous, the need for Explainable AI (XAI) has become critical for transparency and trust among users. A significant challenge in XAI is catering to diverse users, such as data scientists, domain experts, and end-users. Recent research has started to investigate how users' characteristics impact interactions with and user experience of explanations, with a view to personalizing XAI. However, are we heading down a rabbit hole by focusing on unimportant details? Our research aimed to investigate how user characteristics are related to using, understanding, and trusting an AI system that provides explanations. Our empirical study with 149 participants who interacted with an XAI system that flagged inappropriate comments showed that very few user characteristics mattered; only age and the personality trait openness influenced actual understanding. Our work provides evidence to reorient user-focused XAI research and question the pursuit of personalized XAI based on fine-grained user characteristics.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (82)
  1. Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–18. https://doi.org/10.1145/3173574.3174156
  2. Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE access 6 (2018), 52138–52160. https://doi.org/10.1109/access.2018.2870052 Publisher: IEEE.
  3. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 (June 2020), 82–115. https://doi.org/10.1016/j.inffus.2019.12.012
  4. Verónica Benet-Martínez and Oliver P. John. 1998. Los Cinco Grandes across cultures and ethnic groups: Multitrait-multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology 75 (1998), 729–750. https://doi.org/10.1037/0022-3514.75.3.729
  5. GenderMag: A Method for Evaluating Software’s Gender Inclusiveness. Interacting with Computers 28, 6 (Nov. 2016), 760–787. https://doi.org/10.1093/iwc/iwv046
  6. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI ’20). Association for Computing Machinery, New York, NY, USA, 454–464. https://doi.org/10.1145/3377325.3377498
  7. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (April 2021), 188:1–188:21. https://doi.org/10.1145/3449287
  8. John T Cacioppo and Richard E Petty. 1982. The need for cognition. Journal of personality and social psychology 42, 1 (1982), 116. https://doi.org/10.1037/0022-3514.42.1.116 Publisher: American Psychological Association.
  9. Impacts of Personal Characteristics on User Trust in Conversational Recommender Systems. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3491102.3517471
  10. Toxic Comment Classification Challenge. https://kaggle.com/competitions/jigsaw-toxic-comment-classification-challenge
  11. Toward personalized XAI: A case study in intelligent tutoring systems. Artificial Intelligence 298 (2021), 103503. https://doi.org/10.1016/j.artint.2021.103503
  12. Do We Really Like Robots that Match our Personality? The Case of Big-Five Traits, Godspeed Scores and Robotic Gestures. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, Nanjing, China, 626–631. https://doi.org/10.1109/ROMAN.2018.8525672
  13. Shaping Gestures to Shape Personalities: The Relationship Between Gesture Parameters, Attributed Personality Traits and Godspeed Scores. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, Nanjing, China, 699–704. https://doi.org/10.1109/ROMAN.2018.8525739
  14. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction 18, 5 (Nov. 2008), 455–496. https://doi.org/10.1007/s11257-008-9051-3
  15. A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376638
  16. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. https://doi.org/10.48550/arXiv.1810.04805
  17. A Survey on Personality-Aware Recommendation Systems. Artificial Intelligence Review 55, 3 (March 2022), 2409–2454. https://doi.org/10.1007/s10462-021-10063-7
  18. Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI ’19). Association for Computing Machinery, New York, NY, USA, 275–285. https://doi.org/10.1145/3301275.3302310
  19. Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. (March 2017). https://doi.org/10.48550/arxiv.1702.08608
  20. The role of trust in automation reliance. International Journal of Human-Computer Studies 58 (2003), 697–718. https://doi.org/10.1016/S1071-5819(03)00038-7
  21. The Who in Explainable AI: How AI Background Shapes Perceptions of AI Explanations. https://doi.org/10.48550/arXiv.2107.13509
  22. Upol Ehsan and Mark O. Riedl. 2020. Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach. https://doi.org/10.48550/arXiv.2002.01092
  23. Operationalizing Human-Centered Perspectives in Explainable AI. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3411763.3441342
  24. A Study on Trust in Black Box Models and Post-hoc Explanations. In 14th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2019) (Advances in Intelligent Systems and Computing), Francisco Martínez Álvarez, Alicia Troncoso Lora, José António Sáez Muñoz, Héctor Quintián, and Emilio Corchado (Eds.). Springer International Publishing, Cham, 35–46. https://doi.org/10.1007/978-3-030-20055-8_4
  25. Gender and Participation in Open Source Software Development. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2. https://doi.org/10.1145/3555190
  26. Nirit Glazer. 2011. Challenges with graph interpretation: a review of the literature. Studies in Science Education 47, 2 (Sept. 2011), 183–210. https://doi.org/10.1080/03057267.2011.605307
  27. A very brief measure of the Big-Five personality domains. Journal of Research in Personality 37, 6 (Dec. 2003), 504–528. https://doi.org/10.1016/S0092-6566(03)00046-1
  28. XAI—Explainable artificial intelligence. Science Robotics 4, 37 (Dec. 2019), eaay7120. https://doi.org/10.1126/scirobotics.aay7120
  29. How and why humans trust: A meta-analysis and elaborated model. Frontiers in Psychology 14 (March 2023), 1081086. https://doi.org/10.3389/fpsyg.2023.1081086
  30. Laura Hanu and Unitary team. 2020. Detoxify. https://github.com/unitaryai/detoxify
  31. Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, Marta R. Costa-jussà, Christian Hardmeier, Will Radford, and Kellie Webster (Eds.). Association for Computational Linguistics, Barcelona, Spain (Online), 107–124. https://aclanthology.org/2020.gebnlp-1.10
  32. Metrics for Explainable AI: Challenges and Prospects. http://arxiv.org/abs/1812.04608
  33. Who needs explanation and when? Juggling explainable AI and user epistemic uncertainty. International Journal of Human-Computer Studies 165 (2022), 102839. https://doi.org/10.1016/j.ijhcs.2022.102839
  34. The big five inventory—versions 4a and 54. Berkeley, CA: University of California, Berkeley, Institute of Personality.
  35. Paradigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issues. In Handbook of personality: Theory and research, 3rd ed. The Guilford Press, New York, NY, USA, 114–158.
  36. The Roles of Personality Traits, AI Anxiety, and Demographic Factors in Attitudes toward Artificial Intelligence. International Journal of Human–Computer Interaction 0, 0 (Dec. 2022), 1–18. https://doi.org/10.1080/10447318.2022.2151730
  37. The Age of AI: And Our Human Future. John Murray Press.
  38. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction 22, 4 (Oct. 2012), 441–504. https://doi.org/10.1007/s11257-011-9118-4
  39. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI ’15). Association for Computing Machinery, New York, NY, USA, 126–137. https://doi.org/10.1145/2678025.2701399
  40. Tell me more? the effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’12). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/2207676.2207678
  41. Explanatory Debugging: Supporting End-User Debugging of Machine-Learned Programs. In 2010 IEEE Symposium on Visual Languages and Human-Centric Computing. IEEE, NW Washington, DC, USA, 41–48. https://doi.org/10.1109/VLHCC.2010.15
  42. Too much, too little, or just right? Ways explanations impact end users’ mental models. In 2013 IEEE Symposium on Visual Languages and Human Centric Computing. IEEE, San Jose, CA, USA, 3–10. https://doi.org/10.1109/VLHCC.2013.6645235
  43. Why-oriented end-user debugging of naive Bayes text classification. ACM Transactions on Interactive Intelligent Systems 1, 1 (Oct. 2011), 2:1–2:31. https://doi.org/10.1145/2030365.2030367
  44. Fixing the program my computer learned: barriers for end users, challenges for the machine. In Proceedings of the 14th international conference on Intelligent user interfaces (IUI ’09). Association for Computing Machinery, New York, NY, USA, 187–196. https://doi.org/10.1145/1502650.1502678
  45. Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task. Computers in Human Behavior 139 (2023), 107539. https://doi.org/10.1016/j.chb.2022.107539
  46. User Trust in Recommendation Systems: A comparison of Content-Based, Collaborative and Demographic Filtering. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3491102.3501936
  47. Introduction to Explainable AI. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, 1–3. https://doi.org/10.1145/3411763.3445016
  48. Q. Vera Liao and Kush R. Varshney. 2022. Human-Centered Explainable AI (XAI): From Algorithms to User Experiences. https://doi.org/10.48550/arXiv.2110.10790
  49. Brian Y. Lim and Anind K. Dey. 2009. Assessing demand for intelligibility in context-aware applications. In Proceedings of the 11th international conference on Ubiquitous computing (UbiComp ’09). Association for Computing Machinery, New York, NY, USA, 195–204. https://doi.org/10.1145/1620545.1620576
  50. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09). Association for Computing Machinery, New York, NY, USA, 2119–2128. https://doi.org/10.1145/1518701.1519023
  51. Scott Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. https://doi.org/10.48550/arXiv.1705.07874
  52. How Does Domain Expertise Affect Users’ Search Interaction and Outcome in Exploratory Search? ACM Transactions on Information Systems 36, 4 (July 2018), 42:1–42:30. https://doi.org/10.1145/3223045
  53. Jim Mcbeath. 2011. Levels of Expertise. http://jim-mcbeath.blogspot.com/2011/12/levels-of-expertise.html Publication Title: Jim McBeath.
  54. R. R. McCrae and P. T. Costa. 1987. Validation of the five-factor model of personality across instruments and observers. Journal of Personality and Social Psychology 52, 1 (Jan. 1987), 81–90. https://doi.org/10.1037//0022-3514.52.1.81
  55. Jeremy Miles. 2014. Tolerance and variance inflation factor. Wiley statsref: statistics reference online (2014). https://doi.org/10.1002/9781118445112.stat06593 Publisher: Wiley Online Library.
  56. To explain or not to explain: the effects of personal characteristics when explaining music recommendations. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI ’19). Association for Computing Machinery, New York, NY, USA, 397–407. https://doi.org/10.1145/3301275.3302313
  57. What’s in a User? Towards Personalising Transparency for Music Recommender Interfaces. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’20). Association for Computing Machinery, New York, NY, USA, 173–182. https://doi.org/10.1145/3340631.3394844
  58. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence 267 (Feb. 2019), 1–38. https://doi.org/10.1016/j.artint.2018.07.007
  59. Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences. http://arxiv.org/abs/1712.00547
  60. D. A. Norman. 1987. Some observations on mental models. In Human-computer interaction: a multidisciplinary approach. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 241–244. https://dl.acm.org/doi/10.5555/58076.58097
  61. Judea Pearl. 2009. Causality. Cambridge university press. https://doi.org/10.1017/CBO9780511803161
  62. Opening the Black Box: The Promise and Limitations of Explainable Machine Learning in Cardiology. Canadian Journal of Cardiology 38, 2 (Feb. 2022), 204–213. https://doi.org/10.1016/j.cjca.2021.09.004
  63. Mashup-based information retrieval for domain experts. In Proceedings of the 18th ACM conference on Information and knowledge management (CIKM ’09). Association for Computing Machinery, New York, NY, USA, 711–720. https://doi.org/10.1145/1645953.1646044
  64. Evaluating Explainable AI (XAI) in Terms of User Gender and Educational Background. In Artificial Intelligence in HCI (Lecture Notes in Computer Science), Helmut Degen and Stavroula Ntoa (Eds.). Springer Nature Switzerland, Cham, 286–304. https://doi.org/10.1007/978-3-031-35891-3_18
  65. ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16). Association for Computing Machinery, New York, NY, USA, 1135–1144. https://doi.org/10.1145/2939672.2939778
  66. Marco Tulio Correia Ribeiro. 2023. lime. https://github.com/marcotcr/lime
  67. Lloyd S. Shapley. 1951. Notes on the N-Person Game — II: The Value of an N-Person Game. Technical Report. RAND Corporation. https://www.rand.org/pubs/research_memoranda/RM0670.html
  68. Kacper Sokol and Peter Flach. 2018. Glass-box: explaining AI decisions with counterfactual statements through conversation with a voice-enabled virtual assistant. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). AAAI Press, 5868–5870. https://doi.org/10.24963/ijcai.2018/865 Place: Stockholm, Sweden.
  69. Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample. Journal of Personality and Social Psychology 100 (2011), 330–348. https://doi.org/10.1037/a0021717
  70. On domain expertise-based roles in collaborative information retrieval. Information Processing & Management 50, 5 (Sept. 2014), 752–774. https://doi.org/10.1016/j.ipm.2014.04.002
  71. Interacting meaningfully with machine learning systems: Three experiments. International Journal of Human-Computer Studies 67, 8 (Aug. 2009), 639–662. https://doi.org/10.1016/j.ijhcs.2009.03.004
  72. Integrating rich user feedback into intelligent user interfaces. In Proceedings of the 13th international conference on Intelligent user interfaces (IUI ’08). Association for Computing Machinery, New York, NY, USA, 50–59. https://doi.org/10.1145/1378773.1378781
  73. Axiomatic Attribution for Deep Networks. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 3319–3328. https://proceedings.mlr.press/v70/sundararajan17a.html
  74. Visual, textual or hybrid: the effect of user expertise on different explanations. In 26th International Conference on Intelligent User Interfaces (IUI ’21). Association for Computing Machinery, New York, NY, USA, 109–119. https://doi.org/10.1145/3397481.3450662
  75. Human-Centered Responsible Artificial Intelligence: Current & Future Trends. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–4. https://doi.org/10.1145/3544549.3583178
  76. Personality-based tailored explainable recommendation for trustworthy smart learning system in the age of artificial intelligence. Smart Learning Environments 10, 1 (2023), 65. https://doi.org/10.1186/s40561-023-00282-6 Publisher: Springer.
  77. Adapting Recommendation Diversity to Openness to Experience: A Study of Human Behaviour. In User Modeling, Adaptation, and Personalization (Lecture Notes in Computer Science), Sandra Carberry, Stephan Weibelzahl, Alessandro Micarelli, and Giovanni Semeraro (Eds.). Springer, Berlin, Heidelberg, 190–202. https://doi.org/10.1007/978-3-642-38844-6_16
  78. Nava Tintarev and Judith Masthoff. 2007. A Survey of Explanations in Recommender Systems. In 2007 IEEE 23rd International Conference on Data Engineering Workshop. IEEE, Istanbul, Turkey, 801–810. https://doi.org/10.1109/ICDEW.2007.4401070
  79. Marko Tkalcic and Li Chen. 2015. Personality and Recommender Systems. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, Boston, MA, 715–739. https://doi.org/10.1007/978-1-4899-7637-6_21
  80. User Needs for Explanations of Recommendations: In-depth Analyses of the Role of Item Domain and Personal Characteristics. In Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’23). Association for Computing Machinery, New York, NY, USA, 54–65. https://doi.org/10.1145/3565472.3592950
  81. Characterizing the influence of domain expertise on web search behavior. In Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM ’09). Association for Computing Machinery, New York, NY, USA, 132–141. https://doi.org/10.1145/1498759.1498819
  82. Concept-based document readability in domain specific information retrieval. In Proceedings of the 15th ACM international conference on Information and knowledge management (CIKM ’06). Association for Computing Machinery, New York, NY, USA, 540–549. https://doi.org/10.1145/1183614.1183692
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Robert Nimmo (1 paper)
  2. Marios Constantinides (35 papers)
  3. Ke Zhou (48 papers)
  4. Daniele Quercia (77 papers)
  5. Simone Stumpf (16 papers)
Citations (1)
View on Semantic Scholar