Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Can ChatGPT Read Who You Are? (2312.16070v2)

Published 26 Dec 2023 in cs.CY, cs.CL, and cs.HC

Abstract: The interplay between AI and psychology, particularly in personality assessment, represents an important emerging area of research. Accurate personality trait estimation is crucial not only for enhancing personalization in human-computer interaction but also for a wide variety of applications ranging from mental health to education. This paper analyzes the capability of a generic chatbot, ChatGPT, to effectively infer personality traits from short texts. We report the results of a comprehensive user study featuring texts written in Czech by a representative population sample of 155 participants. Their self-assessments based on the Big Five Inventory (BFI) questionnaire serve as the ground truth. We compare the personality trait estimations made by ChatGPT against those by human raters and report ChatGPT's competitive performance in inferring personality traits from text. We also uncover a 'positivity bias' in ChatGPT's assessments across all personality dimensions and explore the impact of prompt composition on accuracy. This work contributes to the understanding of AI capabilities in psychological assessment, highlighting both the potential and limitations of using LLMs for personality inference. Our research underscores the importance of responsible AI development, considering ethical implications such as privacy, consent, autonomy, and bias in AI applications.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. American Psychological Association, 2023. Social Desirability. URL: https://dictionary.apa.org/social-desirability. Accessed: 2023-12-20.
  2. Will Affective Computing Emerge From Foundation Models and General Artificial Intelligence? A First Evaluation of ChatGPT. IEEE Intelligent Systems 38, 15–23.
  3. Why Are Narcissists So Charming at First Sight? Decoding the Narcissism–Popularity Link at Zero Acquaintance. Journal of Personality and Social Psychology 98, 132.
  4. Natural Language Analysis and the Psychology of Verbal Behavior: The Past, Present, and Future States of the Field. Journal of Language and Social Psychology 40, 21–41.
  5. Efficient Utilization of Pre-Trained Models: A Review of Sentiment Analysis via Prompt Learning. Knowledge-Based Systems , 111148.
  6. Reading Between the Lines: Automatic Inference of Self-Assessed Personality Traits from Dyadic Social Chats. Computers in Human Behavior: Artificial Humans 1, 100026.
  7. Meta-Accuracy: Do We Know How Others See Us? .
  8. Statistical Power Analysis for the Behavioral Sciences. Academic press.
  9. Using Language Processing and Speech Analysis for the Identification of Psychosis and Other Disorders. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging 5, 770–779.
  10. Investigating Code Generation Performance of Chat-GPT with Crowdsourcing Social Data, in: Proceedings of the 47th IEEE Computer Software and Applications Conference, pp. 1–10.
  11. Sociální desirabilita (Social Desirability). Ph.D. thesis. Masarykova univerzita. Brno.
  12. The Scientific Credibility of Folk Psychology. Psychology Press.
  13. On the Accuracy of Personality Judgment: A Realistic Approach. Psychological Review 102, 652.
  14. Dimensions Over Categories: A Meta-Analysis of Taxometric Research. Psychological Medicine 50, 1418–1432.
  15. Artificial Intelligence and Social Simulation: Studying Group Dynamics on a Massive Scale. Small Group Research 49, 647–683.
  16. Big Five Inventory: Základní psychometrické charakteristiky české verze BFI-44 a BFI-10 (Big Five Inventory: Basic Psychometric Properties of the Czech Version of BFI-44 and BFI-10). Československá Psychologie 60, 567.
  17. Natural Language Use as a Marker. The Oxford Handbook of Language and Social Psychology , 201–237.
  18. Predicting Personality Using Answers to Open-Ended Interview questions. IEEE Access 8, 115345–115355.
  19. Is ChatGPT a Good Personality Recognizer? A Preliminary Study. arXiv preprint arXiv:2307.03952 .
  20. Paradigm Shift to the Integrative Big Five Trait Taxonomy. Handbook of Personality: Theory and research 3, 114–158.
  21. ChatGPT: Jack of All Trades, Master of None. Information Fusion , 101861.
  22. Personality Trait Recognition Based on Smartphone Typing Characteristics in the Wild. IEEE Transactions on Affective Computing .
  23. Osobnostní markery v textu .
  24. Personality and Word Use: Study on Czech Language and the Big Five. Journal of Psycholinguistic Research 51, 1165–1196.
  25. Výzkum CPACT: Komputační psycholingvistická analýza českého textu (CPACT Research: Computational Psycholinguistic Analysis of Czech Text). Č. Budějovice: Pedagogická fakulta Jihočeské univerzity v Českých Budějovicích .
  26. Tell Me a Story and I Will Tell You Who You Are! Lens Model Analyses of Personality and Creative Writing. Journal of Research in Personality 44, 427–435.
  27. Method, Theory, and Multi-Agent Artificial Intelligence: Creating Computer Models of Complex Social Interaction. Journal for the cognitive science of religion 1, 161.
  28. Machine Learning and Natural Language Processing in Mental Health: Systematic Review. Journal of Medical Internet Research 23, e15708.
  29. Analyzing Dyadic Data with Multilevel Modeling Versus Structural Equation Modeling: A Tale of Two Methods. Journal of Family Psychology 31, 442.
  30. A Reliability Meta-Analysis for 44 Items Big Five Inventory: Based on the Reliability Generalization Methodology. Advances in Psychological Science 23, 755.
  31. Variance in Faking Across Noncognitive Measures. Journal of Applied Psychology 85, 812.
  32. Personality in its Natural Habitat: Manifestations and Implicit Folk Theories of Personality in Daily Life. Journal of Personality and Social Psychology 90, 862.
  33. ChatGPT and Large Language Models in Academia: Opportunities and Challenges. BioData Mining 16, 20.
  34. Sentiment Analysis in Twitter Using Machine Learning Techniques, in: 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), IEEE. pp. 1–5.
  35. The Self-Report Method. Handbook of Research Methods in Personality Psychology.
  36. Linguistic Styles: Language Use as an Individual Difference. Journal of Personality and Social Psychology 77, 1296.
  37. You Don’t Know Me, but I Know You: The Illusion of Asymmetric Insight. Journal of Personality and Social Psychology 81, 639.
  38. A Lexical-Availability-Based Framework from Short Communications for Automatic Personality Identification. Cognitive Systems Research 79, 126–137.
  39. A Cross Sectional Study of ChatGPT in Translation: Magnitude of Use, Attitudes, and Uncertainties. Journal of Psycholinguistic Research , 1–18.
  40. Ethnologue: Languages of the World. SIL International .
  41. Social Media as a Research Tool (SMaaRT) for Risky Behavior Analytics: Methodological Review. JMIR public health and surveillance 6, e21660.
  42. You Are What You Read: Inferring Personality From Consumed Textual Content, in: Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pp. 28–38.
  43. Dotazníky (Questionnaires), in: Svoboda, M., Krejčířová, D., Vágnerová, M. (Eds.), Psychodiagnostika dětí a dospívajících (Psychodiagnostics for Children and Adolescents). Portál, Prague, p. 320.
  44. Attention Is All You Need. Advances in Neural Information Processing Systems 30.
  45. Who Knows What about a Person? The Self–Other Knowledge Asymmetry (SOKA) Model. Journal of Personality and Social Psychology 98, 281.
  46. Zero-Shot Information Extraction via Chatting with ChatGPT. arXiv preprint arXiv:2302.10205 .
  47. Perceiver Effects as Projective Tests: What Your Perceptions of Others Say about You. Journal of Personality and Social Psychology 99, 174–190.
  48. Fine-Tuning Language Models from Human Preferences. arXiv preprint arXiv:1909.08593 .
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Erik Derner (11 papers)
  2. Dalibor Kučera (1 paper)
  3. Nuria Oliver (46 papers)
  4. Jan Zahálka (8 papers)
Citations (4)
View on Semantic Scholar