Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 63 tok/s
Gemini 2.5 Pro 49 tok/s Pro
GPT-5 Medium 11 tok/s Pro
GPT-5 High 10 tok/s Pro
GPT-4o 83 tok/s Pro
Kimi K2 139 tok/s Pro
GPT OSS 120B 438 tok/s Pro
Claude Sonnet 4 38 tok/s Pro
2000 character limit reached

AnthroScore: A Computational Linguistic Measure of Anthropomorphism (2402.02056v1)

Published 3 Feb 2024 in cs.CL, cs.AI, and cs.CY

Abstract: Anthropomorphism, or the attribution of human-like characteristics to non-human entities, has shaped conversations about the impacts and possibilities of technology. We present AnthroScore, an automatic metric of implicit anthropomorphism in language. We use a masked LLM to quantify how non-human entities are implicitly framed as human by the surrounding context. We show that AnthroScore corresponds with human judgments of anthropomorphism and dimensions of anthropomorphism described in social science literature. Motivated by concerns of misleading anthropomorphism in computer science discourse, we use AnthroScore to analyze 15 years of research papers and downstream news articles. In research papers, we find that anthropomorphism has steadily increased over time, and that papers related to LLMs have the most anthropomorphism. Within ACL papers, temporal increases in anthropomorphism are correlated with key neural advancements. Building upon concerns of scientific misinformation in mass media, we identify higher levels of anthropomorphism in news headlines compared to the research papers they cite. Since AnthroScore is lexicon-free, it can be directly applied to a wide range of text sources.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (127)
  1. TensorFlow: a system for large-scale machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16), pages 265–283.
  2. Mirages: On anthropomorphism in dialogue systems. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  3. Euan Adie and William Roe. 2013. Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26(1):11–17.
  4. Gabriella Airenti. 2015. The cognitive bases of anthropomorphism: from relatedness to empathy. International Journal of Social Robotics, 7:117–127.
  5. James Allen. 1995. Natural Language Understanding. Benjamin-Cummings Publishing Co., Inc.
  6. Riveter: Measuring power and social dynamics between entities. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 377–388, Toronto, Canada. Association for Computational Linguistics.
  7. AI Ethics Statements: Analysis and lessons learnt from NeurIPS broader impact statements. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 2047–2056.
  8. Reuven S Avi-Yonah. 2010. Citizens United and the corporate form. Wis. L. Rev., page 999.
  9. Emily M. Bender. 2022. Resisting dehumanization in the age of “AI”. Plenary talk at the 44th Annual Meeting of the Cognitive Science Society (CogSci).
  10. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623.
  11. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics.
  12. Jane Bennett. 2010. Vibrant matter: A political ecology of things. Duke University Press.
  13. The development and psychometric properties of LIWC-22. Austin, TX: University of Texas at Austin, pages 1–47.
  14. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901.
  15. Judith Butler. 2004. Undoing Gender. Psychology Press.
  16. Computational analysis of 140 years of US political speeches reveals more positive but increasingly polarized framing of immigration. Proceedings of the National Academy of Sciences, 119(31):e2120510119.
  17. Who is the “human” in human-centered machine learning: The case of predicting mental health from social media. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–32.
  18. Marked Personas: Using natural language prompts to measure stereotypes in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1504–1532, Toronto, Canada. Association for Computational Linguistics.
  19. CoMPosT: Characterizing and evaluating caricature in LLM simulations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10853–10875, Singapore. Association for Computational Linguistics.
  20. On the use of arXiv as a dataset.
  21. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In ICML, pages 160–167.
  22. Bernard Comrie. 1989. Language universals and linguistic typology: Syntax and morphology. University of Chicago Press.
  23. Anthropomorphism of robots: Study of appearance and agency. JMIR human factors, 6(2):e12629.
  24. Charles Darwin. 1905. Journal of researches. PF Collier.
  25. Charles Darwin and Phillip Prodger. 1998. The expression of the emotions in man and animals. Oxford University Press, USA.
  26. E Emory Davis and Barbara Landau. 2021. Seeing and believing: the relationship between perception and mental verbs in acquisition. Language Learning and Development, 17(1):26–47.
  27. Margo DeMello. 2021. Animals and society: An introduction to human-animal studies. Columbia University Press.
  28. Anthropomorphization of AI: Opportunities and risks. arXiv preprint arXiv:2305.14784.
  29. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  30. Bhopi Dhall and Saurajit Kanungo. 2023. Will AI take over the world? Or will you take charge of your world? Forbes.
  31. Edsger W Dijkstra. 1985. On anthropomorphism in science. EWD936, Sept.
  32. Show your work: Improved reporting of experimental results. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2185–2194, Hong Kong, China. Association for Computational Linguistics.
  33. Someone is pulling the strings: Hypersensitive agency detection and belief in conspiracy theories. Thinking & Reasoning, 22(1):57–77.
  34. On seeing human: a three-factor theory of anthropomorphism. Psychological review, 114(4):864.
  35. Ethan Fast and Eric Horvitz. 2017. Long-term trends in the public perception of artificial intelligence. In Proceedings of the AAAI conference on artificial intelligence, volume 31.
  36. Gabriele Ferri and Inte Gloerich. 2023. Risk and harm: Unpacking ideologies in the ai discourse. In Proceedings of the 5th International Conference on Conversational User Interfaces, pages 1–6.
  37. Anita Fetzer. 2008. “And I think that is a very straightforward way of dealing with it” The communicative function of cognitive verbs in political discourse. Journal of Language and Social Psychology, 27(4):384–396.
  38. Sigmund Freud. 1989. The future of an illusion. W. W. Norton & Company.
  39. Raymond W Gibbs. 1994. The poetics of mind: Figurative thought, language, and understanding. Cambridge University Press.
  40. Linguistic effects on news headline success: Evidence from thousands of online field experiments (registered report). Plos one, 18(3):e0281682.
  41. Robots-Dont-Cry: Understanding falsely anthropomorphic utterances in dialog systems. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3266–3284, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  42. Wei Guo and Aylin Caliskan. 2021. Detecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like biases. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 122–133.
  43. To build our future, we must know our past: Contextualizing paradigm shifts in natural language processing. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13310–13325, Singapore. Association for Computational Linguistics.
  44. Donna Haraway. 1988. Situated knowledges: The science question in feminism and the privilege of partial perspective. Feminist studies, 14(3):575–599.
  45. Sandra Harding. 2013. Rethinking standpoint epistemology: What is “strong objectivity”? In Feminist epistemologies, pages 49–82. Routledge.
  46. Michael J Hathaway. 2022. What a Mushroom Lives For: Matsutake and the worlds they make. Princeton University Press.
  47. Geoffrey Hinton. 2023. ‘Godfather of AI’ says AI could kill humans and there might be no way to stop it. CNN.
  48. Ian Hodder. 2012. Entangled: An archaeology of the relationships between humans and things. John Wiley & Sons.
  49. Message distortion in information cascades. In The World Wide Web Conference, pages 681–692.
  50. David Hume. 1956. The Natural History of Religion. Edited… by HE Root… A. and C. Black.
  51. Tatum Hunter. 2023. 3 things everyone’s getting wrong about AI. Washington Post.
  52. Information retention in the multi-platform sharing of science. In Proceedings of the International AAAI Conference on Web and Social Media, volume 17, pages 375–386.
  53. Ken Hyland. 1998. Boosting, hedging and the negotiation of academic knowledge. Text & Talk, 18(3):349–382.
  54. Cristina Yumie Aoki Inoue. 2018. Worlding the study of global environmental politics in the Anthropocene: Indigenous voices from the Amazon. Global Environmental Politics, 18(4):25–42.
  55. Maria Kallery and Dimitris Psillos. 2004. Anthropomorphism and animism in early years science: Why teachers use them, how they conceptualise them and what are their views on their use. Research in Science Education, 34:291–311.
  56. Dimitri Karadimas. 2012. Animism and perspectivism: Still anthropomorphism? on the problem of perception in the construction of amerindian ontologies. Indiana, 29:25–51.
  57. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences, 103:102274.
  58. Alignment of language agents. arXiv preprint arXiv:2103.14659.
  59. Anthropocentrism: More than just a misunderstood problem. Journal of Agricultural and Environmental Ethics, 31(1):109–127.
  60. Measuring bias in contextualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 166–172, Florence, Italy. Association for Computational Linguistics.
  61. George Lakoff. 2010. Why it matters how we frame the environment. Environmental Communication, 4(1):70–81.
  62. George Lakoff and Mark Johnson. 2008. Metaphors we live by. University of Chicago Press.
  63. Madeline Lamo and Ryan Calo. 2019. Regulating bot speech. UCLA L. Rev., 66:988.
  64. A metaphor-enriched social cognition. Psychological bulletin, 136(6):1045.
  65. Bruno Latour. 2014. Agency at the time of the anthropocene. New literary history, 45(1):1–18.
  66. Multimodal foundation models: From specialists to general-purpose assistants. ArXiv, abs/2309.10020.
  67. Mengjun Li and Ayoung Suh. 2021. Machinelike or humanlike? a literature review of anthropomorphism in AI-enabled technology. In 54th Hawaii International Conference on System Sciences (HICSS 2021), pages 4053–4062.
  68. An NLP analysis of exaggerated claims in science news. In Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism, pages 106–111, Copenhagen, Denmark. Association for Computational Linguistics.
  69. Holistic evaluation of language models. Transactions on Machine Learning Research. Featured Certification, Expert Certification.
  70. Ro{bert}a: A robustly optimized {bert} pretraining approach.
  71. Helen E Longino. 1990. Science as social knowledge: Values and objectivity in scientific inquiry. Princeton University Press.
  72. Tommy L. Lott. 1999. The Invention of Race: Black Culture and the Politics of Representation. Wiley-Blackwell.
  73. Nathalie Maréchal. 2016. Automation, algorithms, and politics| when bots tweet: Toward a normative framework for bots on social networking sites (feature). International Journal of Communication, 10:10.
  74. Sean McManus. 2023. Can AI cut humans out of contract negotiations? BBC.
  75. Witness Mdoka. 2022. Anthropomorphism and anthropocentrism: Human-animal ontology and the environment. The Criterion: An International Journal in English, 13.
  76. Bias against 93 stigmatized groups in masked language models and downstream sentiment classification tasks. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1699–1710.
  77. Efficient estimation of word representations in vector space. In International Conference on Learning Representations.
  78. Fightin’ Words: Lexical feature selection and evaluation for identifying the content of political conflict. Political Analysis, 16(4):372–403.
  79. Anthropomorphism and its adverse effects on the distress and welfare of companion animals. Animals, 11(11):3263.
  80. Large language models shape and are shaped by society: A survey of arXiv publication patterns. arXiv preprint arXiv:2307.10700.
  81. Andreas Musolff. 2010. Metaphor, nation and the holocaust: The concept of the body politic, volume 3. Routledge.
  82. Clifford Nass and Youngme Moon. 2000. Machines and mindlessness: Social responses to computers. Journal of social issues, 56(1):81–103.
  83. Gerald V O’Brien. 2003. Indigestible food, conquering hordes, and waste materials: Metaphors of immigrants and the early immigration restriction debate in the united states. Metaphor and symbol, 18(1):33–47.
  84. When we think about thinking: The acquisition of belief verbs. Cognition, 105(1):125–165.
  85. James W Pennebaker. 2011. The secret life of pronouns. New Scientist, 211(2828):42–45.
  86. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics.
  87. Are corporations people too? the neural correlates of moral judgments about companies and individuals. Social Neuroscience, 10(2):113–125.
  88. Diane Proudfoot. 2011. Anthropomorphism and AI: Turing’s much misunderstood imitation game. Artificial Intelligence, 175(5-6):950–957.
  89. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  90. Connotation frames: A data-driven investigation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 311–321, Berlin, Germany. Association for Computational Linguistics.
  91. Byron Reeves and Clifford Nass. 1996. The Media Equation: How people treat computers, television, and new media like real people. Cambridge, UK, 10(10).
  92. Susanna K Ripken. 2009. Corporations are people too: A multi-dimensional approach to the corporate personhood puzzle. Fordham J. Corp. & Fin. L., 15:97.
  93. ‘Just what do you think you’re doing, Dave?’ A checklist for responsible data use in NLP. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 4821–4833, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  94. The ACL OCL corpus: Advancing open science in computational linguistics. arXiv preprint arXiv:2305.14996.
  95. Anthropomorphized species as tools for conservation: Utility beyond prosocial, intelligent and suffering species. Biodiversity and Conservation, 22:1577–1589.
  96. Anthropomorphism in AI. AJOB neuroscience, 11(2):88–95.
  97. Multitask prompted training enables zero-shot task generalization. In International Conference on Learning Representations.
  98. Otto Santa Ana. 2002. Brown tide rising: Metaphors of Latinos in contemporary American public discourse. University of Texas Press.
  99. Connotation frames of power and agency in modern films. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2329–2334, Copenhagen, Denmark. Association for Computational Linguistics.
  100. Aaron Shackelford. 2010. Dickinson’s animals and anthropomorphism. The Emily Dickinson Journal, 19(2):47–66.
  101. Murray Shanahan. 2022. Talking about large language models. arXiv preprint arXiv:2212.03551.
  102. Human-AI collaboration enables more empathic conversations in text-based peer-to-peer mental health support. Nature Machine Intelligence, 5(1):46–57.
  103. Josepha Sherman. 2015. Storytelling: An encyclopedia of mythology and folklore. Routledge.
  104. Ben Shneiderman. 2020. Design lessons from ai’s two grand goals: human emulation and useful applications. IEEE Transactions on Technology and Society, 1(2):73–82.
  105. Ben Shneiderman. 2022. Human-centered AI. Oxford University Press.
  106. Different models of anthropomorphism across cultures and ontological limits in current frameworks the integrative framework of anthropomorphism. Frontiers in Robotics and AI, page 230.
  107. Lucy Suchman. 2008. Feminist STS and the sciences of the artificial. The handbook of science and technology studies, 3:139–164.
  108. Lucy G Sullivan. 1995. Myth, metaphor and hypothesis: How anthropomorphism defeats science. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 349(1328):215–218.
  109. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ, 349.
  110. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
  111. Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: Liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54.
  112. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  113. Natasha Tiku. 2022. The Google engineer who thinks the company’s AI has come to life. Washington Post.
  114. Caroline Tipler and Janet B Ruscher. 2014. Agency’s role in dehumanization: Non-human metaphors of out-groups. Social and Personality Psychology Compass, 8(5):214–228.
  115. Explanations can reduce overreliance on AI systems during decision-making. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1):1–38.
  116. Echo Wen Wan and Rocky Peng Chen. 2021. Anthropomorphism and object attachment. Current Opinion in Psychology, 39:88–93.
  117. It’s alive! increasing protective action against the coronavirus through anthropomorphism and construal. Journal of the Association for Consumer Research, 7(1):81–88.
  118. David Watson. 2019. The rhetoric and reality of anthropomorphism in artificial intelligence. Minds and Machines, 29(3):417–440.
  119. Social cognition unbound: Insights into anthropomorphism and dehumanization. Current Directions in Psychological Science, 19(1):58–62.
  120. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  121. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 214–229.
  122. Matthew Wood. 2019. The potential for anthropomorphism in communicating science: Inspiration from Japan. Cultures of Science, 2(1):23–34.
  123. Modeling information change in science communication with semantically matched paraphrases. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1783–1807.
  124. Itai Yanai and Martin Lercher. 2020. The two languages of science. Genome Biology, 21:1–9.
  125. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490.
  126. Runting Zhong and Mengyao Ma. 2022. Effects of communication style, anthropomorphic setting and individual differences on older adults using voice assistants in a health context. BMC Geriatrics, 22(1):751.
  127. Multimodal C4: An open, billion-scale corpus of images interleaved with text. arXiv preprint arXiv:2304.06939.
Citations (9)
View on Semantic Scholar

Summary

  • The paper introduces AnthroScore, a method leveraging masked language models to quantify anthropomorphic language in technical texts.
  • It validates the approach with an annotation study that shows strong correlation with human judgments of anthropomorphism.
  • Empirical analysis reveals a rising trend of anthropomorphism in computer science research and media headlines over the past 15 years.

Introduction

Modern discussions about artificial intelligence frequently feature anthropomorphism—the attribution of human qualities to non-human entities. This paper introduces AnthroScore, a computational tool that quantifies implicit anthropomorphism within text. Developed against the backdrop of a surge in anthropomorphic language in technological discourse—which can foster misconceptions about AI capabilities—the authors offer a systematic method to measure such language patterns. Leveraging a masked LLM (MLM), AnthroScore assesses the degree to which non-human entities, particularly technical artifacts, are framed in human-like terms. The paper details the increasing prevalence of anthropomorphism in computer science research, its correlations with developments in neural LLMs, and its amplification in news media headlines.

Anthropomorphism in Language

Key to AnthroScore's approach is its reliance on MLMs for context analysis, determining probabilities that entities within text snippets are implicitly human-framed. Real-world implications drive the work; misleading narratives spawn undue trust in technology and distract from actual risks. Anthropomorphic language's historical uses for both positive and negative outcomes are highlighted, with vivid examples illustrating its power to shape human thought and societal discourse.

Measuring Anthropomorphism

The authors meticulously outline the methodology for computing AnthroScore. It involves creating a data corpus comprising sentences with masked mentions of target entities and using MLMs to predict the likelihood that these entities are described using human or non-human pronouns. A comprehensive annotation paper underpins the validity of AnthroScore, showing a significant correlation to human judgments of anthropomorphism. The method's lexicon-free nature means it is adaptable across textual domains, an edge over prior techniques.

Empirical Observations

Analyzing over 600,000 research papers and 55,000 computational linguistics abstracts, along with thousands of corresponding news headlines, the paper presents sobering findings. Not only has anthropomorphism in CS research papers increased over the last 15 years, but it also peaks in discussions on LLMs. Furthermore, news outlets exacerbate this trend, with significantly higher levels of anthropomorphism in headlines than in the abstracts they summarize—raising concerns about the misrepresentation of research in public forums.

Conclusions

AnthroScore successfully lays bare the ingrained and increasing pattern of anthropomorphism within AI discourse. The authors call for conscious efforts to curb such language, recommending vigilance in verb usage and evaluating the framing of technical artifacts. They propose actionable steps for both researchers and academic communities to mitigate the risks associated with anthropomorphism. Beyond its immediate application, AnthroScore has implications for cross-disciplinary studies and understanding the broader socio-cultural impact of anthropomorphic metaphors.

Understanding the language used in AI and technology contexts is not just an academic exercise. Anthropomorphic language shapes perceptions, influences policy, and has real-world consequences. AnthroScore provides a vital lens through which these dynamics can be observed, analyzed, and, ultimately, addressed. As AI becomes an increasingly integral part of society, tools like AnthroScore are key to maintaining an accurate and responsible dialogue about technology's abilities and limitations.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Lightbulb On Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 3 posts and received 72 likes.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up to View