Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LMs stand their Ground: Investigating the Effect of Embodiment in Figurative Language Interpretation by Language Models (2305.03445v4)

Published 5 May 2023 in cs.CL

Abstract: Figurative language is a challenge for LLMs since its interpretation is based on the use of words in a way that deviates from their conventional order and meaning. Yet, humans can easily understand and interpret metaphors, similes or idioms as they can be derived from embodied metaphors. Language is a proxy for embodiment and if a metaphor is conventional and lexicalised, it becomes easier for a system without a body to make sense of embodied concepts. Yet, the intricate relation between embodiment and features such as concreteness or age of acquisition has not been studied in the context of figurative language interpretation concerning LLMs. Hence, the presented study shows how larger LLMs perform better at interpreting metaphoric sentences when the action of the metaphorical sentence is more embodied. The analysis rules out multicollinearity with other features (e.g. word length or concreteness) and provides initial evidence that larger LLMs conceptualise embodied concepts to a degree that facilitates figurative language understanding.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Lawrence W Barsalou. 1999. Perceptual symbol systems. Behavioral and brain sciences, 22(4):577–660.
  2. Emily M Bender and Alexander Koller. 2020. Climbing towards nlu: On meaning, form, and understanding in the age of data. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 5185–5198.
  3. Experience grounds language. In Proceedings of the 2020 Conference on EMNLP, pages 8718–8735. ACL.
  4. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow.
  5. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  6. Concreteness ratings for 40 thousand generally known english word lemmas. Behavior research methods, 46(3):904–911.
  7. Flute: Figurative language understanding and textual explanations. In Proceedings of the 2022 Conference on EMNLP.
  8. Lucia Colombo and Cristina Burani. 2002. The influence of age of acquisition, root frequency, and context availability in processing nouns and verbs. Brain and Language, 81(1-3):398–411.
  9. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the NAACL: Human Language Technologies, page 4171–4186.
  10. Linguistic bodies: The continuity between life and language. MIT press.
  11. Concretext@ evalita2020: The concreteness in context task. In EVALITA proceedings. CEUR.org.
  12. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv:2209.00840.
  13. Somatotopic representation of action words in human motor and premotor cortex. Neuron, 41(2):301–307.
  14. spacy: Industrial-strength natural language processing in python.
  15. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. arXiv preprint arXiv:2201.07207.
  16. An introduction to statistical learning, volume 112. Springer.
  17. Mark Johnson. 2013. The body in the mind: The bodily basis of meaning, imagination, and reason. University of Chicago press.
  18. Age-of-acquisition ratings for 30,000 english words. Behavior research methods, 44(4):978–990.
  19. Albert: A lite bert for self-supervised learning of language representations. In Proceedings of ICRL 2020.
  20. Testing the ability of language models to interpret figurative language. In Proceedings of the 2022 Conference of the NAACL. ACL.
  21. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  22. Jean M Mandler. 1992. How to build a baby: Ii. conceptual primitives. Psychological review, 99(4):587.
  23. Jean M Mandler and Cristóbal Pagán Cánovas. 2014. On defining image schemas. Language and cognition, 6(4):510–532.
  24. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  25. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1–67.
  26. Armand Stefan Rotaru. 2020. Andi@ concretext: Predicting concreteness in context for english and italian using distributional models and behavioural norms (short paper). In EVALITA proceedings. CEUR.org.
  27. Robustlr: Evaluating robustness to logical perturbation in deductive reasoning. In Proceedings of the 2022 Conference on EMNLP.
  28. Sentential context modulates the involvement of the motor cortex in action language processing: An fmri study. Frontiers in human neuroscience, 7:100.
  29. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10740–10749.
  30. Effects of relative embodiment in lexical and semantic processing of verbs. Acta psychologica, 149:32–39.
  31. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615.
  32. IMPLI: Investigating NLI models’ performance on figurative language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5375–5388, Dublin, Ireland. ACL.
  33. Embodied bert: A transformer model for embodied, language-guided visual task completion. In Novel Ideas in Learning-to-Learn through Interaction at EMNLP2021.
  34. Language (re) modelling: Towards embodied language understanding. pages 6268–6281.
  35. Natural language processing with transformers. O’Reilly Media, Inc.
  36. Subtlex-uk: A new and improved word frequency database for british english. Quarterly journal of experimental psychology, 67(6):1176–1190.
  37. Attention is all you need. Advances in neural information processing systems, 30.
  38. Emergent abilities of large language models. Transactions on Machine Learning Research. Survey Certification.
  39. On the paradox of learning to reason from data. arXiv preprint arXiv:2205.11502.
  40. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.
  41. Rolf A Zwaan. 2014. Embodiment and language comprehension: Reframing the discussion. Trends in cognitive sciences, 18(5):229–234.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now