Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Navigating Brain Language Representations: A Comparative Analysis of Neural Language Models and Psychologically Plausible Models (2404.19364v1)

Published 30 Apr 2024 in cs.CL

Abstract: Neural LLMs, particularly large-scale ones, have been consistently proven to be most effective in predicting brain neural activity across a range of studies. However, previous research overlooked the comparison of these models with psychologically plausible ones. Moreover, evaluations were reliant on limited, single-modality, and English cognitive datasets. To address these questions, we conducted an analysis comparing encoding performance of various neural LLMs and psychologically plausible models. Our study utilized extensive multi-modal cognitive datasets, examining bilingual word and discourse levels. Surprisingly, our findings revealed that psychologically plausible models outperformed neural LLMs across diverse contexts, encompassing different modalities such as fMRI and eye-tracking, and spanning languages from English to Chinese. Among psychologically plausible models, the one incorporating embodied information emerged as particularly exceptional. This model demonstrated superior performance at both word and discourse levels, exhibiting robust prediction of brain activation across numerous regions in both English and Chinese.

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. (2017). Experiential, distributional and dependency-based word embeddings have complementary roles in decoding brain activity. arXiv preprint arXiv:1711.09285.
  2. (2022). Predictive coding or just feature discovery? an alternative account of why language models fit brain data. Neurobiology of Language, 1–16.
  3. (2019). Robust evaluation of language–brain encoding experiments. In International conference on computational linguistics and intelligent text processing (pp. 44–61).
  4. (2016). Toward a brain-based componential semantic representation. Cognitive neuropsychology, 33(3-4), 130–174.
  5. (2011). The neurobiology of semantic memory. Trends in cognitive sciences, 15(11), 527–536.
  6. (2021). Decoding word embeddings with brain-based semantic features. Computational Linguistics, 47(3), 663–698.
  7. (2005). Modality-constrained statistical learning of tactile, visual, and auditory sequences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(1), 24.
  8. Damasio, A. R.  (1989). Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition. Cognition, 33(1-2), 25–62.
  9. (2019). fmriprep: a robust preprocessing pipeline for functional mri. Nature methods, 16(1), 111–116.
  10. (2022). Decoding the information structure underlying the neural representation of concepts. Proceedings of the National Academy of Sciences, 119(6), e2108091119.
  11. (2017). Word predictability and semantic similarity show distinct patterns of brain activity during language comprehension. Language, Cognition and Neuroscience, 32(9), 1192–1203.
  12. (2023). Different computational relations in language are captured by distinct brain systems. Cerebral Cortex, 33(4), 997–1013.
  13. (2017). A map of abstract relational knowledge in the human hippocampal–entorhinal cortex. elife, 6, e17086.
  14. Glenberg, A. M.  (1997). What memory is for. Behavioral and brain sciences, 20(1), 1–19.
  15. (2018). Simple co-occurrence statistics reproducibly predict association ratings. Cognitive science, 42(7), 2287–2312.
  16. (2019). Cognival: A framework for cognitive word embedding evaluation. arXiv preprint arXiv:1909.09001.
  17. (2018). Zuco, a simultaneous eeg and eye-tracking resource for natural sentence reading. Scientific data, 5(1), 1–13.
  18. (2016). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532(7600), 453–458.
  19. (2020). Abstract representations of events arise from mental errors in learning and memory. Nature communications, 11(1), 2313.
  20. (2003). Eye movements reveal the on-line computation of lexical probabilities during reading. Psychological science, 14(6), 648–652.
  21. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
  22. (2008). Predicting human brain activity associated with the meanings of nouns. science, 320(5880), 1191–1195.
  23. (2017). Second-order word embeddings from nearest neighbor topological features. arXiv preprint arXiv:1705.08488.
  24. (2022). Neural language taskonomy: Which nlp tasks are the most predictive of fmri brain activity? arXiv preprint arXiv:2205.01404.
  25. Packard, J. L.  (1999). Lexical access in chinese speech comprehension and production. Brain and Language, 68(1-2), 89–94.
  26. (2022). Neural language models are not born equal to fit brain data, but training helps. arXiv preprint arXiv:2207.03380.
  27. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (emnlp) (pp. 1532–1543).
  28. (2018). Toward a universal decoder of linguistic meaning from brain activation. Nature communications, 9(1), 963.
  29. (2018). A novel co-occurrence-based approach to predict pure associative and semantic priming. Psychonomic Bulletin & Review, 25, 1488–1493.
  30. (2001). The acquisition of language by children. Proceedings of the National Academy of Sciences, 98(23), 12874–12875.
  31. (2013). Neural representations of events arise from temporal community structure. Nature neuroscience, 16(4), 486–492.
  32. (2021). The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45), e2105646118.
  33. (1990). The representation of text-based and situation-based information in discourse comprehension. Journal of Memory and Language, 29(1), 119–132.
  34. Tang, J.  (2021). Differences between chinese and english thinking from the use of nouns in chinese and english. In 2nd international conference on language, art and cultural exchange (iclace 2021) (pp. 20–25).
  35. (2022). An erp megastudy of chinese word recognition. Psychophysiology, 59(11), e14111.
  36. (2024). Computational models to study language processing in the human brain: A survey. arXiv preprint arXiv:2403.13368.
  37. (2022). A synchronized multimodal neuroimaging dataset for studying brain language processing. Scientific Data, 9(1), 590.
  38. (2023). A large dataset of semantic ratings and its computational extension. Scientific Data, 10(1), 106.
  39. (2022). An fmri dataset for concept representation with semantic feature annotations. Scientific Data, 9(1), 721.
  40. (2022). The database of eye-movement measures on words in chinese reading. Scientific Data, 9(1), 411.
  41. (2020). Connecting concepts in the brain by mapping cortical representations of semantic relations. Nature communications, 11(1), 1877.
  42. (2023). A comprehensive neural and behavioral task taxonomy method for transfer learning in nlp. In Findings of the association for computational linguistics: Ijcnlp-aacl 2023 (findings) (pp. 233–241).
  43. (2024). Mulcogbench: A multi-modal cognitive benchmark dataset for evaluating chinese and english computational language models. arXiv preprint arXiv:2403.01116.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yunhao Zhang (19 papers)
  2. Shaonan Wang (19 papers)
  3. Xinyi Dong (3 papers)
  4. Jiajun Yu (4 papers)
  5. Chengqing Zong (65 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets