MEG Evidence That Modality-Independent Conceptual Representations Encode Visual but Not Lexical Representations (2312.10916v2)

Abstract: The semantic knowledge stored in our brains can be accessed from different stimulus modalities. For example, a picture of a cat and the word "cat" both engage similar conceptual representations. While existing research has found evidence for modality-independent representations, their content remains unknown. Modality-independent representations could be abstract, or they might be perceptual or even lexical in nature. We used a novel approach combining word/picture cross-condition decoding with neural network classifiers that learned latent modality-independent representations from MEG data. We then compared these representations to models representing semantic, sensory, and lexical features. Results show that modality-independent representations are not strictly amodal; rather, they also contain visual representations. There was no evidence that lexical properties contributed to the representation of modality-independent concepts. These findings support the notion that perceptual processes play a fundamental role in encoding modality-independent conceptual representations. Conversely, lexical representations did not appear to partake in modality-independent semantic knowledge.