Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain) (1905.11833v4)

Abstract: Neural networks models for NLP are typically implemented without the explicit encoding of language rules and yet they are able to break one performance record after another. This has generated a lot of research interest in interpreting the representations learned by these networks. We propose here a novel interpretation approach that relies on the only processing system we have that does understand language: the human brain. We use brain imaging recordings of subjects reading complex natural text to interpret word and sequence embeddings from 4 recent NLP models - ELMo, USE, BERT and Transformer-XL. We study how their representations differ across layer depth, context length, and attention type. Our results reveal differences in the context-related representations across these models. Further, in the transformer models, we find an interaction between layer depth and context length, and between layer depth and attention type. We finally hypothesize that altering BERT to better align with brain recordings would enable it to also better understand language. Probing the altered BERT using syntactic NLP tasks reveals that the model with increased brain-alignment outperforms the original model. Cognitive neuroscientists have already begun using NLP networks to study the brain, and this work closes the loop to allow the interaction between NLP and cognitive neuroscience to be a true cross-pollination.

Interpreting and Improving NLP Models through Brain Activity Analysis

The paper by Mariya Toneva and Leila Wehbe presents an innovative approach to interpreting NLP models by leveraging insights from neuroscience, specifically brain activity recorded during language processing. This method hinges on aligning the internal representations of NLP models with brain imaging data, thus using the human brain as a benchmark for interpreting and potentially improving NLP systems.

Methodology and Analysis

The paper utilizes brain recordings from functional Magnetic Resonance Imaging (fMRI) and Magnetoencephalography (MEG) while subjects read complex texts. These recordings are used to interpret the embeddings of neural network models such as ELMo, USE, BERT, and Transformer-XL. The focus is on examining how these models' internal representations vary with layer depth, context size, and the type of attention mechanism employed.

The research highlights three main findings:

1. Contextual Representation Variability: There is a variation in how context-related representations are captured across different NLP models. The transformer models showed a notable interaction between layer depth and context length, as well as between layer depth and attention type.
2. Brain Alignment for Improved NLP Performance: The authors propose that modifying BERT to better align with brain recordings could enhance its language comprehension capabilities. Through syntactic NLP tasks, it was demonstrated that the brain-aligned version of BERT outperformed the original model.
3. Cross-Pollination Potential: The paper posits that the synergy between NLP models and cognitive neuroscience can lead to reciprocal advancements - NLP models can aid in understanding brain functions, while insights from neuroscience can inform and improve NLP model designs.

Practical and Theoretical Implications

The practical implications of this research are significant for the development of more robust and linguistically competent NLP models. By aligning NLP model architectures with the neural processes of language understanding, it is possible to enhance model capabilities in language tasks that involve complex interactions, such as syntactic parsing and contextual comprehension.

Theoretically, this approach opens new avenues for exploring how artificial models can mirror the brain's language processing. It suggests that the middle layers of transformer models capture the most brain-relevant context information, which could guide future modifications of model architectures for enhanced performance.

Speculation on Future Developments

This paper marks a crucial step towards integrating cognitive neuroscience insights into the field of NLP, suggesting several future research directions. Potential developments could include:

• Refining alignment techniques to further dissect NLP models' representations and their neural correlates.
• Extending this framework to investigate higher-order cognitive processes beyond language, such as reasoning or decision-making, within NLP models.
• Developing a comprehensive interpretative framework using naturalistic brain imaging data that regularly informs the iterative design of NLP models.

Overall, the cross-disciplinary approach championed in this paper underscores the importance of merging computational linguistics with neuroscience to forge a deeper understanding of both artificial and human language comprehension mechanisms.