Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models (2210.14328v1)

Abstract: Structural probing work has found evidence for latent syntactic information in pre-trained LLMs. However, much of this analysis has focused on monolingual models, and analyses of multilingual models have employed correlational methods that are confounded by the choice of probing tasks. In this study, we causally probe multilingual LLMs (XGLM and multilingual BERT) as well as monolingual BERT-based models across various languages; we do this by performing counterfactual perturbations on neuron activations and observing the effect on models' subject-verb agreement probabilities. We observe where in the model and to what extent syntactic agreement is encoded in each language. We find significant neuron overlap across languages in autoregressive multilingual LLMs, but not masked LLMs. We also find two distinct layer-wise effect patterns and two distinct sets of neurons used for syntactic agreement, depending on whether the subject and verb are separated by other tokens. Finally, we find that behavioral analyses of LLMs are likely underestimating how sensitive masked LLMs are to syntactic information.