UnNatural Language Inference (2101.00010v2)

Abstract: Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to know humanlike syntax, at least to some extent. We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i.e. they are largely invariant to random word-order permutations. This behavior notably differs from that of humans; we struggle with ungrammatical sentences. To measure the severity of this issue, we propose a suite of metrics and investigate which properties of particular permutations lead models to be word-order invariant. In the MNLI dataset, for example, we find almost all (98.7%) examples contain at least one permutation which elicits the gold label. Models are sometimes even able to assign gold labels to permutations that they originally failed to predict correctly. We provide a comprehensive empirical evaluation of this phenomenon, and further show that this issue exists for both Transformers and pre-Transformer RNN / ConvNet based encoders, as well as across multiple languages (English and Mandarin Chinese). Our code and data are available at https://github.com/facebookresearch/unlu.

Overview of UnNatural Language Inference

The paper "UnNatural Language Inference" by Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, and Adina Williams addresses a significant aspect of NLP: the syntactic capabilities of state-of-the-art Natural Language Understanding (NLU) and Natural Language Inference (NLI) models. This paper provides an empirical investigation into whether models that perform exceptionally well in various NLU tasks truly understand syntax in a human-like manner.

Key Findings and Methodology

The authors reveal that current state-of-the-art NLI models, including transformer-based architectures like RoBERTa, BART, GPT-2, and GPT-3, exhibit a surprising insensitivity to word order. The paper challenges the common assumption that these models capture syntactic structures akin to those understood by humans. The paper demonstrates that NLI models often assign the same labels to permuted versions of hypothesis-premise pairs as they do to the original, grammatically correct pairs. This behavior starkly contrasts with human performance, as humans typically struggle with understanding ungrammatical sequences.

The authors introduce a suite of permutation metrics to quantify this insensitivity across various NLI datasets, including MNLI, SNLI, ANLI, and the OCNLI dataset for Mandarin Chinese. These metrics allow an assessment of how likely a model is to produce the correct label on randomly permuted sentences. The results are consistent across various model architectures, including pre-transformer RNNs and ConvNet-based encoders, indicating the pervasiveness of the issue.

Theoretical and Practical Implications

The findings have severe implications for the claims about the syntactic capabilities of these advanced NLP models. The high degree of permutation acceptance suggests that models might be relying more on superficial cues and individual word tokens rather than the syntactic meanings traditionally held by sentences. This raises questions about the extent to which these systems genuinely understand natural language syntax or semantics.

From a practical standpoint, the paper advocates for the development of NLI models that respect syntactic order, paralleling human language understanding. The authors propose a maximum entropy-based method to mitigate the issue, aiming to train models to become more sensitive to word order and thereby potentially enhance their interpretative capabilities.

Future Directions

The paper opens various avenues for future research. There is a need to further explore and verify the syntactic signatures that models might be unintentionally learning. Additionally, developing training paradigms and architectures that genuinely capture linguistic structures can enhance the reliability of NLU systems in handling more complex, syntactically diverse inputs.

In conclusion, this paper critically evaluates the syntactic understanding currently afforded by leading NLP models, highlighting significant gaps in their human-like comprehension capabilities. It underscores the importance of not only benchmarking model performance but also ensuring their interpretability aligns with established linguistic principles. As NLP models increasingly influence applications across various domains, addressing these foundational issues will be crucial to their advancement and trustworthiness.