WT5?! Training Text-to-Text Models to Explain their Predictions (2004.14546v1)

Abstract: Neural networks have recently achieved human-level performance on various challenging NLP tasks, but it is notoriously difficult to understand why a neural network produced a particular prediction. In this paper, we leverage the text-to-text framework proposed by Raffel et al.(2019) to train LLMs to output a natural text explanation alongside their prediction. Crucially, this requires no modifications to the loss function or training and decoding procedures -- we simply train the model to output the explanation after generating the (natural text) prediction. We show that this approach not only obtains state-of-the-art results on explainability benchmarks, but also permits learning from a limited set of labeled explanations and transferring rationalization abilities across datasets. To facilitate reproducibility and future work, we release our code use to train the models.

Overview of "WT5?! Training Text-to-Text Models to Explain their Predictions"

The paper presents a novel approach to enhancing the interpretability of text-to-text LLMs by training them to provide natural language explanations alongside their predictions. This work is situated within the context of the text-to-text framework, where all NLP tasks are reformulated as input-output text pairs, facilitating a unified modeling approach. The authors leverage the Text-to-Text Transfer Transformer (T5) model and propose an innovative method to append explanations to predictions without modifying the underlying loss or training architectures. This approach aligns with the increasing demand for interpretable artificial intelligence models, particularly in sensitive applications.

Key Contributions

1. Unified Training Framework: The method integrates explanation generation directly into the text-to-text paradigm, requiring only the addition of an "explanation" prompt in the input text to guide the model during both training and inference. This simplicity eschews the need for complex architectural changes or additional components.
2. Empirical Evaluation: The authors demonstrated the effectiveness of their approach across several explainability benchmarks, achieving state-of-the-art results. Evaluations span across datasets like e-SNLI, CoS-E, and Movie Reviews, where the model generates coherent and contextually relevant explanations that align with human reasoning.
3. Semi-supervised and Transfer Learning: The model's ability to learn from partially labeled datasets (i.e., datasets with limited annotated explanations) is particularly intriguing. Additionally, the ability to transfer explanation generation capabilities across tasks and domains without extensive retraining suggests a robustness and flexibility in the approach.

Numerical and Experimental Insights

The model, WT5-11B, surpasses previous state-of-the-art models on various explainability benchmarks in terms of accuracy and human-evaluated explanation quality. For instance, in the e-SNLI dataset, WT5-11B achieves a significantly higher BLEU score compared to earlier models, indicating superior ability in generating human-like explanations. Similarly, high human evaluation scores suggest that WT5-11B's explanations are perceived as both plausible and informative by human annotators.

Theoretical and Practical Implications

Theoretically, the work underscores the potential of leveraging large pre-trained models in novel ways to address pressing issues like model interpretability. By embedding the capability to generate natural language explanations, the work aligns with a growing movement toward more transparent AI systems.

Practically, this approach could be widely beneficial across various sectors requiring clear decision rationales from AI systems, such as healthcare diagnostics, legal decision-making processes, and customer service chatbots. The model's ability to generate explanations can aid in debugging model decisions, enhancing user trust and facilitating a deeper understanding of model behavior.

Future Directions

Future work may focus on further enhancing the generalizability of generated explanations, particularly in cross-domain and multilingual settings. Additionally, integrating more stringent evaluation metrics to assess the fidelity of explanations to the model's actual decision-making processes could be explored. A critical avenue is also to paper the impact of explanation generation on model performance under adversarial settings.

In conclusion, this paper provides a significant contribution to the field of interpretable AI by introducing a method to equip LLMs with the ability to articulate their decision-making processes. The method's simplicity, effectiveness, and flexibility mark it as a promising approach for developing more transparent and trustworthy AI systems.