VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers (2203.17247v3)

Abstract: Breakthroughs in transformer-based models have revolutionized not only the NLP field, but also vision and multimodal systems. However, although visualization and interpretability tools have become available for NLP models, internal mechanisms of vision and multimodal transformers remain largely opaque. With the success of these transformers, it is increasingly critical to understand their inner workings, as unraveling these black-boxes will lead to more capable and trustworthy models. To contribute to this quest, we propose VL-InterpreT, which provides novel interactive visualizations for interpreting the attentions and hidden representations in multimodal transformers. VL-InterpreT is a task agnostic and integrated tool that (1) tracks a variety of statistics in attention heads throughout all layers for both vision and language components, (2) visualizes cross-modal and intra-modal attentions through easily readable heatmaps, and (3) plots the hidden representations of vision and language tokens as they pass through the transformer layers. In this paper, we demonstrate the functionalities of VL-InterpreT through the analysis of KD-VLP, an end-to-end pretraining vision-language multimodal transformer-based model, in the tasks of Visual Commonsense Reasoning (VCR) and WebQA, two visual question answering benchmarks. Furthermore, we also present a few interesting findings about multimodal transformer behaviors that were learned through our tool.

An Overview of VL-InterpreT: Enhancing Interpretability of Vision-Language Transformers

The paper presents VL-InterpreT, an interactive visualization tool tailored for investigating the inner workings of vision-language (VL) transformers. The widespread success of transformer-based architectures across various fields, particularly in multimodal applications, underscores the necessity for robust interpretability tools for these complex models. Although interpretability tools for NLP transformers have seen a surge, similar advancements for VL transformers lag behind. VL-InterpreT seeks to bridge this gap by providing a comprehensive platform to analyze the attention mechanisms and hidden states of these models, offering insights into their operational behaviors.

Core Features of VL-InterpreT

VL-InterpreT is characterized by its task-agnostic design, capable of interfacing with different VL transformer models. The tool provides visualizations that elucidate cross-modal and intra-modal attentions, unveiling the interactions between vision and language components. Key capabilities include:

1. Attention Visualization: Users can track and visualize attention scores for both vision and language tokens across all layers and heads. The system delineates different attention components, namely language-to-language (L2L), vision-to-language (V2L), language-to-vision (L2V), and vision-to-vision (V2V), each core to understanding how the model integrates and processes input data.
2. Hidden State Tracking: The tool facilitates interactive exploration of hidden state representations, using dimensionality reduction to visualize the progression of token representations through the transformer layers. This feature aids in identifying how visual and textual tokens relate conceptually.
3. Attention Head Summary: Statistical metrics over attention heads provide a high-level view of attention dynamics throughout the model. Users can define custom metrics to explore modality-specific interactions, enabling deeper insights into model specialization.

Insights and Findings

The paper provides empirical analyses using the KD-VLP model and discusses its performance on Visual Commonsense Reasoning (VCR) and WebQA benchmarks. Notably, the authors use VL-InterpreT to:

• Demonstrate the previously opaque alignment between visual and linguistic concepts via representative examples in the VCR dataset. Evidence suggests that VL transformers can capture semantically rich correspondences between different modalities.
• Analyze model predictions on the WebQA dataset, highlighting cases of both correct inference and erroneous conclusions, allowing users to pinpoint specific regions contributing to a model's decision-making process. This utility is essential for diagnosing and rectifying model failures.

Implications and Future Directions

The introduction of VL-InterpreT marks a significant advancement in the interpretability of multimodal transformer models. By providing a centralized platform for probing and visualizing model components, researchers can gain a more profound understanding of model decisions, contribute to more robust and transparent AI systems, and potentially enhance model architectures through informed insights.

VL-InterpreT's future iterations could incorporate extended functionalities such as aggregated metrics across samples for generalized insights and provide users the capability to define more nuanced metrics that focus on specific interpretability aspects. Furthermore, integrating the ability to analyze real-time manipulations of inputs will bolster the tool's applicability in research and practical domains, promoting the development of trust in AI systems that interact with diverse data modalities. As the ecosystem of transformer models continues to evolve, VL-InterpreT paves the way for enhanced interpretability and understanding in the field of vision-language processing.