Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (2406.15955v3)

Abstract: Though vision transformers (ViTs) have achieved state-of-the-art performance in a variety of settings, they exhibit surprising failures when performing tasks involving visual relations. This begs the question: how do ViTs attempt to perform tasks that require computing visual relations between objects? Prior efforts to interpret ViTs tend to focus on characterizing relevant low-level visual features. In contrast, we adopt methods from mechanistic interpretability to study the higher-level visual algorithms that ViTs use to perform abstract visual reasoning. We present a case study of a fundamental, yet surprisingly difficult, relational reasoning task: judging whether two visual entities are the same or different. We find that pretrained ViTs fine-tuned on this task often exhibit two qualitatively different stages of processing despite having no obvious inductive biases to do so: 1) a perceptual stage wherein local object features are extracted and stored in a disentangled representation, and 2) a relational stage wherein object representations are compared. In the second stage, we find evidence that ViTs can learn to represent somewhat abstract visual relations, a capability that has long been considered out of reach for artificial neural networks. Finally, we demonstrate that failures at either stage can prevent a model from learning a generalizable solution to our fairly simple tasks. By understanding ViTs in terms of discrete processing stages, one can more precisely diagnose and rectify shortcomings of existing and future models.

Analysis of Vision Transformers in Relational Reasoning Tasks

The paper "Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects" offers a detailed investigation into the ability of Vision Transformers (ViTs) to perform tasks requiring relational reasoning—a domain where ViTs have historically shown limitations despite their successes in other vision tasks. The authors employ a mechanistic interpretability framework to dissect the processes ViTs use to determine visual relations, namely through a paper of same-different judgments, which are fundamental to abstract visual reasoning.

Key Contributions and Findings

The paper identifies that ViTs trained on relational reasoning tasks exhibit a novel two-stage processing mechanism consisting of a perceptual stage and a relational stage. This segmentation occurs without an inductive bias explicitly encoded in the network architecture.

1. Two-Stage Processing Pipeline:
   • Perceptual Stage: During this phase, ViTs extract and store local object features in disentangled representations. This suggests that these models can isolate object characteristics such as shape and color effectively.
   • Relational Stage: The subsequent phase involves comparing the object representations to make relational judgments. This stage demonstrates that ViTs can abstract some relational operations like assessing sameness or difference, a capability of particular interest given its debated feasibility in neural networks.
2. Relation Match-to-Sample (RMTS) Task: The authors design a synthetic RMTS task inspired by cognitive science to challenge the capacity of ViTs to handle abstract concepts of "sameness" and "difference." Success in RMTS requires generalization beyond memorized object attributes as that seen in ViTs, distinguishing their learned behavior from associative memorization.
3. Disentangled Representations: The empirical evidence provided using Distributed Alignment Search (DAS) confirms that ViTs develop disentangled object representations, which decisively contributes to their relational reasoning capabilities. This separation of object properties enhances the ability of ViTs to generalize across different tasks.
4. Processing Sequence Similarities: Interestingly, the identified processing stages within ViTs—first forming image representations followed by relational reasoning—mirror the sequence found in biological vision systems, drawing a parallel to the hierarchical processing of visual information in the human brain.

Implications and Future Directions

The paper's findings deepen our understanding of how ViTs, despite their architectural simplicity relative to human cognition, can approximate aspects of human-like relational reasoning. The authors highlight a potential path to improving artificial systems' understanding of complex visual scenes, emphasizing disentangled representations as a critical component for generalization and relational reasoning.

From a theoretical perspective, this work challenges preconceived limitations associated with neural networks' ability to implement abstract computational tasks. Practically, the diagnostic framework provided for analyzing ViTs' latent representations could inform the development of more robust models capable of addressing a broad spectrum of vision tasks, extending beyond routine classification and detection to more intricate reasoning challenges.

This research opens avenues for further examination and refinement in both model training methodologies and architectural innovations that can harness the processing potential seen in biological systems. There is significant scope for future exploration into fine-tuning techniques or hybrid architectures that emphasize relational reasoning in visual contexts. Overall, this paper represents a vital step in bridging the gap between neural computation and natural visual intelligence.