Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods (2212.06872v5)

Abstract: In order to gain insights about the decision-making of different visual recognition backbones, we propose two methodologies, sub-explanation counting and cross-testing, that systematically applies deep explanation algorithms on a dataset-wide basis, and compares the statistics generated from the amount and nature of the explanations. These methodologies reveal the difference among networks in terms of two properties called compositionality and disjunctivism. Transformers and ConvNeXt are found to be more compositional, in the sense that they jointly consider multiple parts of the image in building their decisions, whereas traditional CNNs and distilled transformers are less compositional and more disjunctive, which means that they use multiple diverse but smaller set of parts to achieve a confident prediction. Through further experiments, we pinpointed the choice of normalization to be especially important in the compositionality of a model, in that batch normalization leads to less compositionality while group and layer normalization lead to more. Finally, we also analyze the features shared by different backbones and plot a landscape of different models based on their feature-use similarity.

• The paper introduces sub-explanation counting and cross-testing to systematically compare decision-making mechanisms in Transformers and CNNs.
• The study finds that Transformers and ConvNeXt exhibit compositional behaviors while CNNs and distilled Transformers show disjunctive traits influenced by normalization techniques.
• The results suggest that understanding these mechanisms can guide the design of more robust image recognition models and inform future architecture improvements.

Comparative Analysis of Decision-Making in Transformers and CNNs Using Explanation Methods

The paper, titled "Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods," explores the intricacies of how different visual recognition backbones such as Transformers and Convolutional Neural Networks (CNNs) make decisions. The authors introduce a systematic approach employing deep explanation algorithms to uncover the underlying mechanisms of various models broadly used in image recognition tasks.

Methodology and Findings

The authors propose two novel methodologies: sub-explanation counting and cross-testing. These methods aim to reveal the models' properties in terms of compositionality and disjunctivism, by leveraging data-driven explanations across datasets.

1. Sub-explanation Counting: This methodology looks into how models perform with partial evidences by counting sub-explanations, which are subsets of Minimal Sufficient Explanations (MSEs), the minimal set of image patches necessary to maintain confident predictions. The findings suggest that:
   • Transformers and ConvNeXt exhibit more compositional behavior, indicating that they integrate multiple parts of an image for decision-making.
   • Conversely, traditional CNNs and distilled Transformers display disjunctive traits, relying on fewer but relevant sets of image patches.
   • Importantly, the type of normalization—batch normalization versus group and layer normalization—significantly influences compositionality. Batch normalization leans towards less compositional and more disjunctive behavior, while layer and group normalization support compositional models.
2. Cross-Testing: This approach evaluates the similarity in feature use across different models by testing if masks from explanations of one model are pertinent to another. Results demonstrate that:
   • Different model architectures utilize distinct visual features. However, distillation appears to bring Transformers' feature use closer to CNNs.
   • Transformers, CNNs, and hybrid models like ConvNeXt occupy distinct positions in the feature-use landscape.

Implications

The implications of this research are multifaceted. Practically, understanding these decision-making mechanisms helps in selecting the appropriate network for specific applications, potentially leading to improved image recognition models that are robust to occlusions and adversarial attacks. Theoretically, the findings suggest that compositional models might be better suited for tasks requiring a holistic view of images, such as object detection and segmentation.

Moreover, the insights regarding normalization techniques could guide future model design, pointing towards an exploration of combinations that could blend the advantages of batch and group normalizations. There is also a potential pathway for exploiting ensembles across these model families to harness distinct decision-making processes.

Speculation on Future Directions

Future work could explore the integration of these findings into the design of new architectures that harness both compositional and disjunctive strategies, potentially leading to more robust visual recognition systems. Additionally, these methodologies could be extended to other domains beyond image recognition, such as video analysis or natural language processing, where understanding the decision-making process is equally critical.

In conclusion, this paper's comprehensive analysis of decision-making mechanisms within different neural network backbones provides valuable insights that could inform both practical model applications and future theoretical advancements in AI.