B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable (2411.00715v2)

Abstract: B-cos Networks have been shown to be effective for obtaining highly human interpretable explanations of model decisions by architecturally enforcing stronger alignment between inputs and weight. B-cos variants of convolutional networks (CNNs) and vision transformers (ViTs), which primarily replace linear layers with B-cos transformations, perform competitively to their respective standard variants while also yielding explanations that are faithful by design. However, it has so far been necessary to train these models from scratch, which is increasingly infeasible in the era of large, pre-trained foundation models. In this work, inspired by the architectural similarities in standard DNNs and B-cos networks, we propose 'B-cosification', a novel approach to transform existing pre-trained models to become inherently interpretable. We perform a thorough study of design choices to perform this conversion, both for convolutional neural networks and vision transformers. We find that B-cosification can yield models that are on par with B-cos models trained from scratch in terms of interpretability, while often outperforming them in terms of classification performance at a fraction of the training cost. Subsequently, we apply B-cosification to a pretrained CLIP model, and show that, even with limited data and compute cost, we obtain a B-cosified version that is highly interpretable and competitive on zero shot performance across a variety of datasets. We release our code and pre-trained model weights at https://github.com/shrebox/B-cosification.

• The paper introduces a novel B-cosification technique that converts pre-trained DNNs into inherently interpretable models without retraining.
• It demonstrates that B-cosified models preserve competitive accuracy while achieving up to 9x faster training and improved interpretability.
• The method is successfully applied to models like CLIP, showing robust zero-shot performance and effective localization.

Overview of "B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable"

The paper, "B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable," by Arya, Rao, Böhle, and Schiele, introduces a method, termed B-cosification, to convert pre-trained deep neural networks (DNNs) into inherently interpretable models. This method leverages the architectural similarities between conventional DNNs and alignment-based B-cos networks, aiming to preserve performance while enhancing interpretability through a cost-effective transformation process.

Motivation and Background

Standard neural networks, despite their strong performance on a variety of tasks, have drawbacks concerning interpretability. Traditional post-hoc explanation methods have been critiqued for their potential lack of faithfulness to the original models. B-cos networks address this by aligning inputs and weights to create inherently interpretable networks; however, such networks typically require training from scratch, which is computationally costly with the emergence of large, pre-trained foundational models.

Contribution

This research proposes B-cosification as a method that transforms existing pre-trained models into B-cos networks, thus combining the utility of pre-trained weights with the interpretability of B-cos networks. The key contributions of the paper include:

• B-cosification Technique: The proposed method allows pre-trained DNNs to be fine-tuned for interpretability without retraining from scratch, significantly reducing training costs.
• Empirical Results: B-cosified models maintain competitive accuracy akin to standard models while achieving interpretability scores comparable to those of B-cos models.
• Foundational Model Applications: B-cosification was successfully applied to a pre-trained CLIP model, showing promising zero-shot performance and interpretability enhancements.

Key Experiments and Results

The authors conducted extensive experiments on convolutional neural networks (CNNs) and vision transformers (ViTs), demonstrating that B-cosified models outperform both standard and B-cos models in several cases. The results include:

• Classification Performance: B-cosified models often outperform their B-cos counterparts, with some models reaching accuracy parity with standard pre-trained models at reduced computational costs.
• Interpretability: The Grid Pointing Game results indicate that B-cosified models provide substantial interpretability improvements over standard networks and achieve localization performances similar to B-cos models.
• Efficiency Gains: The transformation process achieves significant reductions in training times, with up to 9x speedup in specific architectures.

Implications and Future Directions

B-cosification offers a practical and efficient approach to incorporating interpretability into widely used pre-trained models, which is crucial for building trust in AI systems and facilitating debugging and understanding of misclassifications. The method highlights the potential of existing architectures to approximate inherently interpretable models through minimal architectural modifications.

In terms of future developments, this work opens avenues to explore the integration of interpretability into even larger models and different domains, reinforcing the importance of aligning model architectures with interpretable frameworks to improve machine learning transparency. Furthermore, advancing this technique could complement other interpretability approaches, such as concept bottleneck or prototype-based models, to provide a more comprehensive understanding of model decision-making processes.