On the Evaluation Consistency of Attribution-based Explanations (2407.19471v1)

Abstract: Attribution-based explanations are garnering increasing attention recently and have emerged as the predominant approach towards \textit{eXplanable Artificial Intelligence}~(XAI). However, the absence of consistent configurations and systematic investigations in prior literature impedes comprehensive evaluations of existing methodologies. In this work, we introduce {Meta-Rank}, an open platform for benchmarking attribution methods in the image domain. Presently, Meta-Rank assesses eight exemplary attribution methods using six renowned model architectures on four diverse datasets, employing both the \textit{Most Relevant First} (MoRF) and \textit{Least Relevant First} (LeRF) evaluation protocols. Through extensive experimentation, our benchmark reveals three insights in attribution evaluation endeavors: 1) evaluating attribution methods under disparate settings can yield divergent performance rankings; 2) although inconsistent across numerous cases, the performance rankings exhibit remarkable consistency across distinct checkpoints along the same training trajectory; 3) prior attempts at consistent evaluation fare no better than baselines when extended to more heterogeneous models and datasets. Our findings underscore the necessity for future research in this domain to conduct rigorous evaluations encompassing a broader range of models and datasets, and to reassess the assumptions underlying the empirical success of different attribution methods. Our code is publicly available at \url{https://github.com/TreeThree-R/Meta-Rank}.

• The paper introduces Meta-Rank, a framework that benchmarks eight attribution methods across diverse models and datasets.
• It reveals that attribution methods exhibit variable performance based on model architecture and evaluation settings.
• Standardized protocols and robust ranking metrics offer actionable insights for enhancing explainability in AI systems.

Evaluation Consistency in Attribution-Based Explanations

The paper "On the Evaluation Consistency of Attribution-based Explanations" presents a significant methodological contribution to the field of Explainable Artificial Intelligence (XAI) by focusing on the evaluation resilience of attribution methods. Attribution methods have gained prominence for their ability to generate saliency maps that highlight input regions relevant to model predictions. Despite their rise in popularity, existing literature lacks consistent evaluation practices due to the absence of agreed-upon configurations, engendering disparate performance claims. This paper introduces a structured approach, termed Meta-Rank, aimed at benchmarking attribution methods across diverse settings.

Key Contributions

1. Meta-Rank Benchmark: The authors propose Meta-Rank, an open platform to benchmark attribution methods specifically in the image domain. Meta-Rank thoroughly examines eight attribution methods across six model architectures and four datasets using Most Relevant First (MoRF) and Least Relevant First (LeRF) evaluation protocols. The comparative benchmark is remarkable because it combines results across different configurations to attain a robust ranking of methods. Unlike previous segmented evaluations, Meta-Rank provides a holistic rank symbolizing overall performance across settings.
2. Findings on Evaluation Variability: Through extensive experimentation, the paper uncovers several non-trivial insights:
   • Attribution methods often exhibit varying effectiveness depending on evaluative settings such as model architecture and dataset type.
   • The ranking of these methods tends to maintain consistency through different training checkpoints, suggesting the stability of ranking after model convergence.
   • Existing consistent evaluation approaches, when scaled to diverse models and datasets, often yield outcomes no better than baseline methods.
3. Standardized Settings: By introducing standardized evaluation settings involving multiple datasets, models, and protocols, the paper advances the state-of-the-art in attribution evaluation. It critically investigates the impact of these factors, corroborating the need for multiperspective benchmarking.

Implications and Future Directions

The work elucidates the necessity for a comprehensive evaluative framework in XAI, particularly for attribution methods. The standardization presented through Meta-Rank addresses a gap, facilitating reliable cross-method comparisons and enabling robust insights into method performances. Future work may extend this framework to include evaluation metrics considering different dimensions of faithfulness, offering a nuanced understanding of attribution method competence.

From a methodological standpoint, while the significant computational efficiency of Meta-Rank highlights its practicality, further optimization incorporating parallel computing could enhance its scalability. Exploring methods to mitigate the missingness bias inherent in feature ablation, possibly through innovative pixel manipulation strategies, would be a persuasive future development.

In summary, this paper offers a foundational benchmark for attribution-based explanations in AI, setting a precedent for methodological rigor and consistency in the evaluation of explainability methods. Such an endeavor is indispensable for the advancement of transparent and accountable AI systems.