Evaluating and Aggregating Feature-based Model Explanations (2005.00631v1)

Abstract: A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.

• The paper introduces a framework that evaluates explanation functions using criteria like sensitivity, faithfulness, and complexity.
• It demonstrates that aggregating multiple explanation methods yields more robust and interpretable insights from complex models.
• The study details aggregation techniques, including convex combinations and centroid methods, to form consensus explanations that mirror human reasoning.

Evaluating and Aggregating Feature-based Model Explanations

The paper "Evaluating and Aggregating Feature-based Model Explanations" addresses a critical challenge in machine learning: explaining the predictions of complex models in a comprehensible manner. Feature-based model explanation functions assign importance scores to each input feature, highlighting their contribution to the model's output. Given the proliferation of explanation methods, this paper presents a framework for evaluating and aggregating these functions based on well-defined quantitative criteria.

Foundations of Feature-based Explanation

Feature-based explanations provide an essential tool for debugging and validating machine learning models, especially black-box models such as deep neural networks. Each input feature receives a score representing its importance in determining the model's output for a particular data point. Common approaches include gradient-based techniques and perturbation methods, with notable examples being Shapley values and Integrated Gradients.

Evaluation Criteria for Explanation Functions

The authors propose three key evaluation criteria for explanation functions: sensitivity, faithfulness, and complexity.

1. Sensitivity: Explanation functions should exhibit low sensitivity, meaning that small perturbations in the input should not lead to significant changes in the explanation if the model's output is stable. This property ensures robustness in the explanation.
2. Faithfulness: An explanation is faithful if it accurately represents the model's decision-making process. The feature importance scores should reflect the true reliance of the model on these features.
3. Complexity: Complexity pertains to the interpretability of the explanation. An explanation with lower complexity is preferable, as it is easier to comprehend and act upon, especially in high-dimensional settings where presenting an overwhelming number of significant features can be counterproductive.

Aggregation Framework

The paper introduces a novel aggregation framework for combining multiple explanation functions to form a consensus explanation. This aggregation aims to produce explanations that perform well across the proposed criteria, particularly by reducing sensitivity and complexity.

Techniques for Aggregation

1. Convex Combination: The paper discusses forming an explanation through a weighted sum of different explanation functions. This approach guarantees that the resulting explanation's sensitivity does not exceed that of the individual functions.
2. Centroid Aggregation: By calculating centroids with respect to distance metrics like $\ell_2$ and $\ell_1$, the authors propose using the mean and median of explanation vectors to achieve aggregate explanations with balanced sensitivity and complexity.
3. Gradient-Descent and Region-Shrinking Methods: These iterative techniques aim to create simpler explanations by incrementally refining the aggregated explanation function based on complexity measures.

Shapley Values and Aggregate Valuation of Antecedents

The authors derive a new explanation function, Aggregate Valuation of Antecedents (AVA), which leverages Shapley values from game theory. AVA explains a test point by aggregating Shapley value explanations from its nearest neighbors, minimizing sensitivity and enhancing the explanatory stability. This method mirrors human reasoning in decision-making, where context and similarity with past experiences inform judgments.

Implications and Future Directions

The proposed framework provides a structured methodology for evaluating and improving feature-based explanations—guiding practitioners in selecting and developing interpretable models. Future work could explore how these evaluative criteria interact or extend the aggregation framework to support multi-objective optimization, balancing all criteria simultaneously.

This research contributes valuable insights into model explanation, promoting transparency and trust in machine learning applications. By establishing rigorous evaluation metrics and offering aggregation techniques, the paper lays the groundwork for more interpretable AI systems, advancing both theoretical understanding and practical deployment in diverse fields.