Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability (1910.06358v3)

Abstract: Explaining AI systems is fundamental both to the development of high performing models and to the trust placed in them by their users. The Shapley framework for explainability has strength in its general applicability combined with its precise, rigorous foundation: it provides a common, model-agnostic language for AI explainability and uniquely satisfies a set of intuitive mathematical axioms. However, Shapley values are too restrictive in one significant regard: they ignore all causal structure in the data. We introduce a less restrictive framework, Asymmetric Shapley values (ASVs), which are rigorously founded on a set of axioms, applicable to any AI system, and flexible enough to incorporate any causal structure known to be respected by the data. We demonstrate that ASVs can (i) improve model explanations by incorporating causal information, (ii) provide an unambiguous test for unfair discrimination in model predictions, (iii) enable sequentially incremental explanations in time-series models, and (iv) support feature-selection studies without the need for model retraining.

• The paper introduces Asymmetric Shapley Values (ASVs) as an extension of traditional Shapley values, relaxing the symmetry axiom to integrate causal knowledge via weighted permutations of feature orderings.
• ASVs improve AI transparency and trustworthiness by aligning explanations with causal data structures, which is vital for debugging models and deploying fair AI systems.
• Applications include assessing fairness under confounding factors, analyzing time-series data, and performing feature selection more efficiently.

Asymmetric Shapley Values: A Framework for Incorporating Causal Knowledge into Model-Agnostic Explainability

The paper introduces a novel extension to the established Shapley value framework for explainability in AI: Asymmetric Shapley Values (ASVs). In the context of model-agnostic explainability, the integration of causal information within Shapley values represents an advancement aimed at enhancing interpretability by adopting a more flexible approach than the traditional, stringent Shapley axioms, particularly the Axiom of Symmetry. Shapley values, rooted in cooperative game theory, have been a cornerstone in AI to allocate contributions of features towards a model's prediction. However, the standard framework does not consider causal dependencies among features, which can significantly affect the interpretation of the data and model predictions.

Summary of Contributions

The paper's primary contribution is the introduction of ASVs, which relax the symmetry axiom of Shapley values while maintaining other crucial properties such as efficiency, linearity, and nullity. This relaxation allows the integration of causal ordering of features, where causal ancestors and descendants can be treated differently within the model-explanation process.

1. Causal Integration: Traditional Shapley values do not account for causal dependencies. By relaxing the symmetry axiom, ASVs incorporate weighted permutations over feature orderings, aligning with known causal relationships. This enables causal insights without the requirement for a complete causal graph.
2. Diverse Applications: The paper outlines four significant applications of ASVs:
   • Model Explanation Enhancement: By embedding causal information into ASVs, explanations become more aligned with the underlying data structure.
   • Fairness Assessment: ASVs are used to evaluate unfair discrimination based on sensitive attributes even when confounding features are present.
   • Time-Series Data: Inherently ordered data, such as time-series, benefit from ASVs by providing a sequential explanation that considers temporal dependencies.
   • Feature Selection: ASVs facilitate assessing the predictive power of features without necessitating retraining, offering direct insights into model accuracies achievable with specific feature subsets.

Implications for AI Safety and Development

ASVs' development contributes significantly to AI's iterative advancement by enhancing transparency and trustworthiness through causal explainability. Incorporating causal understanding allows for nuanced evaluations of model decisions and aligns more closely with real-world data structures. This not only aids in debugging and refining models but also plays a critical role in ethical and fair AI deployment, particularly in scenarios where indirect discrimination must be identified and mitigated.

Future Directions

Looking forward, ASVs pave the way for broader studies in integrating causal inferences within machine learning explainability frameworks. Future work should explore:

• Scalability: Efficient computation of ASVs in large-scale models and high-dimensional data contexts.
• Causal Discovery Integration: Pairing ASVs with automated causal discovery methods to enhance robustness where causal structures are partially known or evolving.
• Interdisciplinary Applications: Leveraging ASVs across fields with complex causal interactions, such as healthcare or socio-economic systems, to derive actionable insights and foster trustworthy AI systems.

Overall, the introduction of ASVs represents a step towards more context-aware and causally grounded machine learning models, which could, in turn, drive better decision-making processes across various domains.