A Survey on the Explainability of Supervised Machine Learning (2011.07876v1)

Abstract: Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or fifinance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.

• The paper formalizes definitions and categorizes techniques into interpretable, surrogate, and explanation generation methods to enhance model transparency.
• The paper details methodologies for inherently interpretable and designed-for-interpretability models, including practical tools like LIME and SHAP.
• The paper discusses the integration of ontologies and regulatory insights to boost trust and guide future research in explainable AI.

A Survey on the Explainability of Supervised Machine Learning

The paper provides a comprehensive survey of techniques in the field of explainable Supervised Machine Learning (SML). It addresses the need for transparency and interpretability in AI models, especially in sensitive domains like healthcare and finance, where understanding model decisions is crucial.

Key Contributions

1. Formal Definitions and Categories: The paper introduces formal definitions for explainability and classifies approaches into interpretable models, surrogate models, and explanation generation techniques. This categorization helps in understanding the landscape of explainable SML and provides a framework for comparing different methods.
2. Interpretable Models: The paper discusses "interpretable by nature" models, such as decision trees and linear models, which inherently support explainability. It also highlights "interpretable by design" models, which incorporate specific design principles to enhance interpretability without compromising accuracy.
3. Surrogate Models: Various surrogate model approaches are reviewed, which involve creating an interpretable model that approximates the predictions of a black box model. The paper details global and local surrogate models, like LIME and SHAP, explaining their methodologies and applications.
4. Explanation Generation: The survey covers global and local explanation methods that directly interpret model predictions without approximating them through surrogates. Techniques such as feature importance, saliency maps, and counterfactual explanations are explored in depth.
5. Application of Ontologies and Data Quality: The integration of ontologies for improving data quality and aiding explainability is discussed. Ontologies can enhance model reliability by embedding domain knowledge, thus helping in generating more meaningful explanations.

Implications and Future Directions

• Trust and Adoption: Explainability is key to gaining trust in AI systems, which is necessary for their broader adoption, especially in safety-critical areas.
• Regulatory Needs: Compliance with regulations like GDPR, which mandates transparency in automated decision-making, underscores the importance of research in explainable AI.
• Research Gaps: The paper identifies open challenges, such as developing standardized metrics for measuring explainability and conducting more user-centered studies to understand the needs of different stakeholders.
• Combination with Ontologies: Future research can focus on the integration of ontologies within explainability frameworks to create more intuitive and human-like explanations.

The survey contributes significantly by mapping out current methodologies in explainable supervised learning and setting a foundation for future research to enhance model transparency, interpretability, and trustworthiness.