Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges (2010.09337v1)

Abstract: We present a brief history of the field of interpretable machine learning (IML), give an overview of state-of-the-art interpretation methods, and discuss challenges. Research in IML has boomed in recent years. As young as the field is, it has over 200 years old roots in regression modeling and rule-based machine learning, starting in the 1960s. Recently, many new IML methods have been proposed, many of them model-agnostic, but also interpretation techniques specific to deep learning and tree-based ensembles. IML methods either directly analyze model components, study sensitivity to input perturbations, or analyze local or global surrogate approximations of the ML model. The field approaches a state of readiness and stability, with many methods not only proposed in research, but also implemented in open-source software. But many important challenges remain for IML, such as dealing with dependent features, causal interpretation, and uncertainty estimation, which need to be resolved for its successful application to scientific problems. A further challenge is a missing rigorous definition of interpretability, which is accepted by the community. To address the challenges and advance the field, we urge to recall our roots of interpretable, data-driven modeling in statistics and (rule-based) ML, but also to consider other areas such as sensitivity analysis, causal inference, and the social sciences.

Interpretable Machine Learning: Historical Overview, Current State, and Challenges

The paper "Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges" presents a comprehensive examination of the interpretable machine learning (IML) domain. It traces IML's nascent roots, reviews state-of-the-art interpretation methodologies, and outlines the field's pressing challenges and open research questions.

Historical Context and Development

Interpretable models find their origins as early as the 19th century through foundational works in regression modeling by Gauss, Legendre, and others. Noting significant growth in the latter half of the 20th century, the paper highlights key advances such as support vector machines and rule-based learning. Statistical models established at this time emphasized intrinsic interpretability through distributional assumptions and complexity restrictions.

The paper also touches on the divergent paths of ML and statistical methodologies, where ML focused on maximizing predictive power rather than interpretability. However, interpretability remained an undercurrent in ML research, as seen in random forest's feature importance measures. The paper cites the proliferation of model-agnostic interpretation methods in the 2010s as a pivotal moment in IML, fueled by the deep learning resurgence and demand for understanding ML-driven decisions.

Current IML Methods

The research highlights three major IML methodological approaches to interpret ML models: component analysis, model sensitivity, and surrogate models.

1. Model Component Analysis: This approach dissects model components for interpretation, specific to inherently understandable models like linear regression or decision trees. Though scalable to more complex models (e.g., CNN feature maps), this method shows limitations in high-dimensional contexts.
2. Model Sensitivity Analysis: Predominantly model-agnostic, these methods evaluate model sensitivity to input perturbations to produce explanations. Techniques like Shapley values and counterfactual explanations stand out for their applicability across models and robust theoretical underpinnings.
3. Surrogate Models: Surrogate models replicate the behavior of complex models using interpretable ones. For instance, LIME uses local surrogate models to offer insights into individual predictions, while globally, these surrogates help verify patterns in model behavior.

Challenges and Future Directions

Though mature, the field of IML faces several pressing issues, which if addressed could enhance its application relevance and reliability across varied domains:

• Statistical Uncertainty and Rigorous Inference: Many IML methods lack uncertainty quantification, which is vital, given the reliance on training data. Future work must incorporate statistical rigor akin to best practices in statistical analysis.
• Causal Interpretability: Predictive models are generally biased towards correlation rather than causation. Bridging this gap is essential, particularly for scientific applications where causal insights inform decision-making.
• Feature Dependence and Interaction: Dependence between features complicates interpretability, causing methods that ignore such dependencies to potentially misinform. Enhanced frameworks that account for feature distributions are needed.
• Definitional Ambiguities: A formal and universally accepted definition of interpretability remains elusive. Establishing connection points with human-centric fields could yield evaluation metrics that are both qualitative and quantitative.

The paper advises a holistic interdisciplinary approach, involving insights from human-computer interaction, social sciences, and core statistical theory. Such a confluence is imperative to overcoming societal challenges presented by rapidly advancing AI technologies while ensuring transparency, accountability, and equity.

By encapsulating the panorama of IML and rooting it in both foundational discipline-specific traditions and emerging cross-disciplinary collaborations, the paper makes a substantial contribution towards understanding and advancing the interpretability of machine learning models across varied applications.