Incremental XAI: Memorable Understanding of AI with Incremental Explanations (2404.06733v1)

Abstract: Many explainable AI (XAI) techniques strive for interpretability by providing concise salient information, such as sparse linear factors. However, users either only see inaccurate global explanations, or highly-varying local explanations. We propose to provide more detailed explanations by leveraging the human cognitive capacity to accumulate knowledge by incrementally receiving more details. Focusing on linear factor explanations (factors $\times$ values = outcome), we introduce Incremental XAI to automatically partition explanations for general and atypical instances by providing Base + Incremental factors to help users read and remember more faithful explanations. Memorability is improved by reusing base factors and reducing the number of factors shown in atypical cases. In modeling, formative, and summative user studies, we evaluated the faithfulness, memorability and understandability of Incremental XAI against baseline explanation methods. This work contributes towards more usable explanation that users can better ingrain to facilitate intuitive engagement with AI.

• The paper introduces Incremental XAI as a novel method that incrementally discloses base and detailed factors to optimize user comprehension.
• It develops a tree-based framework with linear model trees and factor sparsity regularization to balance detail with simplicity.
• Empirical results show improved faithfulness, usability, and memorability compared to traditional global and local explanation approaches.

Incremental XAI: Enhancing User Comprehension Through Gradual Explanation

Introduction

Explainable AI (XAI) represents a vital bridge between advanced AI methods and their users. However, the complexity and variety of these methods often act as a barrier to widespread understanding and adoption. Typical strategies for simplifying explanations, such as using sparse linear models, frequently fall short, either by oversimplifying the model's operations or overwhelming users with too much information at once. Addressing these issues, this paper introduces Incremental XAI, a novel approach that aims to bolster users' understanding by providing explanations that gradually increase in detail.

Framework Overview

Incremental XAI is rooted in pedagogical principles, advocating for a stepwise learning process. It builds on the realization that the full complexity of AI models can often be daunting for users. Instead of offering broad, generalized explanations (Global) or highly specific, instance-based details (Local), Incremental XAI proposes a middle path; it segments models into 'typical' and 'atypical' cases, explaining these with a base set of factors and then incrementally adding details for the complex scenarios.

This method introduces two core concepts: Base and Incremental factors. Initially, users are presented with Base factors that apply to a majority of instances, offering a scaffold for their understanding. When detailed explanation is necessary, additional Incremental factors are revealed, enhancing the explanation's fidelity to the actual AI model's workings without departing from the foundational understanding established by the Base factors.

Implementation and Evaluation

The paper exemplifies this concept through the development of a tree-based incremental explanation framework using linear model trees. This includes additive factors and factor sparsity regularization, optimized to minimize cognitive load by retaining a compact set of factors for users to comprehend.

The empirical evaluation quantifies the Incremental XAI against traditional explanations methods (Global, Subglobal, and Local) across several factors: faithfulness, usability, understandability, and memorability. Results demonstrate that Incremental XAI offers a balanced approach, providing high memorability similar to Global explanations and maintaining a level of detail and faithfulness approaching that of Local explanations.

Theoretical and Practical Implications

Theoretical

The Incremental XAI paradigm challenges current XAI research norms by suggesting a more dynamic, user-centric approach to explanations. It emphasizes the progression of user understanding over time, integrating concepts from cognitive psychology, such as the limits of working memory and the process of schema formation. This aligns with recent calls within the HCI and AI communities for explanations to support user learning and adaptability, rather than merely serving as static descriptions of AI behavior.

Practical

From a practical standpoint, this research underscores the importance of considering the end-user's cognitive process when designing explanation systems. By applying Incremental XAI, developers can create more accessible and effective tools for a range of users, from novices to experts across various domains. The methodology provides a structured approach to simplify the introduction of complex models, potentially enhancing user trust and facilitating broader adoption of AI technologies.

Future Directions

Although the framework presents a promising direction, its application has been primarily demonstrated with linear models. Future work could explore extending the incremental approach to other types of models or explanation formats, such as graphical or rule-based explanations. Additionally, long-term user studies could provide deeper insights into how Incremental XAI supports the learning process over time and its impact on user trust and decision-making.

Conclusion

Incremental XAI represents a significant step forward in the field of explainable AI. By acknowledging and addressing the cognitive challenges users face when interacting with complex AI systems, this approach offers a promising pathway to demystify AI decision-making processes, ultimately enhancing transparency, trust, and user empowerment.