Exploring the Effect of Explanation Content and Format on User Comprehension and Trust in Healthcare (2408.17401v3)

Abstract: AI-driven tools for healthcare are widely acknowledged as potentially beneficial to health practitioners and patients, e.g. the QCancer regression tool for cancer risk prediction. However, for these tools to be trusted, they need to be supplemented with explanations. We examine how explanations' content and format affect user comprehension and trust when explaining QCancer's predictions. Regarding content, we deploy SHAP and Occlusion-1. Regarding format, we present SHAP explanations, conventionally, as charts (SC) and Occlusion-1 explanations as charts (OC) as well as text (OT), to which their simpler nature lends itself. We conduct experiments with two sets of stakeholders: the general public (representing patients) and medical students (representing healthcare practitioners). Our experiments showed higher subjective comprehension and trust for Occlusion-1 over SHAP explanations based on content. However, when controlling for format, only OT outperformed SC, suggesting this trend is driven by preferences for text. Other findings corroborated that explanation format, rather than content, is often the critical factor.

• The paper finds that occlusion-1 explanations yield higher subjective understanding and trust compared to SHAP, despite similar objective comprehension.
• It demonstrates that text-based formats significantly enhance user comprehension and trust over chart-based representations.
• The study reveals that explanation preferences vary with expertise, as medical professionals favor chart-based explanations more than the general public.

Essay on "Exploring the Effect of Explanation Content and Format on User Comprehension and Trust"

The paper "Exploring the Effect of Explanation Content and Format on User Comprehension and Trust" investigates the efficacy of different explainability methods for AI models, specifically in medical contexts. The paper revisits the fundamental question of how 'black-box' AI models can be made more interpretable for users and evaluates if comprehension and trust are influenced by the content and format of explanations. This examination focuses on explanations of a regression tool for assessing cancer risk, particularly the outputs from the QCancer algorithm.

Methodologies and Experiments

The authors conducted user studies to assess two core explanation techniques: the Shapley Additive Explanations (SHAP) and the occlusion-1 method. SHAP, a popular feature attribution method rooted in game theory, provides explanations presented as charts, termed as SHAP chart (SC) explanations. In contrast, occlusion-1, known for its intuitive nature, offers both chart-based (OC) and text-based descriptions (OT). The paper context is the prediction of cancer risk where explanations for QCancer algorithm outputs are directed at both the general public and individuals with medical training, thus reflecting two levels of domain expertise.

Key Findings

1. Content Comparison: When evaluating content alone, participants reported a higher subjective understanding and trust for occlusion-1 over SHAP explanations. However, objective comprehension—measured via a definition recognition task—did not significantly favor one explanation type over the other.
2. Format Influence: Interestingly, direct comparison of SC and OT explanations provided more substantial insights. There appeared a pronounced preference for text-based OT explanations over chart-based ones in terms of subjective comprehension and trust. This result implies that explanatory format could have a surpassing influence over content in enhancing user experience.
3. Target Audience Variability: The paper further delineates its findings against participant expertise. It highlights that despite a universal trend where text-based (OT) explanations were favored, medical students showcased a greater inclination towards chart-based occlusion-1 explanations compared to the general public.

Implications and Future Directions

The findings suggest that the format of an explanation is potentially crucial; choosing between textual and visual representations can dramatically impact user experience. This raises significant implications for the deployment of XAI in healthcare, where user trust, derived from comprehensible explanations, is critical for adoption.

For practitioners and developers of AI systems, the takeaway is clear: the design of interpretability tools must consider both content and format tailored to the target users' expertise and preferences. Future research could expand by exploring richer forms of explanations like dialogues or interactive explanations which might bridge gaps in user trust and understanding. Furthermore, since findings indicated format preference variance across expertise levels, dynamic explainability—where explanations adapt to user expertise and feedback—could be a field to explore.

In essence, this paper contributes to the ongoing dialogue within the AI community about advancing user-centered design in explainability by highlighting the nuanced roles that both the content and format of explanations play in enhancing user comprehension and trust. As AI systems proliferate, especially in sensitive domains like healthcare, such insights become invaluable in fostering dependable human-AI collaboration.