How Useful Are the Machine-Generated Interpretations to General Users? A Human Evaluation on Guessing the Incorrectly Predicted Labels (2008.11721v2)

Abstract: Explaining to users why automated systems make certain mistakes is important and challenging. Researchers have proposed ways to automatically produce interpretations for deep neural network models. However, it is unclear how useful these interpretations are in helping users figure out why they are getting an error. If an interpretation effectively explains to users how the underlying deep neural network model works, people who were presented with the interpretation should be better at predicting the model's outputs than those who were not. This paper presents an investigation on whether or not showing machine-generated visual interpretations helps users understand the incorrectly predicted labels produced by image classifiers. We showed the images and the correct labels to 150 online crowd workers and asked them to select the incorrectly predicted labels with or without showing them the machine-generated visual interpretations. The results demonstrated that displaying the visual interpretations did not increase, but rather decreased, the average guessing accuracy by roughly 10%.

• The paper evaluated machine-generated interpretations, finding they surprisingly decreased general users' accuracy by approximately 10% when attempting to guess deep neural network image classifier errors.
• Interpretations were particularly unhelpful, and sometimes detrimental, for specific error types like misclassifications involving similar-looking objects or erroneous background correlations.
• The study emphasizes the need for caution in deploying machine interpretations for non-expert users and calls for developing more effective interpretation methods that genuinely aid human understanding.

Analyzing the Utility of Machine-Generated Interpretations for General Users

The paper entitled "How Useful Are the Machine-Generated Interpretations to General Users?" authored by Hua Shen and Ting-Hao (Kenneth) Huang provides an empirical evaluation of the effectiveness of machine-generated visual interpretations in aiding general users to comprehend errors produced by deep neural network image classifiers. This paper is critically important as deep neural networks are increasingly employed across significant domains such as healthcare, transportation, and education.

The authors sought to understand if such visual interpretations genuinely enhanced users' ability to predict the output errors of image classifiers, an aspect not yet clearly established in existing literature. Their methodology involved enlisting 150 online crowd workers to attempt to identify errors made by image classifiers on a series of tasks, both with and without the aid of machine-generated interpretations.

Key experimental design included two separate conditions across two controlled experiments: Interpretation where participants were provided visual interpretations, and No-Interpretation where they were not. Intriguingly, findings from both experiments indicated that the availability of these interpretations did not improve, but rather decreased, participants' average accuracy by approximately 10%. Specifically, experiments demonstrated a 0.73 accuracy rate for [Int] as opposed to 0.81 for [No-Int] in the first experiment, and a 0.63 versus 0.73 in the similar respective groups for the second experiment.

The categorization of misclassification errors into types — local character inference, multiple objects selection, similar appearance inference, correlation learning, and incorrect gold-standard labels — provided a deeper insight. Notably, interpretations were least beneficial and even detrimental in cases where distinct but similar-looking objects were misclassified, or where background correlations led to erroneous predictions. It seems that visual interpretations may cloud judgment by highlighting irrelevant cues.

Several hypotheses were proposed for this unexpected outcome: the interpretations themselves may not be sufficiently accurate, salient features may not be adequately captured in the format presented (e.g., saliency maps), or they may inadvertently emphasize non-contributory elements. The limitations of the paper, as acknowledged by the authors, include a modest sample size, potential shortcomings in interpretation diversity, limitations related to recruiting non-expert participants from MTurk, and the narrow focus on image classifiers may restrict the generalizability of the findings.

The authors' exploration underscores the necessity for caution when implementing machine-generated interpretations within systems intended for users unversed in the complexities of deep neural networks. This discourse opens up potential avenues for future investigation, such as developing more sophisticated interpretive models or alternative presentation formats that might more effectively enhance human understanding and error correction capabilities.

The implications of these findings are significant, particularly considering that interpretability in machine learning not only affects model transparency but also has consequential impacts on user trust and decision-making in AI-mediated contexts. This paper prompts the AI research community to reflect on the design and application of model interpretability techniques, steering towards improvements that genuinely aid end-users in meaningful and actionable ways.