On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection

Abstract: Humans are the final decision makers in critical tasks that involve ethical and legal concerns, ranging from recidivism prediction, to medical diagnosis, to fighting against fake news. Although machine learning models can sometimes achieve impressive performance in these tasks, these tasks are not amenable to full automation. To realize the potential of machine learning for improving human decisions, it is important to understand how assistance from machine learning models affects human performance and human agency. In this paper, we use deception detection as a testbed and investigate how we can harness explanations and predictions of machine learning models to improve human performance while retaining human agency. We propose a spectrum between full human agency and full automation, and develop varying levels of machine assistance along the spectrum that gradually increase the influence of machine predictions. We find that without showing predicted labels, explanations alone slightly improve human performance in the end task. In comparison, human performance is greatly improved by showing predicted labels (>20% relative improvement) and can be further improved by explicitly suggesting strong machine performance. Interestingly, when predicted labels are shown, explanations of machine predictions induce a similar level of accuracy as an explicit statement of strong machine performance. Our results demonstrate a tradeoff between human performance and human agency and show that explanations of machine predictions can moderate this tradeoff.

• The paper shows that machine-predicted labels improve human deception detection accuracy by about 21% over baseline performance.
• It finds that combining predicted labels with feature-based explanations further boosts performance and enhances user trust.
• The study highlights the tradeoff between automated assistance and preserving human decision-making agency in ethically sensitive contexts.

Analyzing Human Predictions with Machine Learning Explanations in Deception Detection

The paper "On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection" provides a meticulous exploration of integrating machine learning assistance into human decision-making processes, particularly within the domain of deception detection. The study aims to determine the impact of machine learning model explanations on human performance and agency, using deception detection as a testbed.

Key Insights and Methodology

The paper delineates a spectrum between full human agency and full automation, proposing varying levels of machine assistance. These gradations range from human decision-making with feature-based or example-based explanations from machine learning models, to instances where both predictions and proclaimed accuracy levels are shared. This approach acknowledges the nuanced tradeoffs between enhancing human performance and maintaining human agency — a critical consideration in tasks laden with ethical and legal implications.

The core of the empirical study involves human participants from Amazon Mechanical Turk, tasked with identifying authentic versus deceptive hotel reviews, a dataset drawn from previous research efforts by Ott et al. The machine learning model employed is a linear SVM with a bag-of-words feature set, chosen for its interpretability and historical performance in this domain.

Results Overview

The study unveils several significant findings:

1. Explanations Alone: Feature-based or example-based explanations marginally improve human performance, but do not achieve significant gains without revealing predicted labels. This modest improvement arises partly from training examples, emphasizing the complexity of interpreting the given explanations independently.
2. Predicted Labels Influence: Merely presenting predicted labels from the machine has a notable impact, augmenting human accuracy significantly by approximately 21% relative to baseline human performance. This finding underscores the substantial influence of machine predictions on human decision processes.
3. Combination with Explanations: When combining predicted labels with feature-based explanations (such as heatmaps), human performance further improves, suggesting explanations bolster the perceived credence of machine predictions without an explicit accuracy statement.
4. Trust Dynamics: The extent to which participants trust machine predictions correlates with the nature of assistance provided. Explanations increase trust in machine predictions, with heatmap explanations proving more influential than randomly generated ones, signaling the potential role explanations play in enhancing trust.
5. Varying Accuracy Statements: Intriguingly, the trust and reliance of humans on machine predictions remain relatively high, even when the machine's purported accuracy is falsely reduced, indicating a propensity for humans to trust numerical accuracy statements, notwithstanding the actual probability of correctness.

Implications and Future Directions

The study delineates crucial implications for deploying machine learning systems in decision support roles. It elucidates the importance of facilitating human understanding through explanatory models, while cautioning against undue reliance on machine predictions that might undermine human agency. Acknowledging the risk of excessive priming with predictive labels, the potential development of tutorials or narrative explanations could empower users with deeper comprehension, thus harmonizing the tradeoff between performance and autonomy.

Furthermore, the exploration of trust dynamics reveals a need for a more profound inquiry into how humans interpret probabilistic information and how this understanding shapes decision-making behaviors in contexts subject to ambiguity and uncertainty.

Ultimately, this work lays an essential foundation for further research in effectively integrating AI models into human-centric domains, where maintaining transparency and interpretability is crucial for ethical and responsible AI exploitation. Future endeavors might expand beyond deception detection to other domains where human intuition and judgment benefit from machine assistance, thereby bridging the gap between automation capabilities and human cognitive strengths.