Fostering human learning is crucial for boosting human-AI synergy (2512.13253v1)

Abstract: The collaboration between humans and AI holds the promise of achieving superior outcomes compared to either acting alone. Nevertheless, our understanding of the conditions that facilitate such human-AI synergy remains limited. A recent meta-analysis showed that, on average, human-AI combinations do not outperform the better individual agent, indicating overall negative human-AI synergy. We argue that this pessimistic conclusion arises from insufficient attention to human learning in the experimental designs used. To substantiate this claim, we re-analyzed all 74 studies included in the original meta-analysis, which yielded two new findings. First, most previous research overlooked design features that foster human learning, such as providing trial-by-trial outcome feedback to participants. Second, our re-analysis, using robust Bayesian meta-regressions, demonstrated that studies providing outcome feedback show relatively higher synergy than those without outcome feedback. Crucially, when feedback is paired with AI explanations we tend to find positive human-AI synergy, while AI explanations provided without feedback were strongly linked to negative synergy, indicating that explanations are useful for synergy only when humans can learn to verify the AI's reliability through feedback. We conclude that the current literature underestimates the potential for human-AI collaboration because it predominantly relies on experimental designs that do not facilitate human learning, thus hindering humans from effectively adapting their collaboration strategies. We therefore advocate for a paradigm shift in human-AI interaction research that explicitly incorporates and tests human learning mechanisms to enhance our understanding of and support for successful human-AI collaboration.

• The paper demonstrates that outcome feedback is a critical moderator, enhancing human–AI synergy as evidenced by robust Bayesian meta-regressions.
• The study reanalyzes 74 studies and reveals that coupling outcome feedback with AI explanations yields more positive synergy scores (Hedges' g = 0.3).
• The findings suggest that neglecting user learning in experimental designs underestimates the collaborative benefits possible in human–AI systems.

Fostering Human Learning as a Prerequisite for Human–AI Synergy

Background and Motivation

The prospect of achieving superior joint performance through human–AI collaboration—referred to as human–AI synergy—stands as a major aspiration in applied and cognitive AI research. Despite this theoretical potential, empirical evidence from a recent meta-analysis by Vaccaro et al. (2024) reveals that, on average, human–AI combinations underperform compared to the better single agent (human or AI), indicating negative synergy. This paper systematically interrogates the underlying causes of this counterintuitive pattern and argues that the literature's neglect of human learning mechanisms, particularly via outcome feedback, is a critical confounder in current assessments of human–AI synergy.

Figure 1: Modes of human–AI interaction, contrasting conditions with and without learning opportunities, and summary of meta-analytic Bayesian results on synergy as a function of feedback and explanation.

Meta-Analytic Re-Examination: Design Features and Human Learning

A comprehensive re-analysis of all 74 studies included in the original meta-analysis is conducted, with a focus on experimental design elements that enable human learning. Notable among these elements are trial-level outcome feedback and, to a lesser extent, access to AI explanations. The coding of all 370 experimental conditions exposes that the vast majority of studies omitted outcome feedback, operated in artificial settings, and employed short tasks with limited practice. Only 14% of studies included outcome feedback, and within those, very few tracked how human–AI integration evolved over time, precluding analysis of trialwise adaptation dynamics.

Crucially, robust Bayesian model-averaged meta-regressions demonstrate clear differentiation: studies that incorporate outcome feedback yield higher (sometimes positive) human–AI synergy scores than those that do not. Bayesian factors ($BF_{inclusion}$) support feedback as a non-trivial moderator. Specifically, combined availability of outcome feedback and AI explanations is associated with the strongest positive synergy (Hedges' $g = 0.3$, 95% CI $-0.18$ to $0.48$), while explanations provided in the absence of feedback actually exacerbate negative synergy ($g = -0.31$, 95% CI $-0.48$ to $-0.11$). These effects are summarized visually in the meta-regression panel of Figure 1.

Theoretical and Numerical Claims

This analysis surfaces two strong claims, supported by both meta-analytic evidence and the broader literatures on adaptive expertise and decision science:

1. Outcome feedback is an underappreciated determinant of human–AI synergy: In studies lacking feedback, the joint system's performance often regresses below the superior single agent, reflecting a failure in adaptive calibration of trust and use of AI suggestions.
2. Synergy is maximized only under coupled conditions of feedback and explanation: Explanations alone (without grounding via feedback) can mislead or overwhelm users, reinforcing maladaptive strategies; feedback contextualizes explanations, allowing users to discern when and how to exploit AI assistance.

Despite only modest credible intervals (due to the limited number of studies with feedback), the directionality and magnitude of effects are robust across analytic priors.

Implications for Research and AI System Design

The findings have critical implications for both experimental research and practical system design. From a research perspective, they call into question the external validity of prior pessimistic claims regarding the limits of human–AI synergy. Studies that do not provide learning opportunities systematically underestimate the achievable benefits of collaborative intelligence. The paper advocates for a theory-driven expansion in experimental design, incorporating not only outcome feedback but also richer measures of user adaptation, trust calibration, and domain-specific expertise evolution.

Practically, these results suggest that effective human–AI systems must be developed with explicit mechanisms for ongoing user learning—e.g., through trialwise feedback, actionable explanations, and interactive modules that foster metacognitive insight into when AI outputs are reliable. Neglecting these features in deployed decision support systems risks not only underutilized AI capabilities but potentially degraded overall performance due to miscalibrated reliance.

Future Directions

This work opens multiple avenues for subsequent investigation. Expanded meta-analyses should systematically capture older literature, fields outside mainstream human–computer interaction (such as judgment and decision-making under uncertainty), and studies with varying degrees of user agency and context complexity. Experiments that manipulate the timing, nature, and adaptivity of feedback—along with longitudinal tracking of user strategies—are needed to explain the learning curve toward optimal synergy. Further, standardization of performance metrics and data availability would enable direct cross-paper synthesis and benchmarking.

Conclusion

The paper establishes that the empirical demonstration of human–AI synergy is critically contingent upon supporting human learning within the experimental paradigm. Outcome feedback, especially when paired with explanatory AI outputs, is a principal driver of positive interactive effects; explanations without feedback, conversely, may be detrimental. Current literature has therefore underestimated the attainable benefits of collaborative intelligence by neglecting this learning dimension. Future research and AI system development must systematically integrate mechanisms for user learning to realize the full promise of human–AI synergy.