Robust Human-AI Complementarity under Uncertainty

Abstract: Machine learning models are often intended to augment rather than replace human decision makers, by providing information that is complementary to human judgement. Yet, in practice, human decision makers routinely fail to realize such complementary gains, even when models provide useful signal. In this work, we study how asymmetric information about the quality of information available to a human decision maker vs. an AI impacts the ability of a decision maker to extract complementary value from AI predictions. We show that a key factor is the error correlation structure between human and AI predictions. In particular, when the AI's prediction errors are \textit{negatively correlated} with those of the human, the decision maker can construct robust strategies which guarantee improvements in expected utility. We empirically investigate whether these conditions for complementarity arise in practice, using real-world forecasting benchmarks.

• The paper demonstrates that robust human-AI complementarity is achieved only when human and AI prediction errors are negatively correlated.
• It introduces a decision theory framework that rigorously characterizes conditions—such as nontrivial AI signal correlation and conservative linear combinations—for minimizing prediction loss.
• Empirical analysis on forecasting benchmarks reveals that prevalent positive error correlation limits joint performance, highlighting the need for new training and fusion strategies.

Summary of "Robust Human-AI Complementarity under Uncertainty" (2607.06656)

This paper investigates the foundational conditions under which combining human and AI predictions confers robust utility gains, particularly in high-stakes, uncertain settings. Recognizing that most real-world applications involve decision making under model misspecification and information asymmetry, the authors develop a rigorous statistical decision theory framework to analyze when and why human-AI complementarity is provable under uncertainty, and empirically ground their claims using data from forecasting benchmarks.

Theoretical Framework

The core analytical contribution is a model of joint decision making wherein humans and AI provide separate, noisy signals about an unobserved ground-truth state, and the human decision maker seeks to minimize expected loss (MSE or more complex utilities). Crucially, the framework stipulates that the human does not know the true joint distribution of AI signals and outcomes, but rather has an uncertainty set capturing plausible parameters—especially over AI-ground truth correlation.

Key Propositions

• Necessity of Error Correlation Structure: The presence of uncertainty significantly narrows the settings in which the AI signal robustly improves human decision making. If a decision maker knows the joint distribution, any non-redundant (i.e., informative) AI signal is always potentially useful, but under uncertainty, improvement is only guaranteed if the errors of human and AI predictions are negatively correlated.
• Analytic Characterization: Two sufficient conditions are provided:
  1. The AI signal must have nontrivial correlation with the ground truth.
  2. The conditional correlation of human and AI prediction errors must be non-positive (ideally negative).

When these hold, a conservative linear combination of human and AI predictions guarantees uniform reduction in loss across the uncertainty set. If errors are positively correlated, such improvement is only possible if the AI signal is substantially better and/or the human signal is substantially noisier.

• Extension to Nonlinear, Non-Gaussian Models: The core findings generalize beyond the linear-Gaussian regime, applying to any monotone signal structure with negativity of error dependence and a minimum AI sensitivity to the state.

Synthetic and Empirical Evidence

Simulated Validation

Synthetic experiments rigorously validate these theoretical results. By controlling the error correlation in simulated data, the authors demonstrate that robust joint estimators confer substantial gains only when human and AI errors are negatively correlated. With positive error correlation, the combined estimator never outperforms the better single agent unless the human signal is significantly weaker.

Empirical Analysis

The paper evaluates error structures using two real-world prediction datasets:

• ForecastBench: Human expert and LLM forecasts of time series and prediction market outcomes.

• TESS studies: Human and LLM predictions of effect sizes in social science experiments.

Across both datasets, errors between human and LLM predictions are strongly positively correlated, especially in the most practically relevant settings. Prompting interventions aimed at reducing error correlation—contrarian or divergent reasoning, explicit error-correction objectives, altered information contexts—reduce but generally do not invert sign of error correlation. Only in isolated cases (e.g., particular contrarian prompting in TESS) does error correlation become negative, and the resulting numerical gains (e.g., reduction in MSE) are small, particularly when the human baseline is strong.

Distribution Shift

The authors examine whether human-AI complementarity is stable under topic/domain shift, concluding that error structure and complementarity prospects are highly variable—sometimes degrading sharply as task domain shifts, reflecting the instability of the human-model informational relationship.

Practical and Theoretical Implications

The principal implication is that complementarity between humans and current AI models is not robustly assured by accuracy alone, but rather is strongly constrained by the error correlation regime. Even when AI models have independent information (signal), if their errors are positively correlated with those of humans, the scope for guaranteed improvements via combination is severely limited, especially under realistic distributional and model uncertainty.

Standard model evaluation—focused on marginal accuracy—misleads about potential complementarity. Instead, training and selection protocols must target explicit decorrelation or negative correlation of model error with expected human error, especially for deployment in tasks where robust improvement over expert judgment is essential (e.g., science, medicine, policy).

Furthermore, the inefficacy of post-hoc prompt engineering (contrarian or information-restricted prompts) at reliably inducing the needed error structure suggests a need for fundamentally new training objectives or architectures. There is a clear requirement for methodologies that directly optimize for complementarity constraints, rather than treating them as an emergent property.

Directions for Future Research

• Model Training for Complementarity: Development of training or selection methods that explicitly optimize for negative error correlation with expected human errors, possibly via adversarial or co-training schemes.
• Adaptive Fusion Strategies: Design of decision protocols that detect context- or instance-specific error correlation, gating AI influence dynamically.
• Robustness under Distribution Shift: Investigation of error structure stability across domains, with mechanisms for rapid adaptation when human–AI relationships change.
• Human-AI Task Allocation: Algorithmic approaches for task triage and decision allocation in team settings that factor in error correlation in real time.

Conclusion

This paper rigorously delineates the statistical and empirical criteria for robust human-AI complementarity under uncertainty and demonstrates that such complementarity is both theoretically constrained and empirically rare under current modeling and deployment paradigms. The results advocate for a paradigm shift in both AI model construction and evaluation, emphasizing error structure with respect to human experts as the critical axis for future progress in human–AI collaborative decision making.