Machine Predictions and Human Decisions with Variation in Payoffs and Skill

Abstract: Human decision-making differs due to variation in both incentives and available information. This constitutes a substantial challenge for the evaluation of whether and how machine learning predictions can improve decision outcomes. We propose a framework that incorporates machine learning on large-scale data into a choice model featuring heterogeneity in decision maker payoff functions and predictive skill. We apply this framework to the major health policy problem of improving the efficiency in antibiotic prescribing in primary care, one of the leading causes of antibiotic resistance. Our analysis reveals large variation in physicians' skill to diagnose bacterial infections and in how physicians trade off the externality inherent in antibiotic use against its curative benefit. Counterfactual policy simulations show that the combination of machine learning predictions with physician diagnostic skill results in a 25.4 percent reduction in prescribing and achieves the largest welfare gains compared to alternative policies for both estimated physician as well as conservative social planner preference weights on the antibiotic resistance externality.