Aggregating predictions from experts: a scoping review of statistical methods, experiments, and applications

Abstract: Forecasts support decision making in a variety of applications. Statistical models can produce accurate forecasts given abundant training data, but when data is sparse, rapidly changing, or unavailable, statistical models may not be able to make accurate predictions. Expert judgmental forecasts---models that combine expert-generated predictions into a single forecast---can make predictions when training data is limited by relying on expert intuition to take the place of concrete training data. Researchers have proposed a wide array of algorithms to combine expert predictions into a single forecast, but there is no consensus on an optimal aggregation model. This scoping review surveyed recent literature on aggregating expert-elicited predictions. We gathered common terminology, aggregation methods, and forecasting performance metrics, and offer guidance to strengthen future work that is growing at an accelerated pace.