On the Design and use of Ensembles of Multi-model Simulations for Forecasting

Abstract: Probability forecasting is common in the geosciences, the finance sector, and elsewhere. It is sometimes the case that one has multiple probability-forecasts for the same target. How is the information in these multiple forecast systems best "combined"? Assuming stationary, then in the limit of a very large forecast-outcome archive, each model-based probability density function can be weighted to form a "multi-model forecast" which will, in expectation, provide the most information. In the case that one of the forecast systems yields a probability distribution which reflects the distribution from which the outcome will be drawn, then Bayesian Model Averaging will identify this model as the number of forecast-outcome pairs goes to infinity. In many applications, like those of seasonal forecasting, data are precious: the archive is often limited to fewer than $2^6$ entries. And no perfect model is in hand. In this case, it is shown that forming a single "multi-model probability forecast" can be expected to prove misleading. These issues are investigated using probability forecasts of a simple mathematical system, which allows most limiting behaviours to be quantified.