Ecosystem models cannot predict the consequences of conservation decisions

Abstract: Ecosystem models are often used to predict the consequences of management decisions in applied ecology, including fisheries management and threatened species conservation. These models are high-dimensional, parameter-rich, and nonlinear, yet limited data is available to calibrate them, and they are rarely tested or validated. Consequently, the accuracy of their forecasts, and their utility as decision-support tools is a matter of debate. In this paper, we calibrate ecosystem models to time-series data from 110 different experimental microcosm ecosystems, each containing between three and five interacting species. We then assess how often these calibrated models offer accurate and useful predictions about how the ecosystem will respond to a set of standard management interventions. Our results show that for each timeseries dataset, a large number of very different parameter sets offer equivalent, good fits. However, these calibrated ecosystem models have poor predictive accuracy when forecasting future dynamics and offer ambiguous predictions about how species in the ecosystem will respond to management interventions. Closer inspection reveals that the ecosystem models fail because calibration cannot determine the types of interactions that occur within the ecosystem. Our findings call into question claims that ecosystem modelling can support applied ecological decision-making when they are calibrated against real-world datasets.