Balls, cups, and quasi-potentials: quantifying stability in stochastic systems

Abstract: When a system has more than one stable state, how can the stability of these states be compared? This deceptively simple question has important consequences for ecosystems, because systems with alternative stable states can undergo dramatic regime shifts. The probability, frequency, duration, and dynamics of these shifts will all depend on the relative stability of the stable states. Unfortunately, the concept of stability in ecology has suffered from substantial confusion and this is particularly problematic for systems where stochastic perturbations can cause shifts between coexisting alternative stable states. A useful way to visualize stable states in stochastic systems is with a ball-in-cup diagram, in which the state of the system is represented as the position of a ball rolling on a surface, and the random perturbations can push the ball from one basin of attraction to another. The surface is determined by a potential function, which provides a natural stability metric. However, systems amenable to this representation, called gradient systems, are quite rare. As a result, the potential function is not widely used and other approaches based on linear stability analysis have become standard. Linear stability analysis is designed for local analysis of deterministic systems and, as we show, can produce a highly misleading picture of how the system will behave under continual, stochastic perturbations. In this paper, we show how the potential function can be generalized so that it can be applied broadly, employing a concept from stochastic analysis called the quasi-potential. Using three classic ecological models, we demonstrate that the quasi-potential provides a useful way to quantify stability in stochastic systems.