Emergent self-inhibition governs the landscape of stable states in complex ecosystems

Abstract: Species-rich ecosystems often exhibit multiple stable states with distinct species compositions. Yet, the factors determining the likelihood of each state's occurrence remain poorly understood. Here, we characterize and explain the landscape of stable states in the random Generalized Lotka-Volterra (GLV) model, in which multistability is widespread. We find that the same pool of species with random initial abundances can result in different stable states, whose likelihoods typically differ by orders of magnitude. A state's likelihood increases sharply with its total biomass, or inverse self-inhibition. We develop a simplified model to predict and explain this behavior, by coarse-graining ecological interactions so that each stable state behaves as a unit. In this setting, we can accurately predict the entire landscape of stable states using only two macroscopic properties: the biomass of each state and species diversity. Our theory also provides insight into the biomass-likelihood relationship: High-biomass states have low self-inhibition and thus grow faster, outcompete others, and become much more likely. These results reveal emergent self-inhibition as a fundamental organizing principle for the attractor landscape of complex ecosystems - and provide a path to predict ecosystem outcomes without knowing microscopic interactions.