Fixation probabilities in populations under demographic fluctuations

Abstract: We study the fixation probability of a mutant type when introduced into a resident population. As opposed to the usual assumption of constant pop- ulation size, we allow for stochastically varying population sizes. This is implemented by a stochastic competitive Lotka-Volterra model. The compe- tition coefficients are interpreted in terms of inverse payoffs emerging from an evolutionary game. Since our study focuses on the impact of the competition values, we assume the same birth and death rates for both types. In this gen- eral framework, we derive an approximate formula for the fixation probability {\phi} of the mutant type under weak selection. The qualitative behavior of {\phi} when compared to the neutral scenario is governed by the invasion dynamics of an initially rare type. Higher payoffs when competing with the resident type yield higher values of {\phi}. Additionally, we investigate the influence of the remaining parameters and find an explicit dependence of {\phi} on the mixed equilibrium value of the corresponding deterministic system (given that the parameter values allow for its existence).