The Brownian Spatial Coalescent (2401.08557v2)

Abstract: We introduce the Brownian spatial coalescent, a class of Markov spatial coalescent processes on the $d$-dimensional torus in continuous space, that is axiomatically defined by the following property: conditional on the times and locations of all coalescence events, lineages follow independent Brownian bridges backwards in time along the branches of the coalescence tree. We prove that a Brownian spatial coalescent is characterised by a set of finite "transition measures" on the torus, in analogy to the transition rates that characterise a non-spatial coalescent. We prove that a Brownian spatial coalescent is sampling consistent, in a novel sense, if and only if all transition measures are constant multiples of Lebesgue measure, and the multiples are the transition rates of a $\Xi$-coalescent, or a $\Lambda$-coalescent if simultaneous mergers are not allowed. This defines the "Brownian spatial $\Xi$-coalescent", which we prove describes the genealogies of the $\Xi$-Fleming-Viot process - a generalisation of the well known Fleming-Viot process - at stationarity, and in fact describes its full time reversal if augmented with "resampling" steps at the times of coalescence events. An important consequence of our results is that all spatial population models in which individuals follow independent Brownian motions and the branching mechanism depends non-trivially on the spatial distribution, for example through local regulation, have non-Markovian genealogies. Byproducts of our results include explicit formulas for samples from the stationary distribution of a $\Xi$-Fleming-Viot process, Wright-Malecot type formulas, and a representation of the backward dynamics of lineages in terms of Brownian motions with coupled drift. This includes calculations of the drift that leads to multiple or even simultaneous mergers in any dimension.