Maximum likelihood estimators for scaled mutation rates in an equilibrium mutation-drift model

Abstract: The stationary sampling distribution of a neutral decoupled Moran or Wright-Fisher diffusion with neutral mutations is known to first order for a general rate matrix with small but otherwise unconstrained mutation rates. Using this distribution as a starting point we derive results for maximum likelihood estimates of scaled mutation rates from site frequency data under three model assumptions: a twelve-parameter general rate matrix, a nine-parameter reversible rate matrix, and a six-parameter strand-symmetric rate matrix. The site frequency spectrum is assumed to be sampled from a fixed size population in equilibrium, and to consist of allele frequency data at a large number of unlinked sites evolving with a common mutation rate matrix without selective bias. We correct an error in a previous treatment of the same problem (Burden and Tang, 2017) affecting the estimators for the general and strand-symmetric rate matrices. The method is applied to a biological dataset consisting of a site frequency spectrum extracted from short autosomal introns in a sample of Drosophila melanogaster individuals.