Bayesian Estimation of Population Size Changes by Sampling Tajima's Trees

Julia Palacios (1) , Amandine Véber (2) , Lorenzo Cappello (1) , Zhangyuan Wang (1) , John Wakeley (3) , Sohini Ramachandran (4)

1 Stanford University
2 CMAP - Centre de Mathématiques Appliquées - Ecole Polytechnique
3 Harvard University
4 Brown University

The large state space of gene genealogies is a major hurdle for inference methods based on Kingman's coalescent. Here, we present a new Bayesian approach for inferring past population sizes which relies on a lower resolution coalescent process we refer to as "Tajima's coalescent". Tajima's coalescent has a drastically smaller state space, and hence it is a computationally more efficient model, than the standard Kingman coalescent. We provide a new algorithm for efficient and exact likelihood calculations for data without recombination, which exploits a directed acyclic graph and a correspondingly tailored Markov Chain Monte Carlo method. We compare the performance of our Bayesian Estimation of population size changes by Sampling Tajima's Trees (BESTT) with a popular implementation of coalescent-based inference in BEAST using simulated data and human data. We empirically demonstrate that BESTT can accurately infer effective population sizes, and it further provides an efficient alternative to the Kingman's coalescent. The algorithms described here are implemented in the R package phylodyn, which is available for download at https://github.com/JuliaPalacios/phylodyn.

Domaines: Statistiques [math.ST]; Génétique des populations [q-bio.PE]

Julia Palacios, Amandine Véber, Lorenzo Cappello, Zhangyuan Wang, John Wakeley, et al.. Bayesian Estimation of Population Size Changes by Sampling Tajima's Trees. Genetics, In press. ⟨hal-02285644⟩