Search for Stochastic Gravitational-Wave Background from Massive Gravity in the NANOGrav 12.5-Year Data Set

Abstract: Gravitational waves offer a new window to probe the nature of gravity, including answering if the mediating particle, graviton, has a non-zero mass or not. Pulsar timing arrays measure stochastic gravitational wave background (SGWB) at $\sim1-100$~nanohertz. Recently, the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) collaboration reported an uncorrelated common-spectrum process in their 12.5-year data set with no substantial evidence that the process comes from the SGWB predicted by general relativity. In this work, we explore the possibility of an SGWB from massive gravity in the data set and find that a massless graviton is preferred because of the relatively larger Bayes factor. Without statistically significant evidence for dispersion-related correlations predicted by massive gravity, we place upper limits on the amplitude of the SGWB for graviton mass smaller than $10^{-23}$~eV as $A_{\rm{MG}}<3.21\times 10^{-15}$ at $95\%$ confidence level.