Uncertainty in 2-point correlation function estimators and BAO detection in SDSS DR7 (1009.1232v2)

Abstract: We study the uncertainty in different two-point correlation function (2PCF) estimators in currently available galaxy surveys. This is motivated by the active subject of using the baryon acoustic oscillations (BAOs) feature in the correlation function as a tool to constrain cosmological parameters, which requires a fine analysis of the statistical significance. We discuss how estimators are affected by both the uncertainty in the mean density $\bar{n}$ and the integral constraint $\frac{1}{V^2}\int_{V^2} \hat{\xi} (r) d^3r =0$ which necessarily causes a bias. We quantify both effects for currently available galaxy samples using simulated mock catalogues of the Sloan Digital Sky Survey (SDSS) following a lognormal model, with a Lambda-Cold Dark Matter ($\Lambda\text{CDM}$) correlation function and similar properties as the samples (number density, mean redshift for the $\Lambda\text{CDM}$ correlation function, survey geometry, mass-luminosity bias). Because we need extensive simulations to quantify small statistical effects, we cannot use realistic N-body simulations and some physical effects are neglected. Our simulations still enable a comparison of the different estimators by looking at their biases and variances. We also test the reliability of the BAO detection in the SDSS samples and study the compatibility of the data results with our $\Lambda\text{CDM}$ simulations.