Parameters not empirically identifiable or distinguishable, including correlation between Gaussian observations

Abstract: Note: Accepted version, published in Statistical Papers, https://doi.org/10.1007/s00362-023-01414-3. It is shown that some theoretically identifiable parameters cannot be empirically identified, meaning that no consistent estimator of them can exist. An important example is a constant correlation between Gaussian observations (in presence of such correlation not even the mean can be empirically identified). Empirical identifiability and three versions of empirical distinguishability are defined. Two different constant correlations between Gaussian observations cannot even be empirically distinguished. A further example are cluster membership parameters in $k$-means clustering. Several existing results in the literature are connected to the new framework. General conditions are discussed under which independence can be distinguished from dependence.