On econometric inference and multiple use of the same data

Abstract: In fields that are mainly nonexperimental, such as economics and finance, it is inescapable to compute test statistics and confidence regions that are not probabilistically independent from previously examined data. The Bayesian and Neyman-Pearson inference theories are known to be inadequate for such a practice. We show that these inadequacies also hold m.a.e. (modulo approximation error). We develop a general econometric theory, called the neoclassical inference theory, that is immune to this inadequacy m.a.e. The neoclassical inference theory appears to nest model calibration, and most econometric practices, whether they are labelled Bayesian or `a la Neyman-Pearson. We derive a general, but simple adjustment to make standard errors account for the approximation error.