Non-parametric finite-sample credible intervals with one-dimensional priors: a middle ground between Bayesian and frequentist intervals

Abstract: We propose a new type of statistical interval obtained by weakening the definition of a p% credible interval: Having observed the interval (rather than the full dataset) we should put at least a p% belief in it. From a decision-theoretical point of view the resulting intervals occupy a middle ground between frequentist and fully Bayesian statistical intervals, both practically and philosophically: To a p% Bayesian credible interval we should assign (at least a) p% belief also after seeing the full dataset, while p% frequentist intervals we in general only assign a p% belief before seeing either the data or the interval. We derive concrete implementations for two cases: estimation of the fraction of a distribution that falls below a certain value (i.e., the CDF), and of the mean of a distribution with bounded support. Even though the problems are fully non parametric, these methods require only one-dimensional priors. They share many of the practical advantages of Bayesian methods while avoiding the complexity of assigning high-dimensional priors altogether. Asymptotically they give intervals equivalent to the fully Bayesian approach and somewhat wider intervals, respectively. We discuss promising directions where the proposed type of interval may provide significant advantages.