Large Deviation Asymptotics and Bayesian Posterior Consistency on Stochastic Processes and Dynamical Systems (2106.06894v2)

Abstract: We consider generalized Bayesian inference on stochastic processes and dynamical systems with potentially long-range dependency. Given a sequence of observations, a class of parametrized model processes with a prior distribution, and a loss function, we specify the generalized posterior distribution. The problem of frequentist posterior consistency is concerned with whether as more and more samples are observed, the posterior distribution on parameters will asymptotically concentrate on the "right" parameters. We show that posterior consistency can be derived using a combination of classical large deviation techniques, such as Varadhan's lemma, conditional/quenched large deviations, annealed large deviations, and exponential approximations. We show that the posterior distribution will asymptotically concentrate on parameters that minimize the expected loss and a divergence term, and we identify the divergence term as the Donsker-Varadhan relative entropy rate from process-level large deviations. As an application, we prove new quenched and annealed large deviation asymptotics and new Bayesian posterior consistency results for a class of mixing stochastic processes. In the case of Markov processes, one can obtain explicit conditions for posterior consistency, whenever estimates for log-Sobolev constants are available, which makes our framework essentially a black box. We also recover state-of-the-art posterior consistency on classical dynamical systems with a simple proof. Our approach has the potential of proving posterior consistency for a wide range of Bayesian procedures in a unified way.