Covariance-Corrected WAIC for Bayesian Sequential Data Models

Abstract: This paper introduces and develops a theoretical extension of the widely applicable information criterion (WAIC), called the Covariance-Corrected WAIC (CC-WAIC), that applied for Bayesian sequential data models. The CC-WAIC accounts for temporal or structural dependence by incorporating the full posterior covariance structure of the log-likelihood contributions, in contrast to the classical WAIC that assumes conditional independence among data. We exploit the limitations of classical WAIC in the sequential data contexts and derive the CC-WAIC criterion under a theoretical framework. In addition, we propose a bias correction based on effective sample size to improve estimation from Markov Chain Monte Carlo (MCMC) simulations. Furthermore, we highlight the advantages of CC-WAIC in terms of stability and appropriateness for dependent data. This new criterion is supported by formal mathematical derivations, illustrative examples, and discussion of implications for model selection in both classical and modern Bayesian applications. To evaluate the reliability of CC-WAIC under varying data regimes, we conduct simulation experiments across multiple time series lengths (small, medium, and large) and different levels of temporal dependence, enabling a comprehensive performance assessment.