Frequentist forecasting in regime-switching models with extended Hamilton filter

Abstract: Psychological change processes, such as university student dropout in math, often exhibit discrete latent state transitions and can be studied using regime-switching models with intensive longitudinal data (ILD). Recently, regime-switching state-space (RSSS) models have been extended to allow for latent variables and their autoregressive effects. Despite this progress, estimation methods for handling both intra-individual changes and inter-individual differences as predictors of regime-switches need further exploration. Specifically, there's a need for frequentist estimation methods in dynamic latent variable frameworks that allow real-time inferences and forecasts of latent or observed variables during ongoing data collection. Building on Chow and Zhang's (2013) extended Kim filter, we introduce a first frequentist filter for RSSS models which allows hidden Markov(-switching) models to depend on both latent within- and between-individual characteristics. As a counterpart of Kelava et al.'s (2022) Bayesian forecasting filter for nonlinear dynamic latent class structural equation models (NDLC-SEM), our proposed method is the first frequentist approach within this general class of models. In an empirical study, the filter is applied to forecast emotions and behavior related to student dropout in math. Parameter recovery and prediction of regime and dynamic latent variables are evaluated through simulation study.