Context Tree Prior Distributions based on Node Weighting with exact Bayes Factors

Abstract: Variable-length Markov chains (VLMCs) are a flexible class of higher-order Markov models that admit a natural representation as context trees. The Bayesian approach to VLMCs involves assigning prior distributions to both the tree structure and the transition probabilities, with inference performed by integrating over nuisance parameters. We propose a Bayesian framework for context trees based on a novel class of prior distributions on tree space. The approach extends recursive methods used to compute the marginal likelihood and identify a posterior mode tree, accommodating flexible weighting schemes corresponding to these prior distributions. The proposed method generalizes existing approaches, as this class includes, among others, distributions generated by branching processes. This flexibility enables the selection of priors based on desired properties. Our framework enables efficient posterior exploration and allows model comparison and hypothesis testing via Bayes factors for context trees. To show the power of our methods, we conducted a simulation study comparing different prior choices, along with procedures based on Bayes factors to select the maximal depth and perform model selection.