Oracle approach and slope heuristic in context tree estimation

Abstract: We introduce a general approach to prove oracle properties in context tree selection. The results derive from a concentration condition that is verified, for example, by mixing processes. Moreover, we show the superiority of the oracle approach from a non-asymptotic point of view in simulations where the classical BIC estimator has nice oracle properties even when it does not recover the source. Our second objective is to extend the slope algorithm of \cite{AM08} to context tree estimation. The algorithm gives a practical way to evaluate the leading constant in front of the penalties. We study the slope heuristic underlying this algorithm and obtain the first results on the slope phenomenon in a discrete, non i.i.d framework. We illustrate in simulations the improvement of the oracle properties of BIC estimators by the slope algorithm.