Solomonoff induction of selected bits and online classification

Abstract: Solomonoff induction is a general solution for sequence prediction that started the field of algorithmic information theory around 1960. Let $M$ be the universal semimeasure. After observing a string $ω<em>1 \cdots ω</em>{n-1}$, it predicts the next bit $ω<em>n$ to be $b$ with conditional probability proportional to $M(ω_1 \cdots ω</em>{n-1} b)$. Thus the error of this probability is [ e_ω(n) = 1 - \frac {M(ω1 \cdots ω{n-1}ωn)} {M(ω_1 \cdots ω{n-1} 0) + M(ω{1} \cdots ω{n-1} 1)}. ] If $ω$ a computable infinite sequence with a self-delimiting description of at most $k$ bits, then $\sum_n e_ω(n) \le k\ln 2+O(1)$. Lattimore, Hutter and Gavane consider the problem of predicting computable patterns in a sequence, for example, the pattern of repeated bits $ω= 00\,11\,00\,00\,11\,00\cdots$ They show that the error probabilities of the bits fixed by a computable pattern have limit zero, for example, for repeated bits, one has $\lim_n e_ω(2n) = 0$. We show that for some computable pattern, the sum of $e_ω(n)$ over positions $n$ from the pattern is not bounded. However, the sum of squares $(e_ω(n))<sup>2$ is bounded exponentially in the program length of the pattern. It is also shown that bits of $ω$ that are fixed by a computable classification function $f:{0,1}^* \rightarrow {0,1}$, converges faster: the sum of $(e_ω(n))<sup>2$ is bounded by a polynomial in the program length for programs whose runtime is bounded by any fixed computable time bound.