Optimal regret for context-lumpable bandits

Establish an online regret bound of order \(\widetilde O(\sqrt{r(S+K)T})\) for stochastic context-lumpable bandits with \(S\) contexts, \(K\) arms, and \(r\) unknown context blocks, under arbitrary context and block distributions and without separation assumptions between the reward vectors of distinct blocks.

Background

Context-lumpable stochastic bandits partition $S$ contexts into $r$ unknown blocks such that every arm has the same mean reward across contexts in a common block. The learner observes context identities but not block assignments, and must share observations across contexts while controlling the regret caused by exploration in contexts where an informative arm may perform poorly.

Prior work established nearly optimal sample complexity for finding a near-optimal policy, of order $\widetilde O(r(S+K)/\epsilon^2)$, but its online regret algorithm achieved only $\widetilde O(\sqrt{r^3(S+K)T})$. The unresolved target was to remove the additional factor of $r$ and achieve the lower-bound rate $\widetilde O(\sqrt{r(S+K)T})$. The present paper claims to resolve this problem by controlling exploration during candidate-arm selection without explicitly recovering the context partition.