On the Power of Adaptivity for $\varepsilon$-Best Arm Identification in Linear Bandits

Abstract: We study the minimax sample complexity of $\varepsilon$-best arm identification in linear bandits. Given a compact action set $\mathcal{X}$ that spans $\mathbb{R}<sup>d$ and an unknown reward vector $θ\in\mathbb{R}<sup>d$, the goal is to output an arm $\widehat{x}\in\mathcal{X}$ such that $$\langle \widehat{x},θ\rangle \ge \max_{x\in\mathcal{X}} \langle x,θ\rangle - \varepsilon$$ with probability at least $1-δ$, using as few samples as possible. First, we present a non-adaptive fixed-design method with sample complexity $$\mathcal{O}!\left(\frac{d\log(1/δ)}{\varepsilon<sup>2}+\frac{w(\mathcal{X})<sup>2}{\varepsilon<sup>2}\right)$$, where $w(\mathcal{X})$ is a Gaussian width term dependent on $\mathcal{X}$, and we prove a matching lower bound $$Ω!\left(\frac{d\log(1/δ)}{\varepsilon<sup>2}+\frac{w(\mathcal{X})<sup>2}{\varepsilon<sup>2}\right)$$ for all non-adaptive fixed-design methods. We then turn to adaptive sampling. We raise an important structural question: beyond the canonical basis, are there structured action sets for which adaptivity yields only logarithmic-factor improvements over the optimal non-adaptive rate? We answer in the affirmative for several natural action sets, namely the hypercube, the $\ell_2$ ball, $m$-sets, and multi-task multi-armed bandits. Finally, we provide the first construction of an action set $\mathcal{X}$ for which adaptivity yields a polynomial-factor improvement over every non-adaptive algorithm. A key ingredient behind this separation is an $\ell_2$-norm estimation subroutine: we design an adaptive algorithm that uses $$\mathcal{O}!\left(\frac{d\log(1/δ)}{\varepsilon<sup>2}\right)$$ samples from the unit $\ell_2$ ball in $\mathbb{R}<sup>d$ and outputs an estimate $\widehat r$ satisfying $|\widehat r-|θ|_2|\le \varepsilon$ with probability at least $1-δ$, where $θ$ is the unknown reward vector.