Quantum Multi-Armed Bandits and Linear Bandits: Lower Bounds and Algorithms

Abstract: We study quantum multi-armed bandits (QMAB) and quantum linear bandits (QLB) in the model of Wan et al. [2023], where the learner queries each arm or action through a quantum reward oracle or its inverse. Prior work gives algorithms over horizon $T$ with regret $O(K\log T)$ for QMAB with $K$ arms and $O(d^{2\operatorname{polylog}} T)$ for $d$-dimensional QLB. This leaves open whether the $K\log T$ scale is unavoidable and whether the $d<sup>2$ dependence can be improved. We prove the first minimax lower bounds of $Ω(K\log(T/K))$ for QMAB and $Ω(d\log(T/d))$ for finite-action QLB, resolving the question raised by Wan et al. [2023] of whether regret independent of $T$ is achievable. At the heart of our argument is a high-confidence single-arm quantum testing lower bound for distinguishing a fixed reward mean from an interval of alternatives, proved by the polynomial method and a Remez-type inequality for trigonometric polynomials. A bandit-to-testing reduction then lifts it to the QMAB lower bound, while a linear embedding gives the finite-action QLB lower bound. Complementing the lower bounds, we give a design-based elimination algorithm for finite-action QLB. When the action set has size $\operatorname{poly}(d)$, its regret is linear in $d$, improving the prior $d<sup>2$ dependence and matching our lower bound up to polylogarithmic factors. The algorithm couples a low-bias low-variance quantum mean estimator with a small-support $G$-optimal design through a query allocation matched to the design weights. The design-based elimination reduces the dimension dependence from $d<sup>2$ to $d<sup>{3/2}$ when using Quantum Monte Carlo estimates. The low-variance estimator then makes reconstruction error aggregate through variance rather than worst-case absolute error, removing the remaining $\sqrt d$ factor.

• The paper proves minimax regret lower bounds of Ω(K log(T/K)) for quantum multi-armed bandits and Ω(d log(T/d)) for finite-action quantum linear bandits using point-versus-interval testing and trigonometric-polynomial methods.
• The paper introduces LV-G-Elim, which combines small-support G-optimal designs with a low-variance quantum mean estimator to achieve nearly linear regret dependence on dimension d for polynomial-size action sets.
• The results show that T-independent regret is impossible in the quantum reward-oracle model, while leaving open optimal guarantees for general action sets, exponentially large action spaces, and reduced logarithmic factors.

Overview

This paper studies stochastic bandit problems in the quantum reward oracle model introduced by Wan et al., where each arm or action is accessed through a unitary encoding its reward distribution, and the learner may apply the oracle or its inverse at most once per round. Prior work in this model achieved regret $O(K\log T)$ for quantum multi-armed bandits (QMAB) with $K$ arms and $O(d^2\operatorname{polylog}T)$ for quantum linear bandits (QLB) of dimension $d$. Two questions were left open: whether the logarithmic dependence on the horizon $T$ is intrinsic, and whether the quadratic dimension dependence for linear bandits can be improved. The paper resolves both: it proves minimax lower bounds of $\Omega(K\log(T/K))$ for QMAB and $\Omega(d\log(T/d))$ for finite-action QLB, and gives an algorithm, LV-G-Elim, whose regret is nearly linear in $d$ when the action set has polynomial size.

The lower bound mechanism: point-versus-interval testing

Neither standard classical lower-bound technique transfers directly to the quantum setting. The two-point change-of-measure argument fails because quantum mean estimation distinguishes two distributions with means differing by $\epsilon$ using only $O(1/\epsilon)$ queries rather than $K$0 samples. Instance-dependent bounds fail for a different reason: distinguishing a fixed pair of distinct unitary oracles costs a constant number of coherent queries regardless of the target error probability, so no $K$1 growth arises from point-versus-point comparisons.

The paper recovers the required logarithmic dependence by replacing the fixed alternative with an interval of alternatives. The hard family uses a needle-in-a-haystack structure: arm 1 has mean $K$2, all other arms have baseline mean $K$3 under the null instance, and each alternative instance elevates one candidate arm's mean to an unknown value in $K$4. A low-regret policy must distinguish, for every suboptimal arm, the baseline mean from this entire interval — a task that provably requires growing query cost as the error probability shrinks.

The technical core is a single-arm quantum testing lemma. For a Bernoulli oracle $K$5 parameterized by $K$6, the acceptance probability of any $K$7-query test is a real trigonometric polynomial of degree at most $K$8. A Remez-type inequality for trigonometric polynomials (due to Ganzburg) then shows that if such a polynomial is bounded below by a constant at the baseline angle while uniformly at most $K$9 on an interval separated from it, its degree must be $O(d^2\operatorname{polylog}T)$0. This forces $O(d^2\operatorname{polylog}T)$1. A truncation argument extends the fixed-query bound to tests making a random number of queries, which is what bandit policies induce.

The bandit-to-testing reduction converts a policy with worst-case regret $O(d^2\operatorname{polylog}T)$2 into single-arm tests with both error probabilities $O(d^2\operatorname{polylog}T)$3, by declaring "alternative" whenever a candidate arm is selected more than half the time; either error event forces $O(d^2\operatorname{polylog}T)$4 regret on some instance. Combining with the testing bound yields $O(d^2\operatorname{polylog}T)$5, hence $O(d^2\operatorname{polylog}T)$6. Realizing the same hard family as a linear bandit with exactly $O(d^2\operatorname{polylog}T)$7 actions — using actions that share a common reference direction so that $O(d^2\operatorname{polylog}T)$8 rather than $O(d^2\operatorname{polylog}T)$9 — yields the finite-action QLB lower bound of $d$0.

Two consequences deserve emphasis. First, these are the first minimax regret lower bounds in this model, and they resolve the open question of Wan et al.: $d$1-independent regret is impossible; logarithmic regret is the worst-case scale even for constant-gap instances. Second, since the hard instances use Bernoulli rewards, the bounds apply equally under bounded-variance assumptions. The reduction does assume classical per-round arm selection, so it does not extend to models permitting superposed arm selection over multiple arms.

LV-G-Elim: design-based elimination with low-variance estimation

The algorithmic contribution addresses the finite-action regime directly, where prior guarantees came only from general-action analyses. LV-G-Elim follows classical phased elimination with $d$2-optimal exploration, but the quantum setting changes the accounting in two ways that shape the design.

First, the support size becomes a leading term rather than an additive overhead. Quantum mean estimation has leading cost $d$3, but each support action requires its own estimator call with a constant minimum number of queries, and each query incurs regret proportional to the phase accuracy. The paper therefore constructs a small-support approximate $d$4-optimal design via the Kiefer–Wolfowitz theorem combined with the rounding method of Allen-Zhu et al., achieving support size $d$5 and $d$6-value $d$7 on an active set of rank $d$8. In the classical setting small support is merely a computational convenience; here it is essential.

Second, the choice of estimator determines how errors aggregate during extrapolation from support estimates to all active actions. Using QMC, worst-case absolute errors propagate through the prediction weights and inflate by $d$9, forcing support accuracy $T$0 and yielding a $T$1 regret dependence. LV-G-Elim instead uses the nondestructive unbiased amplitude estimator of Cornelissen and Hamoudi, specialized to reward oracles without requiring controlled oracle access. With a query allocation $T$2 matched to the design weights, independent fluctuations aggregate through variance, and the $T$3 loss disappears entirely. Median amplification over $T$4 independent runs then lifts constant-probability accuracy to high probability over the full active set.

The resulting guarantee is expected regret $T$5. When $T$6, this is nearly linear in $T$7 and matches the $T$8 lower bound up to polylogarithmic factors — improving the prior $T$9 dependence. The QMC variant, QMC-G-Elim, achieves $\Omega(K\log(T/K))$0: it avoids the $\Omega(K\log(T/K))$1 factor (union bounding only over the support) but pays the $\Omega(K\log(T/K))$2 loss. The paper notes the trade-off explicitly: for exponentially large action sets with $\Omega(K\log(T/K))$3, the QMC variant can give better dimension dependence.

Limitations and open questions

The paper is explicit about several gaps. For QMAB, logarithmic slack remains between the $\Omega(K\log(T/K))$4 lower bound and the $\Omega(K\log(T/K))$5 upper bound. For general action sets, the optimal dimension dependence is unresolved: upper bounds stand at $\Omega(K\log(T/K))$6 (Wan et al.) and $\Omega(K\log(T/K))$7 (a specialization of Hikima et al.'s kernelized analysis given in the appendix), against the $\Omega(K\log(T/K))$8 lower bound. Whether the $\Omega(K\log(T/K))$9 factor of LV-G-Elim can be reduced toward the classical $\Omega(d\log(T/d))$0 while retaining linear dimension dependence is left open, as is the possibility of mild $\Omega(d\log(T/d))$1-dependence in the lower bound for larger fixed action sets. The extension of the upper bound to bounded-variance rewards is sketched but not completed, pending a deterministic-budget version of the variance-aware estimator.

Conclusion

The paper establishes that logarithmic-in-$\Omega(d\log(T/d))$2 regret is unavoidable for quantum multi-armed bandits and finite-action quantum linear bandits in the reward oracle model, via a novel point-versus-interval testing lower bound built on the polynomial method and a Remez-type inequality for trigonometric polynomials. Its LV-G-Elim algorithm nearly matches the linear bandit lower bound for polynomial-size action sets, reducing the dimension dependence from $\Omega(d\log(T/d))$3 to $\Omega(d\log(T/d))$4 by coupling small-support $\Omega(d\log(T/d))$5-optimal designs with a low-bias low-variance quantum mean estimator.