Optimal Regret for Single Index Bandits

Abstract: We study the $\textit{single-index bandit}$ problem, where rewards depend on an unknown one-dimensional projection of high-dimensional contexts through an unknown reward function. This model extends linear and generalized linear bandits to a nonparametric setting, and is particularly relevant when the reward function is not known in advance. While optimal regret guarantees are known for monotone reward functions, the general non-monotone case remains poorly understood, with the best known bound being $\tilde{\mathcal{O}}(T^{3/4})$ (under standard boundedness and Lipschitz assumptions on the reward function [Kang et al., 2025]). We close this gap by establishing the optimal regret for general single-index bandits. We propose a simple two-phase algorithm, namely, Zoomed Single Index Bandit with Upper Confidence Bound ($\texttt{ZoomSIB-UCB}$), that first estimates the projection direction via a normalized Stein estimator, and then reduces the problem to a one-dimensional bandit using discretization and finally use UCB. This approach achieves a regret of $\tilde{\mathcal{O}}(T^{2/3})$, and improves significantly upon prior work without any additional assumptions. We also prove a matching minimax lower bound of $\tildeΩ(T^{2/3})$, showing that the upper bound is essentially tight. Our upper and lower bounds together provide a sharp characterization of the regret in single-index bandits. Moreover, the empirical results further demonstrate the effectiveness and robustness of our approach.

• The paper introduces ZoomSIB-UCB, using normalized truncated Stein estimation to achieve an estimation error of order T^(-1/3) during the exploration phase.
• The paper leverages one-dimensional discretization with a UCB policy to ensure cumulative regret scales as O(T^(2/3)) even in the sleeping bandit setting.
• The paper establishes a matching minimax lower bound, confirming the optimality of the regret rate for non-monotone, bounded Lipschitz reward functions.

Optimal Regret for Single Index Bandits

Problem Formulation and Context

The paper "Optimal Regret for Single Index Bandits" (2605.09454) addresses the sequential decision-making problem where rewards are governed by a single-index model (SIM): the reward depends on an unknown parameter projection of high-dimensional context vectors through an unknown, possibly non-monotone, function. This generalizes the linear and generalized linear bandit frameworks into a nonparametric regime, accommodating settings where the reward function is not available a priori—e.g., recommendation systems, clinical trials, and online optimization with complex environmental feedback.

Prior work in this area established optimal $\mathcal{O}(\sqrt{T})$ regret for monotone reward functions but only obtained $\tilde{\mathcal{O}}(T^{3/4})$ regret for the general (non-monotone) bounded Lipschitz case [kang2025single]. The central question the paper resolves is: what is the minimax-optimal regret rate for general single-index bandits with bounded, Lipschitz reward functions?

Algorithmic Contributions

The authors introduce ZoomSIB-UCB, a two-phase algorithm leveraging the single-index structure:

Phase 1: Index Parameter Estimation. The algorithm pulls arms uniformly to estimate the unknown projection direction $\theta_*$ using a normalized truncated Stein estimator. Stein's identity ensures that even with an entirely unknown reward function, the expectation aligns with the true parameter vector, up to scaling. The period for exploration is rigorously calibrated to ensure the estimation error is of order $T^{-1/3}$, matching the optimal bin width in Phase 2.

Phase 2: One-Dimensional Discretization and UCB. Once the parameter is estimated, arms' contexts are projected onto the estimated one-dimensional index. The real line is discretized into $N = O(T^{1/3})$ bins, each representing groups of near-identical projected values. At each round, only bins corresponding to the available arms are considered (the "sleeping bandit" setting), and a UCB policy is executed over these bins. The bin width and number are chosen to optimally balance the approximation bias and the statistical cost of exploration.

This algorithm achieves regret $\tilde{\mathcal{O}}(T^{2/3})$, strictly improving upon previous results and does not require any knowledge about the underlying reward function beyond boundedness and Lipschitz continuity.

Regret Guarantee and Lower Bound

Tight Upper Bound

The main theorem establishes—with high probability—a cumulative regret bound:

$$R_T \leq O\left( d^2 T^{2/3} \mathsf{polylog}\left(\frac{dT}{\delta}\right) \right)$$

where $d$ is the feature dimension, and $\delta$ is the failure probability. This is a significant improvement over previous kernel-based methods, while notably incurring a sharper dependence on dimension compared to [kang2025single].

Minimax Lower Bound

A matching information-theoretic lower bound is proven: for any policy and sufficiently large $T$, there exists an instance such that

$\tilde{\mathcal{O}}(T^{3/4})$0

ensuring the upper bound is tight and $\tilde{\mathcal{O}}(T^{3/4})$1 is the correct scaling for regret with general bounded, Lipschitz reward functions.

The proof constructs an explicit family of adversarial reward functions, each embedding localized bump perturbations respecting the functional constraints, and shows via KL analysis and the Bretagnolle-Huber inequality that any algorithm must suffer at least this order of regret due to unavoidable statistical indistinguishability.

Figure 1: Log-log scaling of regret on a quadratic reward function, with ZoomSIB-UCB empirical exponent strictly less than $\tilde{\mathcal{O}}(T^{3/4})$2, confirming minimax-optimal sublinear scaling.

Technical Innovations

ZoomSIB-UCB integrates normalized Stein estimation with discretization strategies from Lipschitz bandits and addresses stochastic action availability using sleeping bandit analysis. The algorithm ensures bin assignments are robust to estimation error, and the regret decomposition factors in both discretization bias and stochastic exploration trade-off. The analysis precisely matches the bias-variance balance, leading to optimal bin width and count.

Sleeping bandit analysis is critical—since action availability is random, standard UCB analysis fails; regret must be evaluated relative to the best available bin at each round, not the global optimum.

Experimental Validation

Empirical results demonstrate strong robustness and efficiency of ZoomSIB-UCB on synthetic topologies (quadratic, asymmetric, zigzag link functions) and high-dimensional real-world datasets.

Figure 2: Empirical log-log regression of cumulative regret; ZoomSIB-UCB demonstrates robust sublinear scaling and outperforms kernel-based and monotone baselines.

ZoomSIB-UCB consistently achieves lower regret across all environments, surpassing the prior GSTOR baseline. The adaptive stopping mechanism enables early exploitation, significantly reducing regret compared to fixed-length exploration phases.

Figure 3: Cumulative regret for ZoomSIB-UCB on the KDD Cup 99 dataset ($\tilde{\mathcal{O}}(T^{3/4})$3), showcasing efficient adaptation to high-dimensional real-world contexts.

Empirical scaling analysis confirms the theoretical scaling exponent; observed regret grows at a rate strictly below $\tilde{\mathcal{O}}(T^{3/4})$4 even in adversarially smooth link environments, with kernel-based baselines adhering to their established $\tilde{\mathcal{O}}(T^{3/4})$5 scaling.

Figure 4: Log-log regret scaling on an asymmetric reward with ZoomSIB-UCB confirming correct sublinear scaling even for flat, zero-gradient tails.

In misspecified models, monotone algorithms collapse to linear regret, while ZoomSIB-UCB maintains sublinear performance—a critical requirement for practical deployment in environments with unknown reward function shapes.

Implications and Future Directions

The results resolve the optimal regret landscape for non-monotone single-index bandits and lay foundational guarantees for nonparametric online decision-making under structural uncertainty. Practically, ZoomSIB-UCB is deployable without prior knowledge of the exploration duration or the reward function, supporting robust applications in recommendation systems, experiment design, and adaptive control.

The minimax characterization suggests that improved regret rates are fundamentally unattainable with only boundedness and Lipschitz continuity. Future research directions include relaxing regularity assumptions, exploring multi-index bandits, and developing alternative estimation procedures beyond Stein's approach for settings where $\tilde{\mathcal{O}}(T^{3/4})$6 may vanish or be ill-conditioned.

Conclusion

The paper rigorously resolves the minimax-optimal regret rate for general single-index bandits under bounded, Lipschitz reward functions, establishing both explicit algorithmic guarantees and matching lower bounds. The ZoomSIB-UCB algorithm integrates Stein estimation, optimal discretization, and sleeping bandit analysis to achieve $\tilde{\mathcal{O}}(T^{3/4})$7 regret. Empirical results confirm robustness, practical deployability, and strict superiority over prior methods. This work solidifies the theoretical foundation for sequential nonparametric contextual learning and opens new directions for extending bandit optimality into more complex function classes and broader online learning regimes.