Spectral bandits

Abstract: Smooth functions on graphs have wide applications in manifold and semi-supervised learning. In this work, we study a bandit problem where the payoffs of arms are smooth on a graph. This framework is suitable for solving online learning problems that involve graphs, such as content-based recommendation. In this problem, each item we can recommend is a node of an undirected graph and its expected rating is similar to the one of its neighbors. The goal is to recommend items that have high expected ratings. We aim for the algorithms where the cumulative regret with respect to the optimal policy would not scale poorly with the number of nodes. In particular, we introduce the notion of an effective dimension, which is small in real-world graphs, and propose three algorithms for solving our problem that scale linearly and sublinearly in this dimension. Our experiments on content recommendation problem show that a good estimator of user preferences for thousands of items can be learned from just tens of node evaluations.

• The paper introduces a stochastic bandit framework on graphs that exploits reward smoothness via the Laplacian eigenstructure.
• The methodology includes SpectralUCB, SpectralTS, and Spectral Eliminator algorithms that reduce regret scaling to the effective dimension d, rather than the total number of arms.
• Empirical results on synthetic and real-world datasets show that spectral methods outperform classical linear bandit approaches in both regret minimization and computational efficiency.

Spectral Bandits: Analysis and Algorithms for Graph-Structured Bandit Problems

Problem Formulation and Setting

The paper "Spectral bandits" (2604.25272) defines a stochastic bandit framework where the reward function is smooth with respect to the structure of an underlying graph. Formally, each action (or "arm") corresponds to a node in the (known) undirected graph. Payoffs are modeled such that neighboring nodes have similar expected rewards due to the spectral smoothness hypothesis, which leverages the eigenstructure of the graph Laplacian.

The key technical insight is to exploit the low-frequency eigenvectors of the Laplacian to efficiently approximate the reward function, thus reducing the effective learning dimensionality. The standard multi-armed bandit and linUCB solutions are fundamentally inadequate: in scenarios where the arm set is very large (such as recommender systems), the ambient dimension $D$ is prohibitive. The proposed approach aims to scale regret not with $N$ or $D$ but with a refined, typically much smaller, "effective dimension" $d$—a function of the reward smoothness and the graph spectrum.

Figure 1: Difference between the new effective dimension $d$ and the prior definition $D_{\text{old}}$ for real-world datasets. Left: Flixster ($N=972$), Middle: Movielens ($N=618$), Right: LastFM ($N=804$).

Theoretical Tools: Smoothness and Effective Dimension

A central concept is the graph smoothness of the reward function $f$, given by:

$N$0

which can be succinctly represented in the Laplacian eigenbasis as $N$1, where $N$2 are the coefficients in the Laplacian eigenbasis and $N$3 are eigenvalues.

A rigorous measure of statistical complexity is the "effective dimension" $N$4 defined as:

$N$5

This is generally much less than $N$6, especially when the Laplacian spectrum increases steeply, corresponding to rapidly decaying high-frequency components of the reward function.

Algorithms

SpectralUCB

SpectralUCB is an adaptation of LinUCB, estimating the Laplacian-eigenbasis coefficients with a Laplacian-based regularization, and using data-dependent confidence widths informed by the effective dimension $N$7. The algorithm achieves cumulative regret:

$N$8

with high probability, where $N$9 is typically $D$0.

SpectralTS

SpectralTS generalizes Thompson Sampling to the spectral bandit model, sampling from a Gaussian posterior over the coefficients. The confidence ellipsoid and variance adapt to the observed data via the Laplacian-based regularization. Regret upper bounds are also in $D$1, with dependence on logarithmic spectral properties.

Spectral Eliminator

A phase-based arm-elimination procedure akin to ImprovedUCB eliminates arms whose empirical means (with spectral confidence bounds) are estimated to be suboptimal, ensuring regret scales as $D$2.

Lower Bounds

The paper proves that the "effective dimension" is also a lower bound for regret: for any $D$3 and $D$4, there exists an instance such that any algorithm incurs $D$5 regret. Thus, the spectral algorithms are information-theoretically optimal up to log factors.

Empirical Results

The algorithms are evaluated on both synthetic graphs (Erdős-Rényi, Barabási–Albert, and lattices) and real-world datasets (Movielens, Flixster, LastFM). In all cases, spectral methods achieve orders-of-magnitude lower regret compared to classical LinUCB or LinearTS—especially when the number of rounds is much smaller than the number of arms.

Figure 2: Movielens dataset—cumulative regret for 20 randomly selected users. Spectral algorithms consistently outperform linear baselines even for short horizons.

Figure 3: Movielens dataset—cumulative regret for one random user. Robust improvement of spectral methods.

Computationally, techniques such as lazy updates (for UCB width caching) and the Sherman-Morrison inversion formula are employed, resulting in practical wall-clock performance comparable to or better than the baselines, especially when using low-rank approximations of the eigenbasis.

Figure 4: The impact of lazy updates and Sherman-Morrison formula: significant speedup in runtime.

Implications, Practical Guidance, and Future Directions

Spectral bandits enable scalable online learning for graph-structured action spaces, relevant for recommender systems, networked experimental design, and semi-supervised learning settings where response surfaces are smooth in the graph spectrum. The proposed algorithms are the first to achieve minimax optimal rates (up to log factors) as a function of a data-dependent, computable notion of effective dimension.

Practically, the construction of the similarity graph (choice of neighbors, weighting) and selection of Laplacian eigenvectors (basis truncation) critically affect the regret and computational profile. Over-regularizing (large $D$6) can discard useful smoothness structure; optimal empirical performance requires calibration of hyperparameters and graph construction.

Theoretically, the work connects capacity of Gaussian channels to online learning (the effective dimension formula has an information-theoretic interpretation), and demonstrates that in the regime $D$7, graph structure can be exploited for substantial regret reductions.

A major open question is the extension to changing or learned graphs, temporal non-stationarity, and adversarial feedback. Furthermore, exploration of non-linear reward models (e.g., via graph kernels or neural embeddings) is a promising future direction.

Conclusion

"Spectral bandits" provides a comprehensive framework and scalable algorithms for stochastic multi-armed bandits on graphs with smooth reward functions. By leveraging the Laplacian spectral structure, the authors achieve minimax-optimal regret scaling in terms of the effective dimension, validated by both theoretical analysis and extensive empirical evidence. The results set a benchmark for structured bandit learning, with implications across online decision making in structured domains.