Spectral bandits for smooth graph functions

Abstract: Smooth functions on graphs have wide applications in manifold and semi-supervised learning. In this paper, we study a bandit problem where the payoffs of arms are smooth on a graph. This framework is suitable for solving online learning problems that involve graphs, such as content-based recommendation. In this problem, each item we can recommend is a node and its expected rating is similar to its neighbors. The goal is to recommend items that have high expected ratings. We aim for the algorithms where the cumulative regret with respect to the optimal policy would not scale poorly with the number of nodes. In particular, we introduce the notion of an effective dimension, which is small in real-world graphs, and propose two algorithms for solving our problem that scale linearly and sublinearly in this dimension. Our experiments on real-world content recommendation problem show that a good estimator of user preferences for thousands of items can be learned from just tens of nodes evaluations.

• The paper introduces the effective dimension derived from the graph Laplacian to regularize learning and reduce dependence on high-dimensional node spaces.
• It presents two algorithms—Spectral UCB and Spectral Eliminator—that achieve regret scaling as d√T, outperforming classical methods like LinUCB.
• Empirical evaluations on recommender systems confirm the approach's sample efficiency and scalability, with significant reductions in cumulative regret.

Spectral Bandits for Smooth Graph Functions: Summary and Implications

Problem Formulation and Motivation

The paper "Spectral Bandits for Smooth Graph Functions" (2604.18420) addresses sequential learning and decision-making problems where action payoffs are assumed to vary smoothly across a predefined graph structure. Specifically, the model considers the scenario where arms in a stochastic bandit setting correspond to nodes of a graph, and their expected rewards are governed by a smooth function that can be decomposed into the eigenvectors of the graph Laplacian with low eigenvalues. This structure is motivated by several domains: content-based recommendation, social network advertising, and other settings wherein the similarity structure of actions is encoded prior to learning.

The significance of this formulation is its departure from classical linear bandits, where action features are usually assumed independent or equally informative. Here, the graph structure is exploited to regularize learning, reflecting real-world assumptions of smoothness—neighboring items or users in the graph are more likely to yield similar rewards.

Technical Contributions and Algorithmic Framework

The primary technical innovation is the introduction of the effective dimension $d$, derived from the spectral properties of the Laplacian. The effective dimension acts as a proxy for the number of significant spectral components active within the time horizon $T$ and is typically much lower than the ambient dimension $D = N$ (number of nodes). The formal definition incorporates the growth rate of eigenvalues and their relation to $T$, thereby quantifying the concentration of reward functions in "smooth" directions.

Two algorithms are proposed:

1. Transductive Upper Confidence Bound (Spectral UCB): This algorithm adapts LinUCB to the spectral domain, utilizing penalization via a spectral norm that preferentially regularizes components aligned with high eigenvalues (less smooth directions). Predictions and confidence intervals are computed in the graph spectral basis, and the cumulative regret is shown to scale as $d\sqrt{T}$ (see Theorem 1), in contrast to classical linear bandits with regret scaling as $D\sqrt{T}$.
2. Spectral Eliminator: Inspired by phase-based elimination strategies (e.g., Improved UCB, SupLinUCB), this algorithm iteratively focuses exploration on remaining promising arms and achieves the regret upper bound $\tilde{O}(\sqrt{d T})$ (Theorem 2). The approach leverages the independence of reward samples across phases, facilitating tighter concentration via self-normalized martingale inequalities.

Both algorithms avoid dependence on $N$ via the effective dimension and spectral regularization, making them theoretically and practically superior when $T < N$ and graph smoothness assumptions are valid.

Analytical Results

The core of the analysis relies on bounding confidence ellipsoids and leveraging matrix determinant inequalities (Sylvester’s theorem, regularization geometry), enabling regret bounds that are independent of the ambient dimension. The key is that the confidence width and cumulative regret depend only logarithmically on $T$ and linearly on $T$0.

A notable result is the formal proof (via determinant maximization under coordinate constraints) that the spectral penalty and algorithmic confidence intervals can be computed in a manner that scales computationally and statistically with $T$1, rather than $T$2. The analysis also demonstrates that the effective dimension grows sublinearly with $T$3 in practical graphs, especially those with rapidly increasing Laplacian eigenvalues (social networks, recommendation item graphs).

Experimental Evaluation

Empirical validation is conducted on both synthetic (Erdős–Rényi, Barabási–Albert, lattice graphs) and real-world datasets (MovieLens, Flixster), employing item/user rating prediction in recommender systems. Strong numerical results are reported:

• In the MovieLens and Flixster recommendation experiments, average cumulative regret for Spectral UCB was roughly one-third to one-fourth that of LinUCB when $T$4, with the effective dimension much smaller than $T$5 (e.g., $T$6, $T$7).
• Experiments show that a good estimator for user preferences over thousands of items can be constructed from just tens of node evaluations, highlighting sample efficiency.
• Computational scalability is addressed via spectral approximation (fast SDD solvers), lazy UCB updates, and iterative matrix inversion. The reduced basis experiments further demonstrate that using a small fraction ($T$8) of the spectral basis maintains performance while significantly accelerating computation.

Practical and Theoretical Implications

The results have substantial implications for online learning, recommendation systems, and social network analysis:

• Regret Scaling: Algorithms exploiting graph smoothness achieve regret bounds that do not scale with the potentially massive action spaces, as long as smoothness in the graph holds. This is critical for real-world graphs with thousands or millions of nodes.
• Sample Efficiency: In applications like content-based recommendation, a small number of item evaluations suffices to estimate user preferences accurately, enabling effective cold-start solutions.
• Spectral Regularization: Penalizing high-frequency (non-smooth) Laplacian directions is a principled mechanism for biasing learning towards plausible reward functions, an approach that generalizes the concept of regularization in kernel and linear bandits to the graph domain.
• Algorithmic Simplicity and Scalability: Fast solvers and reduced spectral basis selection render the approach computationally feasible at scale, facilitating deployment in real-world recommender systems.

Theoretically, the approach motivates future work on sparsity and low-rank spectral representations (e.g., reward functions as combinations of a small number of Laplacian eigenvectors), connections to manifold learning, and extension to partial observability and adversarial settings. The connection between effective dimension and metric dimension of ellipsoids opens avenues for further exploration in high-dimensional statistics and online convex optimization.

Conclusion

The paper presents a rigorous spectral bandit framework for learning smooth functions over graphs, effectively leveraging graph structure to achieve regret bounds and computational efficiency unattainable in classical bandit models. The introduction of the effective dimension, together with practical algorithms and theoretical guarantees, enables robust solutions for sequential decision-making in large-scale, structured action spaces with applications in recommender systems and network analysis. Future directions include extension to sparser spectral models and richer bandit feedback structures.