Bandit PCA with Minimax Optimal Regret

Abstract: We study the bandit-feedback version of online principal component analysis (Bandit PCA): in each round $t = 1,\dots,T$, the adversary selects a $d \times d$ symmetric gain matrix $G_t$ with spectrum in $[0,1]$ and rank at most $r$; the learner simultaneously selects a unit vector $w_t \in S<sup>{d-1}$ and receives the reward $w_t^\top G_t w_t$. The learner receives no other feedback, and aims to minimize the regret against the best unit vector in hindsight. This problem was introduced by Kotlowski and Neu (2019), who gave an algorithm with regret $O(d\sqrt{rT \log T})$ and showed the lower bound of $Ω(r\sqrt{T/\log T})$. We improve upon both of these bounds and essentially bridge the gap between them, establishing the minimax regret of order $r\sqrt{dT}$ up to polylogarithmic factors in $d$ and $T$. The upper bound is attained by a novel algorithm, which combines online mirror descent on the spectrahedron of (real) density matrices with a multiscale exploration scheme in which the eigenspaces with different spectral magnitudes are updated at different rates. For the lower bound, we construct an adaptive adversary that refines a hidden large-reward subspace based on the learner's actions, in such a way that low regret is impossible without estimating the subspace; as a result, lower-bounding the regret reduces to studying the arising subspace estimation problem. Finally, we discuss connections of Bandit PCA with adaptive-measurement quantum tomography.

• The paper establishes a minimax regret rate of O(r√(dT)) (up to polylog factors) by employing a novel online mirror descent procedure on density matrices.
• The paper leverages multiscale layered exploration and variance reduction to efficiently estimate gains and achieve precise subspace discovery in high-dimensional settings.
• The paper validates its approach with a sharp adaptive adversary lower bound that bridges gaps between classical bandit optimization and quantum measurement models.

Minimax Optimal Regret in Bandit PCA

Problem Setting and Prior Work

The paper investigates the online principal component analysis (PCA) problem in the bandit feedback regime, referred to as Bandit PCA (2607.10936). Here, at each round $t$, an adversary selects a positive semi-definite (PSD) gain matrix $G_t$ with spectral norm bounded by $1$ and rank at most $r$; the learner chooses a unit vector $w_t\in S^{d-1}$, and observes only the scalar reward $w_t^\top G_t w_t$. The goal is to minimize regret relative to the best fixed unit vector in hindsight. This is the matrix analog of adversarial linear bandits, with noncommutative geometry: actions are projectors rather than basis vectors, and the feasible set is the spectrahedron.

The Bandit PCA problem was introduced in prior work by Kotłowski and Neu, who established an algorithm with an $O(d\sqrt{rT\log T})$ regret upper bound and a $\Omega(r\sqrt{T/\log T})$ lower bound, leaving a substantial minimax gap—especially in the dependence on $d$ and $r$. The present paper provides a minimax characterization of regret, closing the gap up to polylogarithmic factors. This is accomplished via a tight $G_t$0 upper bound and matching lower bound, both nontrivial extensions of known techniques from bandit linear optimization and matrix analysis.

Algorithmic Framework: Layered OMD, Multiscale Exploration, and Variance Control

The upper bound is achieved via a novel Online Mirror Descent (OMD) procedure over density matrices, combined with a multiscale exploration and layered variance-reduction estimation mechanism. The learner maintains a distribution over the spectrahedron (i.e., the set of PSD matrices with unit trace), employing the $G_t$1 regularizer, updating after estimating the gain with carefully structured bandit feedback. Key innovations involve:

1. Multiscale Layered Exploration: The eigenstructure of the current iterate is partitioned into $G_t$2 layers, grouping eigenvectors by eigenvalue magnitude (dyadically). Exploration and update rates are scheduled hierarchically—layers corresponding to higher-magnitude eigenspaces are explored and estimated more frequently, allowing for efficient allocation of feedback.
2. Layer-wise Batching and Variance Reduction: Gain estimation for off-diagonal blocks within each layer is batched across epochs aligned with the layer's hierarchy, resulting in improved operator-norm concentration. The approach exploits the concentration of random projections on spheres (i.e., Wishart-type bounds), ensuring that the estimation error in any layer scales essentially with the block's dimension rather than full $G_t$3. Diagonal elements are treated using an importance-sampling estimator.
3. Efficient Computational Implementation: The eigenvalue decompositions and other dominant computations are restricted to relevant blocks, amortizing total runtime to $G_t$4 by leveraging the layered schedule.

By synthesizing these ideas, the algorithm achieves regret

$G_t$5

thus qualitatively matching information-theoretic limits in all regimes of $G_t$6 and $G_t$7.

Lower Bound: Adaptive Adversary and Subspace Estimation Complexity

To match the upper bound, a sharp lower bound is proved by constructing an adaptive adversary which forces the learner to solve a sequence of subspace discovery problems. At a high level, the adversary:

• Samples a hidden $G_t$8-dimensional subspace $G_t$9.
• At each round, it constructs gain matrices that “reward” alignment with a direction in $1$0 not yet frequently selected by the learner. As new substantial directions are discovered (detected adaptively based on rewards), the gain matrix is updated orthogonally, compelling continual subspace exploration.

Through a reduction to the problem of estimating subspaces with adaptive queries (whose sample complexity is analyzed via likelihood-ratio bounds on posterior measures on the Grassmannian), it is shown that any learner must incur regret

$1$1

which, up to logarithmic factors, coincides with the upper bound.

Crucially, this lower bound is not merely a direct summation of classical MAB subproblems: it leverages the geometry of the matrix setting, showing that the cost of discovering new principal directions—and the impossibility of focusing exploitative queries without sufficient exploration—raises sample complexity in the high-dimensional regime. The analysis resonates with lower bounds for adaptive quantum tomography, and the adversarial instance connects explicitly to quantum measurement models.

Implications and Theoretical Consequences

Strong Claims:

• The paper establishes, for the first time, the minimax regret rate $1$2 (up to $1$3) in Bandit PCA with bounded-rank losses. Both previous upper and lower bounds are shown to be loose in key regimes.
• The analysis shows that for fixed $1$4 the regret scales as $1$5 (tightening prior dependence on $1$6), while for $1$7 the gap collapses, confirming that the hardness of bandit PCA is not solely dictated by rank.

Connections:

• There are formal and operational links to quantum state estimation and quantum tomography with adaptive measurement—Bandit PCA can be framed as adversarial online tomography for pure or low-rank states.
• The layered, multiscale approach is suggestive for other online matrix optimization settings, including $1$8-PCA and broader measurement-adaptive control in quantum information.

Practical and Algorithmic Impact:

• The proposed algorithm offers computational efficiency competitive with the best previously known, while attaining minimax regret.
• The adaptive exploration mechanism exhibits potential for direct extension to rates in higher-rank or more general online projection problems, and for implementation in practical adaptive sensing settings.

Limitations and Future Directions

There remain several theoretical and algorithmic open questions:

• Reducing or removing the remaining $1$9 factors in the upper and $r$0 in the lower bounds, particularly those arising from matrix concentration and boosting stages in the analysis.
• Extending the analysis formally to the $r$1-PCA selection regime (where the learner plays higher-rank projectors), where only pessimistic bounds exist at the time of writing.
• Further tightening the connection to quantum learning complexity, possibly enabling sharper sample complexity bounds for adaptive tomography in dynamic or adversarial scenarios.
• Optimization of computational complexity further, potentially exploiting the block structure more aggressively to reach $r$2 time in all regimes.

Conclusion

The paper rigorously characterizes the minimax regret in Bandit PCA with bounded-rank gains, expressing both the statistical and computational intricacies intrinsic to noncommutative online optimization. Through the synthesis of multiscale exploration, hierarchical variance control, and sharp geometric lower bounds, it bridges a foundational gap in matrix-valued bandit learning, and provides conceptual tools likely to influence further investigation in both online learning and quantum statistics (2607.10936).