Power guarantees under arbitrarily time-varying gaps

Establish convergence guarantees based on the time-averaged mean gap for best-arm identification when the mean rewards change arbitrarily over time within a bounded range while the identity of the best arm remains fixed.

Background

The paper proves safety for a setting broader than Shifting Means: the mean rewards of all arms may change arbitrarily over time, provided that the best arm remains the same at every timestep. However, ISM’s sample-complexity analysis relies on estimating fixed pairwise gaps and plugging those estimates into the sampling and evidence parameters.

When gaps themselves vary adversarially, the estimated gaps are no longer fixed and may even become zero. The paper identifies convergence guarantees expressed through a time-averaged gap as a concrete unresolved direction for extending power and sample-complexity results to this more general setting.

References

However, approaching that setting from the power side is highly non-trivial. Our current approach is based around plugging in the estimated gaps, which are then not fixed any more, will change adversarially over time, and can even vanish. Showing, for example, convergence guarantees based on the time averaged gap remains a major challenge. We leave that very interesting setting open for future work.

— Best Arm Identification for Bandits with Shifting Means  (2610.10488 - Zierahn et al., 7 Oct 2026) in Section “Discussion {content} Conclusion,” final paragraph