Develop an adaptive FTRL algorithm that achieves the best of both regret bounds without prior smoothness knowledge

Investigate whether there exists a single version of follow-the-regularised-leader with ellipsoidal smoothing that, by adaptively tuning the learning rates (and smoothing parameters), simultaneously achieves the stronger regret bound for smooth losses and the regret bound for general bounded convex losses, without requiring prior knowledge of smoothness.

Background

The chapter analyzes a follow-the-regularised-leader algorithm with ellipsoidal smoothing, providing one regret bound for general bounded convex losses and an improved bound for smooth losses, each requiring different learning-rate and smoothing settings.

The author raises the question of adaptivity: whether a single algorithm can automatically tune to achieve the stronger of the two bounds depending on loss regularity, and states that this is currently unknown.