Adaptive first- and second-order regret bounds for NormalHedge

Prove that the NormalHedge algorithm admits adaptive first-order and second-order regret bounds in the full-information experts setting, matching the adaptive guarantees known for optimally tuned Hedge; specifically, establish bounds that scale with the square root of (i) the cumulative loss of the best expert times log N (first-order) and (ii) the cumulative variance under the algorithm’s weight distribution times log N (second-order).

Background

The paper analyzes regret guarantees for online drafter selection and discusses classical adaptive bounds for Hedge. Cesa-Bianchi et al. (2007) showed that when the learning rate is optimally tuned for Hedge, the algorithm enjoys adaptive first-order (small-loss) and second-order (small-variance) regret bounds.

The authors adopt NormalHedge as the base learner in HedgeSpec and note that, unlike Hedge with tuned learning rate, NormalHedge has not been proven to satisfy these strong adaptive bounds, although this has been conjectured in prior work. They report empirical evidence consistent with the conjecture, highlighting a theoretical gap regarding NormalHedge’s adaptivity.