Regret-Oracle Complexity Tradeoffs in Agnostic Online Learning

Abstract: Agnostic online learning is classically solved via a reduction to the realizable setting, utilizing Littlestone's Standard Optimal Algorithm (SOA) as a base learner. However, the SOA is computationally intractable to execute even for a single round. To overcome this barrier, recent work in oracle-efficient online learning replaces the SOA with a realizable base learner that accesses the concept class exclusively through an offline empirical risk minimization (ERM) oracle. While such agnostic learners achieve near-optimal expected regret, they suffer from a doubly-exponential oracle complexity of $O\big(T^{2^{O(d_\mathrm{LD})}}\big)$, where $d_\mathrm{LD}$ is the Littlestone dimension and $T$ is the number of rounds. In this work, we significantly improve this oracle complexity while relying on an even weaker primitive: a weak-consistency oracle, which merely decides whether a given labeled dataset is realizable. At the core of our approach is an adaptive and dynamic agnostic-to-realizable reduction that actively prunes non-realizable label sequences on the fly. By using the VC dimension ($d_\mathrm{VC}$) to bound the number of dynamically maintained active paths, our algorithm reduces the total query complexity down to $O(T^{d_\mathrm{VC}+1})$ while perfectly preserving near-optimal expected regret. Crucially, this dynamic pruning also yields a memory reduction over the standard reduction. Furthermore, we formally quantify the regret--oracle complexity tradeoff, providing upper bounds that smoothly interpolate between restricted query budgets and attainable expected regret. We complement these with lower bounds proving that any learner restricted to $Q = o(\sqrt{T})$ queries must suffer an expected regret of $Ω(T/Q)$.

• The paper introduces ADEPT, which prunes unrealizable label sequences using a weak-consistency oracle to reduce query complexity to O(T^(VC+1)) while preserving near-optimal regret.
• The paper demonstrates that leveraging the VC dimension reduces memory requirements from O(T^(LD)) to O(t^(VC)), leading to significant computational improvements.
• The paper formalizes a tradeoff between regret and oracle queries, establishing lower bounds that underscore the challenges of achieving low regret with limited query budgets.

Regret-Oracle Complexity Tradeoffs in Agnostic Online Learning

Introduction and Framework

This paper addresses the fundamental tradeoff between expected regret and oracle query complexity in agnostic online learning for general concept classes. The classical information-theoretic approach for agnostic online learning exploits Littlestone’s Standard Optimal Algorithm (SOA), which achieves optimal regret via a reduction to the realizable setting. However, the SOA is computationally intractable—requiring expensive Littlestone dimension calculations and leading to exponential oracle complexity even for finite classes.

Past work on oracle-efficient online learning replaces direct manipulation of the concept class with offline optimization primitives, typically an ERM oracle. These approaches yield near-optimal regret but suffer from a double-exponential query complexity: $\mathcal{O}(T^{2^{\mathcal{O}(LD)}})$, where $LD$ is the Littlestone dimension and $T$ is the number of rounds.

This paper substantially improves oracle complexity, relying on an even weaker primitive: the weak-consistency oracle, which decides only if a dataset is realizable in the concept class (not full ERM optimization). By dynamically pruning unrealizable label sequences in the agnostic-to-realizable reduction, and bounding the number of maintained paths using the VC dimension ($VC$), the proposed algorithm achieves query complexity $\mathcal{O}(T^{VC+1})$ without sacrificing near-optimal regret.

Bottlenecks in Classical Online Learning

Two main computational bottlenecks are highlighted:

• Explosion of Unrealizable Experts: The classical reduction maintains $\mathcal{O}(T^{LD})$ pseudo-label histories under the assumption that all are realizable. By Sauer's Lemma, only $\mathcal{O}(t^{VC})$ dichotomies are realizable at round $t$, so most paths are irrelevant, leading to wasteful computation.
• Intractability of SOA: The SOA requires calculating the Littlestone dimension at every round, a task proven to be computationally intractable and impossible in general for many concept representations. Therefore, query-efficient reductions via offline oracles, such as ERM or weak-consistency oracles, are crucial to achieving tractable algorithms.

Adaptive Dynamic Expert Pruning (ADEPT)

The paper introduces the Adaptive Dynamic Expert Pruning Tree (ADEPT), a reduction that maintains only realizable pseudo-label prefixes, actively pruning impossible extensions using the weak-consistency oracle. This reduction preserves the regret properties of the classical approach but reduces computational complexity as follows:

• Query Complexity Reduction: At each round, ADEPT maintains at most $\mathcal{O}(t^{VC})$ active paths, issuing $\mathcal{O}(t^{VC+1})$ total oracle queries. This is exponential to double-exponential lower in dimension dependence than prior approaches that scale with $LD$0.
• Memory Reduction: While classical reductions require maintaining $LD$1 experts, ADEPT maintains only $LD$2 active paths per round.
• Regret Preservation: ADEPT achieves expected regret $LD$3 for base learners with mistake bound $LD$4, perfectly matching the classical approach when $LD$5 or $LD$6 for oracle-efficient algorithms.

Additionally, ADEPT supports instance-dependent (first-order) regret bounds of the form $LD$7, where $LD$8 is the optimal hindsight loss.

Regret–Oracle Complexity Tradeoff

The paper formalizes the tradeoff frontier between attainable expected regret and the total number of oracle calls. For any $LD$9, there exists a learner requiring at most $T$0 queries and achieving regret $T$1. This interpolation accommodates low-query regimes yielding sublinear regret, and high-query regimes achieving near-optimal rates.

Lower Bounds

Strong lower bounds are established: any learner restricted to $T$2 queries must incur regret $T$3. Even for classes with $T$4, any constant-query algorithm sustains linear regret, closing known gaps and showing the computational barrier's information-theoretic inevitability.

Implications and Future Directions

• Decoupling of Statistical and Computational Complexity: The results demonstrate that, while the statistical price (regret) is governed by the Littlestone dimension, computational cost (oracle queries and memory) is controlled by the VC dimension, which may be arbitrarily lower.
• Practical Impact: The reduction enables practical online learning with strong regret guarantees for classes with small VC dimension, even when the Littlestone dimension is large, using only weak-consistency oracles—thereby broadening the class of efficiently learnable problems.
• Limitations of Oracle-Based Methods: The lower bounds formalize that, for severely restricted query budgets, the oracle-efficient paradigm cannot escape linear or superlinear regret, even for elementary classes.
• Theoretical Open Questions: The gap between attainable regret and queries in certain intermediate complexity regimes remains open, potentially requiring fundamentally novel algorithmic ideas beyond reductions to realizable learners.

Conclusion

This work establishes new theoretical and practical foundations for oracle-efficient agnostic online learning, offering exponential reductions in query and memory complexity using weak-consistency oracles, while maintaining near-optimal regret. The precise regret–oracle tradeoff is mapped, and new lower bounds clarify inherent limitations. The results provide structural insights into the interplay between statistical and algorithmic complexity, guiding future research toward new approaches for efficient online learning in agnostic settings (2605.07155).