Prediction with Expert Advice

Prediction with expert advice is a classical paradigm in online learning in which a learner combines sequential advice from a set of experts to make predictions, adapting dynamically to minimize its loss relative to the best-performing expert or combination of experts in hindsight. The objective is to develop algorithms with tight regret guarantees, efficient computation, and robustness to various forms of expert behavior, including adaptive and "second-guessing" experts whose advice depends on the learner's own actions. This framework underpins many developments in sequential prediction, online convex optimization, and adaptive ensemble methods.

1. Theoretical Foundations and Problem Setting

In the prototypical prediction with expert advice scenario, the learner operates over $N$ rounds. At round $n$:

• Each expert $k$ provides a predictive function $\gamma^k: [0,1] \to [0,1]$ specifying advice, which may be a static recommendation or, in the general framework, a function of the learner’s prediction $p_n$.
• The learner selects its own forecast $p_n \in [0,1]$.
• The actual outcome $w_n \in \{0,1\}$ is revealed.
• Losses are incurred according to a specified loss function $\lambda(w, p)$, such as the quadratic loss $(p - w)^2$ or the log loss $-\ln(p)$ for $w=1$, $-\ln(1-p)$ for $w=0$.

The learner's performance is measured by its cumulative loss $L_N = \sum_{n=1}^N \lambda(w_n, p_n)$ relative to each expert's realized loss $L^k_N = \sum_{n=1}^N \lambda(w_n, \gamma^k(p_n))$. The standard regret guarantee sought is: $L_N \leq \min_{k=1,\dotsc,K} L^k_N + a_K,$ where $a_K$ is a constant depending on the number of experts and the loss function.

2. Defensive Forecasting: Core Principles and Algorithm

Defensive forecasting is a methodology that constructs the learner’s predictions so as to ensure the non-increase of a carefully chosen forecast-continuous supermartingale over time. The predictive process is defined as follows:

1. At each round, all experts announce continuous prediction functions $\gamma^k$.
2. The learner selects $p_n \in [0,1]$.
3. The actual binary outcome $w_n$ is revealed.
4. Losses are incurred as described above.

A central mathematical device is the supermartingale: $$S_N := \sum_{k=1}^K w_k \exp \left( \kappa \sum_{n=1}^N \left[ \lambda(w_n, p_n) - \lambda(w_n, \gamma^k(p_n)) \right] \right),$$ where $w_k$ are normalized weights, and $\kappa$ is a parameter reflecting the mixability of the loss.

The key mechanism, formalized by a lemma of Levin and Takemura, is that for any forecast-continuous supermartingale, the learner can choose $p_n$ at each round such that $S$ does not increase, regardless of the experts’ strategies. This property ensures robust hedging against all possible outcome realizations. The existence of such a $p_n$ follows from Ky Fan’s minimax theorem, applicable due to the continuity in the learner’s prediction and experts' advice.

3. Regret Guarantees for Mixable Loss Functions

For perfectly mixable loss functions (such as quadratic and log losses), defensive forecasting achieves tight regret bounds:

• Quadratic loss $(p-w)^2$, with $\kappa=2$:

$L_N \leq L^k_N + \frac{\ln K}{2}$

• Log loss, with $\kappa=1$:

$L_N \leq L^k_N + \ln K$

• General perfectly mixable loss:

$L_N \leq L^k_N + \frac{\ln K}{\eta}$

where $\eta$ is the maximal mixability constant for the loss.

These bounds exactly match those attainable by the Aggregating Algorithm (AA) in the fixed-expert case, as formalized in Watkins’s theorem.

4. Handling Second-Guessing Experts

A significant extension provided by defensive forecasting is the capacity to handle “second-guessing” experts, whose prediction functions $\gamma^k(p)$ depend continuously on the learner’s current $p$. The only requirement is the continuity of $\gamma^k$ in $p$; no further regularity is needed.

Classic approaches, such as the AA, cannot generally accommodate this scenario because they rely on fixed expert predictions and thus cannot account for the feedback loop induced by second-guessing. Defensive forecasting, by ensuring the supermartingale property for all possible dependencies, remains robust and optimal in this more general setting.

5. Comparison to Aggregating Algorithm and Practical Implications

The Aggregating Algorithm is optimal among algorithms using exponential weighting of fixed expert advice under perfectly mixable losses, providing the same regret guarantees as defensive forecasting in the fixed-expert regime. However, defensive forecasting surpasses the AA in generality:

• It supports experts whose advice changes adaptively (second-guessing),
• It does not require randomization or any specific exponential weighting of predictions,
• It is constructive: at each round, the learner’s action is computed as the minimax solution to a continuous function, typically implementable via root finding over $[0,1]$ due to the continuity properties.

A summary table: | Setting | Regret Bound | Handles second-guessing experts? | |--------------------------|------------------------------|---------------------| | AA (fixed experts) | $\frac{\ln K}{\eta}$ | No | | Defensive Forecasting| $\frac{\ln K}{\eta}$ | Yes |

6. Algorithmic and Practical Considerations

To implement defensive forecasting:

• At each round and for each possible outcome, the learner computes the effect of a prospective $p_n$ on the supermartingale and selects a prediction to ensure $S$ does not increase.
• For mixable losses with simple form (like square or log loss), the required computation reduces to solving a one-dimensional minimization or root-finding problem, which is efficient in practice.
• Performance (in terms of regret) is not affected by the adaptivity or the continuity of the expert’s advice, provided mixability holds.

Potential limitations include the need for numerically solving the minimax problem at each round, though for many losses and settings this is tractable and can be parallelized or approximated as needed.

7. Significance and Influence

Defensive forecasting both matches the optimal regret rates of classical approaches in standard settings and greatly expands the scope of sequential prediction methods to more complex expert behaviors, including continuous and adaptive response to the learner’s strategy. This advances theoretical understanding and enables new applications in scenarios where experts are learning, strategic, or adaptive, laying the groundwork for robust online ensemble prediction in adversarial and feedback-rich environments.