Efficient Fixed-Time Regret with Two Experts

• The paper develops an algorithm that achieves the optimal fixed-time regret bound of √(T/(2π)) + O(1), outperforming classical MWU methods.
• It implements a discretization of the backward heat equation via stochastic calculus, ensuring O(1) time per round with efficient evaluation of smooth functions.
• The analysis unifies probability, stochastic calculus, and information theory to derive sharp, nonasymptotic bounds and validate performance against established lower bounds.

Efficient and optimal fixed-time regret with two experts addresses the foundational online learning problem of prediction with expert advice under a finite horizon and a fixed, small number of experts—most notably, $n=2$. For this classical setting, the objective is to develop algorithms that guarantee the smallest possible fixed-time regret compared to the best expert in hindsight, with stringent computational efficiency and sharp, nonasymptotic bounds. This topic unifies insights from probability, stochastic calculus, and information theory, culminating in algorithms that optimally balance statistical performance and computational requirements (Greenstreet et al., 2022, Eldowa et al., 2023).

1. Model Specification: Two-Expert Online Learning

The formal setup consists of $T \in \mathbb{N}$ rounds with two experts $(i=1,2)$. At each round $t \in \{1,\ldots,T\}$, the adversary reveals a loss vector $\ell_t \in [0,1]^2$. Before observing $\ell_t$, the algorithm chooses a distribution $x_t \in \Delta_2$ (assigning probabilities $x_t(1), x_t(2)$ to each expert), then suffers the expected loss $x_t \cdot \ell_t$, and observes $\ell_t$. The core metric is regret: $$R_T := \sum_{t=1}^T x_t \cdot \ell_t - \min_{i \in \{1,2\}} \sum_{t=1}^T \ell_t(i)$$ Define the cumulative losses $L_t(i) := \sum_{s=1}^t \ell_s(i)$ and the gap $g_t := |L_t(1) - L_t(2)|$. In the restricted binary adversary case $(\ell_t \in \{(1,0), (0,1)\})$, the regret admits the decomposition $R_T = \sum_{t=1}^T p_t \cdot (g_t - g_{t-1})$ (Proposition 2.3), where $p_t$ is the probability assigned to the lagging expert at time $t$ and $g_0=0$, with ties broken evenly (Greenstreet et al., 2022).

2. Optimal Fixed-Time Regret: Main Theorem and Bounds

Efficiently minimizing regret in the fixed-time regime requires surpassing the performance of general-purpose algorithms tailored for large $n$. The presented algorithm guarantees, for arbitrary losses in $[0,1]$ and any $T$,

$R_T \leq \sqrt{\frac{T}{2\pi}} + 1.24$

This matches Cover's 1967 lower bound in the binary loss case up to additive $O(1)$ and demonstrates asymptotic optimality as $T \to \infty$. In comparison, classical Multiplicative Weights Update (MWU) achieves $R_T \leq \sqrt{(T/2)\ln 2} \approx 0.707\sqrt{T}$, which is suboptimal by a factor of approximately $20\%$ relative to the optimal constant $\sqrt{1/2\pi} \approx 0.3989$ (Greenstreet et al., 2022). The result holds regardless of adversary adaptivity or loss distribution within $[0,1]$.

3. Algorithmic Structure and Stochastic Calculus Methods

The algorithm is constructed as a discretization of a solution to the backward heat equation via stochastic (Itô) calculus. Two key smooth functions are defined over $(t,g) \in [0,T] \times \mathbb{R}$:

• $$Q(t,g) = \frac{1}{2} \operatorname{erfc}\left( g/ \sqrt{2(T-t)} \right)$$
• $$R(t,g) = \int_0^g Q(t,x) dx - \frac{1}{2} \int_0^t \partial_g Q(s,0) ds$$

$R$ satisfies the backward heat equation: $\partial_t R + \frac{1}{2} \partial_{gg}R = 0$ and $\partial_g R = Q$. The actual algorithm computes, on each round:

1. The current gap $g = |L_{t-1}(1) - L_{t-1}(2)|$.
2. Sets $p = Q(t, g)$.
3. Picks the lagging expert with probability $p$; the leader otherwise.
4. Observes $\ell_t$ and updates losses.

Pseudocode for the method is:

initialize L0(1)=L0(2)=0 for t=1 to T: g = abs(L[t-1][1] - L[t-1][2]) if g > 0: p = Q(t, g) else: p = 0.5 choose expert i (lagging) with probability p, else leader observe ℓ_t, update L

This approach achieves $O(1)$ time per round, with $Q$ evaluated using $\operatorname{erfc}$ and basic arithmetic, yielding overall $O(T)$ total runtime. There is no need for $O(T^2)$ preprocessing as in Cover's 1967 dynamic programming approach (Greenstreet et al., 2022).

4. Analytical Foundations and Regret Decomposition

The regret guarantees follow from a detailed analysis leveraging discrete and continuous versions of Itô's formula. For the restricted-binary adversary, the regret is recast as $\sum p_t (g_t - g_{t-1})$ and bounded using a discrete Itô formula (Theorem 5.1) for grid functions. The main analytic steps are:

• Application of discrete Itô to $R(t, g_t)$ along the sequence of gap states.
• The dominant term corresponds directly to the regret sum via $p_t = \partial_g R(t,g_{t-1})$.
• Bounds on the terminal and initial values $R(T-1, g_{T-1}) \leq \sqrt{T/(2\pi)}$, $R(0,0) = 0$, and discretization errors $O(1)$ via careful estimation of $(T-t)^{-3/2}$-decaying terms.

The methodology extends to unrestricted losses in $[0,1]$ by exploiting the Lipschitz property and ensuring the discretization error remains $O(1)$. Proposition 2.3 formalizes the key regret-gap relationship in the two-expert case (Greenstreet et al., 2022).

5. Comparison with Prior and Related Approaches

The original algorithm by Cover (1967) achieved $R_T \leq \sqrt{T/(2\pi)} + 1/2$ for binary costs using dynamic programming with $O(T^2)$ preprocessing or per-round costs scaling in $T$. MWU yields $R_T \leq \sqrt{(T/2)\ln 2}$, which is not optimal for two experts. The present algorithm uniquely achieves the minimax optimal rate $R_T \leq \sqrt{T/(2\pi)} + O(1)$ for real-valued losses and $O(1)$ computational complexity per round (Greenstreet et al., 2022).

Extending to bandits with expert advice and fixed, known policies, the EXP4 algorithm specialized to two experts achieves $R_T = O(\sqrt{T\,TV(p,q)})$, where $TV(p, q)$ is the total variation distance between the expert distributions. An alternative KL-divergence-based (OSMD) algorithm provides regret bounds of $O(\sqrt{T K D_{KL}(p \| q)})$, with $K$ the number of actions. Lower bounds show that $\Omega(\sqrt{T D_{KL}(p\|q)})$ is unimprovable in the fixed-time regime up to constants (Eldowa et al., 2023).

6. Information-Theoretic Perspectives and Lower Bounds

In the fixed two-expert setting, regret bounds are tightly controlled by information-theoretic distances between policies. Specifically:

• EXP4 achieves $R_T = O(\sqrt{T\,TV(p, q)})$ with matching lower bound $\Omega(TV(p,q)\sqrt{T})$ up to constants.
• OSMD-style algorithms leveraging KL divergence achieve $R_T = O(\sqrt{T K D_{KL}(p\|q)})$.
• For sufficiently large $T$, all algorithms must satisfy $R_T \geq c\sqrt{H^2(p,q)T}$, with $H^2$ the squared Hellinger distance, and $H^2(p,q) \leq D_{KL}(p\|q)$ (Eldowa et al., 2023).

These results demonstrate the minimax-optimality of the presented methods, characterizing the precise fixed-time regret tradeoffs as a function of expert similarity.

7. Technical Conditions and Scope

All results assume losses in $[0,1]$, guaranteeing validity of exponential update schemes and Pinsker-type inequalities. For the adversarial setup, every action must have nonzero probability under at least one expert to permit unbiased importance-weighted estimation. For the OSMD approach, existence of an interior point in the convex hull of the expert distributions ensures well-defined KL-projections.

A plausible implication is that these algorithms, by optimally leveraging statistical structure and computational efficiency, form a natural benchmark for regret minimization with a small number of experts and fixed horizon, unifying classical, stochastic, and information-theoretic regimes (Greenstreet et al., 2022, Eldowa et al., 2023).