From Betting to Empirical Bernstein LIL

Abstract: This is a verbatim copy of a technical report I wrote in 2017-2018 to obtain the law of the iterated logarithm using the guarantee on the wealth of an online betting strategy.

• The paper establishes time-uniform empirical Bernstein LIL bounds for bounded martingale differences using online betting methods.
• The methodology leverages convex duality and martingale analysis to yield adaptive, variance-sensitive concentration inequalities.
• The explicit betting strategies offer practical sequential decision-making tools with rigorous nonasymptotic performance guarantees.

Technical Overview of "From Betting to Empirical Bernstein LIL" (2605.22124)

Introduction

The paper presents a rigorous derivation of time-uniform concentration inequalities, specifically empirical Bernstein-type law of the iterated logarithm (LIL) bounds, using an explicit connection to online betting algorithms. The work leverages martingale analysis and convex duality to establish tight, adaptive, and empirical LIL bounds for bounded martingale difference sequences. The approach bridges online learning theory—specifically regret-minimizing betting strategies—with classical probabilistic concentration, providing both fresh theoretical insights and practical methodology.

Main Results

The report first develops technical tools, including convex analysis of specific log-barrier-like conjugates and properties of the $\Psi$ function and its inverse, which are central to the resulting concentration inequalities. The main theorem constructs a betting strategy over a bounded outcome sequence $g_t \in [-1,1]$, guaranteeing that the cumulative wealth is lower-bounded by an explicit exponential expression involving $\Psi(|\sum_{i=1}^t g_i| / \sum_{i=1}^t g_i^2)$ and penalty terms reflecting adaptivity and time-uniformity.

A key technical contribution is as follows:

• For any martingale difference sequence $g_1, \ldots, g_t$ with $|g_t| \leq 1$, the result establishes that with probability at least $1-\delta$, uniformly for all $t$,

$$\left|\sum_{i=1}^t g_i\right| \leq \max\left( S_t \, \Psi^{-1}\left(\frac{1}{\alpha S_t}\log \frac{A_t}{\delta}\right), \sqrt{\frac{2}{\alpha}S_t} \right),$$

where $S_t = \sum_{i=1}^t g_i^2$ denotes the quadratic variation and $A_t$ captures log-factors including problem-dependent parameters such as $g_t \in [-1,1]$0 and $g_t \in [-1,1]$1.

• The authors provide explicit, simplifying upper bounds, demonstrating that the LIL bound adapts both to the empirical variance and tightens with larger $g_t \in [-1,1]$2 and smaller $g_t \in [-1,1]$3. The empirical, variance-adaptive structure directly parallels empirical Bernstein bounds, but with strong, non-asymptotic law-of-the-iterated-logarithm characteristics.

Methodological Innovations

The analysis reframes the problem of deriving LIL-type empirical bounds as ensuring guaranteed nonnegative wealth for carefully crafted online linear betting algorithms. This nonasymptotic controlling of cumulative loss (or regret) is mapped to probabilistic concentration via Doob's maximal inequality for martingales, combined with technical manipulation of conjugate functions and careful choice of parameterized betting distributions.

A critical observation is the embedding of stochastic process concentration in the wealth dynamics of universal portfolio strategies. By optimizing dual parameters using convex conjugates, the paper achieves variance scaling and time-uniformity without appeal to the standard union bound or worst-case covering arguments. The explicit formulas also clarify dependencies on the empirical variance and the tuning constants, yielding not just asymptotic optimality but also practically computable bounds.

Numerical Strengths and Claims

The bounds established are:

• Time-uniform: They hold simultaneously for all $g_t \in [-1,1]$4, unlike classical concentration that typically applies for fixed $g_t \in [-1,1]$5.
• Variance-adaptive: The bounds scale with the observed quadratic variation $g_t \in [-1,1]$6, improving tightness for sequences with small variance.
• Optimal up to log factors: The upper bounds match LIL lower bounds up to explicit logarithmic and constant terms.
• Explicit construction: The proofs yield implementable betting strategies, with calculable penalty terms, avoiding loose constants.

These properties assert that the method delivers nonasymptotic LIL-type control in a fully empirical and adaptive fashion—substantially strengthening the practical applicability of such inequalities in online and sequential settings.

Theoretical and Practical Implications

The methodology unifies concepts from online learning (adaptive betting/wealth processes) and martingale concentration, illustrating a general blueprint for deriving tight, time-uniform inequalities in sequential analysis and stochastic optimization. The extension to Banach space-valued random processes in later work suggests wide applicability to high-dimensional or non-Euclidean online problems.

Practically, these bounds enable high-confidence, sequential decision-making—critical in adaptive data analysis, sequential hypothesis testing, and robust online learning. The explicit formulas offer improved empirical calibration, especially when the quadratic variation is much smaller than the time horizon.

Future Developments

The approach motivates further exploration of:

• Extensions to unbounded or heavy-tailed noise environments via robust variants of online betting.
• Application to general martingale frameworks, including vector-valued and non-linear processes.
• Improved constants or further tightening of log-factor dependencies, yielding sharper empirical performance.
• Direct translation of betting strategies into practical sequential algorithms for high-stakes or risk-sensitive online decision problems.

Conclusion

"From Betting to Empirical Bernstein LIL" (2605.22124) delivers a technical, principled derivation of empirical, time-uniform LIL bounds for martingale difference sequences through an explicit connection to online betting strategies. The framework achieves variance adaptation, time-uniform control, and practical computability, enriching the theory and application of sequential concentration inequalities in machine learning and statistics.