Bernoulli-KL Certificate

• Bernoulli-KL certificate is a rigorous statistical guarantee that uses exact KL divergence bounds on Bernoulli data to ensure non-asymptotic accuracy.
• It underpins bandit algorithms like KL-UCB, providing tight confidence intervals and optimal regret control in sequential decision-making.
• The method’s computational efficiency and adaptability extend its use to general estimation tasks in high-probability inference.

The Bernoulli-KL certificate offers a rigorous, data-driven high-probability guarantee and tight non-asymptotic statistical bounds in settings involving Bernoulli (binary) distributions and Kullback–Leibler (KL) divergence. Originating in finite-time bandit algorithms, Chernoff-type deviation inequalities, and non-asymptotic information bounds, the Bernoulli-KL certificate formalizes an exact probabilistic assertion about the gap between an empirical mean and the true mean, or an equivalently sharp lower bound on Kullback–Leibler divergence, with significance both in statistical estimation and sequential decision-making contexts (Garivier et al., 2011, Katariya et al., 2017, Nishiyama, 2019).

1. The Bernoulli Kullback-Leibler Divergence

For $p, q \in [0,1]$, the KL divergence between $\mathrm{Bernoulli}(p)$ and $\mathrm{Bernoulli}(q)$ is defined as

$$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$

with the conventions $0\ln 0 = 0$ and $x\ln(x/0) = +\infty$ for $x > 0$ (Garivier et al., 2011, Katariya et al., 2017, Nishiyama, 2019). This functional form provides the fundamental building block for deriving concentration inequalities and bandit confidence intervals for binary data, and serves as the exact divergence appearing in non-asymptotic lower bounds via the Hammersley–Chapman–Robbins framework (Nishiyama, 2019).

2. KL-UCB Index and High-Probability Certificates

In the stochastic $K$-armed bandit problem with Bernoulli rewards, the KL-UCB algorithm uses the empirical mean $\hat\mu_a(t) = S_a(t)/N_a(t)$ after $N_a(t)$ plays of arm $\mathrm{Bernoulli}(p)$0 to construct a one-sided confidence upper bound: $\mathrm{Bernoulli}(p)$1 At round $\mathrm{Bernoulli}(p)$2, the next action is chosen as $\mathrm{Bernoulli}(p)$3 (Garivier et al., 2011). A key deviation inequality asserts that if $\mathrm{Bernoulli}(p)$4 are i.i.d. $\mathrm{Bernoulli}(p)$5-valued with $\mathrm{Bernoulli}(p)$6, then for any $\mathrm{Bernoulli}(p)$7,

$\mathrm{Bernoulli}(p)$8

and more generally, the random upper bound $\mathrm{Bernoulli}(p)$9 (with $\mathrm{Bernoulli}(q)$0 the empirical mean) satisfies

$\mathrm{Bernoulli}(q)$1

Specializing to $\mathrm{Bernoulli}(q)$2 and applying a union bound gives the Bernoulli–KL certificate: $\mathrm{Bernoulli}(q)$3 That is, with probability at least $\mathrm{Bernoulli}(q)$4, the index $\mathrm{Bernoulli}(q)$5 upper-bounds the true mean $\mathrm{Bernoulli}(q)$6; this is the "certificate" guaranteeing tight high-probability coverage (Garivier et al., 2011).

3. KL-Based Confidence Intervals and the Bernoulli-KL Certificate

The KL-based approach to confidence intervals generalizes the high-probability certificate. For $\mathrm{Bernoulli}(q)$7 i.i.d. Bernoulli$\mathrm{Bernoulli}(q)$8 samples with empirical mean $\mathrm{Bernoulli}(q)$9, and confidence level $$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$0, set $$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$1 and define

$$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$2

$$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$3

These bounds satisfy $$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$4, $$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$5, so with probability $$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$6,

$$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$7

$$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$8, $$d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q}$$9 are the tightest one-sided KL bounds and can be computed efficiently by binary search, exploiting the convexity of $0\ln 0 = 0$0 (Katariya et al., 2017). This construction is operationally the Bernoulli-KL certificate in estimation (Katariya et al., 2017, Garivier et al., 2011).

4. Non-Asymptotic and Exact KL Lower Bounds: Information-Theoretic Perspective

A distinct but closely related Bernoulli-KL certificate emerges in the context of lower-bounding $0\ln 0 = 0$1 for general distributions $0\ln 0 = 0$2, $0\ln 0 = 0$3 through the Hammersley–Chapman–Robbins bound (HCRB). Let $0\ln 0 = 0$4 be a function of interest. The HCRB reads: $0\ln 0 = 0$5 By integrating with respect to an interpolation mixture $0\ln 0 = 0$6, $0\ln 0 = 0$7, and evaluating a closed-form, one obtains: $0\ln 0 = 0$8 with $0\ln 0 = 0$9 and $x\ln(x/0) = +\infty$0 (Nishiyama, 2019). When $x\ln(x/0) = +\infty$1 are Bernoulli and one takes $x\ln(x/0) = +\infty$2, the bound is tight and recovers the canonical KL divergence. Thus, the expression is an exact, data-driven certificate for Bernoulli distributions, as all terms can be estimated empirically.

5. Applications to Multi-Armed Bandits and Regret Optimality

The practical utility of the Bernoulli-KL certificate is most pronounced in bandit algorithms—specifically, KL-UCB. For suboptimal arm $x\ln(x/0) = +\infty$3 with mean $x\ln(x/0) = +\infty$4, the expected number of draws up to time $x\ln(x/0) = +\infty$5 satisfies

$x\ln(x/0) = +\infty$6

implying, for total regret,

$x\ln(x/0) = +\infty$7

These bounds match the Lai–Robbins lower limits for the Bernoulli case, confirming asymptotic optimality (Garivier et al., 2011). Crucially, the KL-index-based choice is justified by the Bernoulli–KL certificate: $x\ln(x/0) = +\infty$8 covers $x\ln(x/0) = +\infty$9 with high probability, ensuring both efficiency and statistical safety.

6. Extensions, Scaling Properties, and Computational Aspects

KL-based bounds extend beyond bandits to general estimation and ranking problems for Bernoulli models. The KL-scaling lemma for $x > 0$0 states

$x > 0$1

and, using Pinsker’s inequality $x > 0$2,

$x > 0$3

These results are instrumental in regret analysis when means are small (Katariya et al., 2017). From a computational perspective, the required KL bounds can always be computed via cheap, one-dimensional convex search, and their complexity is negligible relative to typical algorithmic costs (Katariya et al., 2017).

7. Asymptotics, Information Geometry, and Limit Behavior

In the infinitesimal regime, the Bernoulli-KL certificate aligns with classical information-theoretic limits. If $x > 0$4 is an infinitesimal perturbation of $x > 0$5, Taylor expansions yield

$x > 0$6

where the bound via the HCRB matches the Cramér-Rao lower bound. For Bernoulli, the equivalence holds exactly, tying the Bernoulli-KL certificate to foundational limits in asymptotic parametric inference (Nishiyama, 2019).

---

Summary Table: Key Bernoulli–KL Results

Concept	Formula/Result	Source
Bernoulli KL-divergence	$x > 0$7	(Garivier et al., 2011)
KL-UCB index	$x > 0$8	(Garivier et al., 2011)
High-prob. certificate	$x > 0$9	(Garivier et al., 2011)
Confidence intervals	$K$0	(Katariya et al., 2017)
KL lower bound (exact, Bern.)	$K$1	(Nishiyama, 2019)
Regret bound	$K$2	(Garivier et al., 2011)

These results collectively establish the Bernoulli-KL certificate as a keystone of non-asymptotic statistical inference and online learning with binary data, ensuring exactness, computational practicality, and rigorous high-probability control.