--- title: Bernoulli-KL Certificate url: https://www.emergentmind.com/topics/bernoulli-kl-certificate type: topic --- # Bernoulli-KL Certificate The Bernoulli-KL certificate offers a rigorous, data-driven high-probability guarantee and tight non-asymptotic statistical bounds in settings involving Bernoulli (binary) distributions and Kullback–Leibler (KL) divergence. Originating in finite-time bandit algorithms, Chernoff-type deviation inequalities, and non-asymptotic information bounds, the Bernoulli-KL certificate formalizes an exact probabilistic assertion about the gap between an empirical mean and the true mean, or an equivalently sharp lower bound on Kullback–Leibler divergence, with significance both in statistical estimation and sequential decision-making contexts [1102.2490][1703.06513][1907.00288]. ## 1. The Bernoulli Kullback-Leibler Divergence For $p, q \in [0,1]$, the KL divergence between $\mathrm{Bernoulli}(p)$ and $\mathrm{Bernoulli}(q)$ is defined as \[ d(p, q) = p \ln \frac{p}{q} + (1 - p) \ln \frac{1-p}{1-q} \] with the conventions $0\ln 0 = 0$ and $x\ln(x/0) = +\infty$ for $x > 0$ [1102.2490][1703.06513][1907.00288]. This functional form provides the fundamental building block for deriving concentration inequalities and bandit confidence intervals for binary data, and serves as the exact divergence appearing in non-asymptotic lower bounds via the Hammersley–Chapman–Robbins framework [1907.00288]. ## 2. KL-UCB Index and High-Probability Certificates In the stochastic $K$-armed bandit problem with Bernoulli rewards, the KL-UCB algorithm uses the empirical mean $\hat\mu_a(t) = S_a(t)/N_a(t)$ after $N_a(t)$ plays of arm $a$ to construct a one-sided confidence upper bound: \[ U_a(t) = \max \left\{ q \in [\hat\mu_a(t), 1] : N_a(t)\,d(\hat\mu_a(t), q) \le \log t + c \log \log t \right\} \] At round $t$, the next action is chosen as $A_t = \arg\max_a U_a(t)$ [1102.2490]. A key deviation inequality asserts that if $X_1, \dots, X_n$ are i.i.d. $\{0, 1\}$-valued with $\mathbb{E}[X_i] = p$, then for any $q > p$, \[ \Pr\left( \frac{1}{n} \sum_{s=1}^n X_s \ge q \right) \le \exp\left(-n\,d(p, q)\right) \] and more generally, the random upper bound $u_n = \max\{q > \hat p_n : n\,d(\hat p_n, q) \le \delta\}$ (with $\hat p_n$ the empirical mean) satisfies \[ \Pr(u_n < p) \le e \lceil \delta \log n \rceil\,e^{-\delta} \] Specializing to $\delta = \log t + 3 \log \log t$ and applying a union bound gives the Bernoulli–KL certificate: \[ \Pr(U_a(t) < \mu_a) \le O\left((t \log^3 t)^{-1}\right) \] That is, with probability at least $1 - 1/t$, the index $U_a(t)$ upper-bounds the true mean $\mu_a$; this is the "certificate" guaranteeing tight high-probability coverage [1102.2490]. ## 3. KL-Based Confidence Intervals and the Bernoulli-KL Certificate The KL-based approach to confidence intervals generalizes the high-probability certificate. For $n$ i.i.d. Bernoulli$(p)$ samples with empirical mean $\hat p$, and confidence level $1-\delta$, set $\delta' = \delta/2$ and define \[ \mathrm{UCB} = \sup\{q \in [\hat p, 1] : n\,d(\hat p, q) \le \ln(1/\delta')\} \] \[ \mathrm{LCB} = \inf\{q \in [0, \hat p] : n\,d(\hat p, q) \le \ln(1/\delta')\} \] These bounds satisfy $\Pr(p > \mathrm{UCB}) \le \delta'$, $\Pr(p < \mathrm{LCB}) \le \delta'$, so with probability $\ge 1-\delta$, \[ \mathrm{LCB} \le p \le \mathrm{UCB} \] $U$, $L$ are the tightest one-sided KL bounds and can be computed efficiently by binary search, exploiting the convexity of $q \mapsto d(\hat p, q)$ [1703.06513]. This construction is operationally the Bernoulli-KL certificate in estimation [1703.06513][1102.2490]. ## 4. Non-Asymptotic and Exact KL Lower Bounds: Information-Theoretic Perspective A distinct but closely related Bernoulli-KL certificate emerges in the context of lower-bounding $\mathrm{KL}(P\|Q)$ for general distributions $P$, $Q$ through the Hammersley–Chapman–Robbins bound (HCRB). Let $f$ be a function of interest. The HCRB reads: \[ \chi^2(P\|Q) = \int \frac{(q - p)^2}{q} \ge \frac{\left(E_Q[f] - E_P[f]\right)^2}{\operatorname{Var}_Q(f)} \] By integrating with respect to an interpolation mixture $r(x;t) = p(x) + t(q(x) - p(x))$, $t \in [0,1]$, and evaluating a closed-form, one obtains: \[ \mathrm{KL}(P\|Q) \ge \frac{A-2V_P}{D} \tanh^{-1}\left(\frac{D}{A}\right) + \frac{1}{2} \ln \frac{V_P}{V_Q} \] with $A = (E_Q[f] - E_P[f])^2 + V_P + V_Q$ and $D = \sqrt{A^2 - 4V_PV_Q}$ [1907.00288]. When $P, Q$ are Bernoulli and one takes $f(x) = x$, the bound is tight and recovers the canonical KL divergence. Thus, the expression is an exact, data-driven certificate for Bernoulli distributions, as all terms can be estimated empirically. ## 5. Applications to Multi-Armed Bandits and Regret Optimality The practical utility of the Bernoulli-KL certificate is most pronounced in bandit algorithms—specifically, KL-UCB. For suboptimal arm $a$ with mean $\mu_a < \mu^*$, the expected number of draws up to time $T$ satisfies \[ \mathbb{E}[N_a(T)] \le \frac{\log T}{d(\mu_a, \mu^*)}(1+\epsilon) + C_1 \log\log T + O(T^{-\beta(\epsilon)}) \] implying, for total regret, \[ \mathbb{E}[R_T] \le \sum_{a: \Delta_a>0} \frac{\Delta_a}{d(\mu_a, \mu^*)} \log T + O(\log\log T) \] These bounds match the Lai–Robbins lower limits for the Bernoulli case, confirming asymptotic optimality [1102.2490]. Crucially, the KL-index-based choice is justified by the Bernoulli–KL certificate: $U_a(t)$ covers $\mu_a$ with high probability, ensuring both efficiency and statistical safety. ## 6. Extensions, Scaling Properties, and Computational Aspects KL-based bounds extend beyond bandits to general estimation and ranking problems for Bernoulli models. The KL-scaling lemma for $p,q,c \in [0,1]$ states \[ c(1-\max\{p, q\})\,d(p, q) \le d(cp, cq) \le c\,d(p, q) \] and, using Pinsker’s inequality $d(u, v) \ge 2(u-v)^2$, \[ d(cp, cq) \ge 2c \max\{c, 1-\max\{p, q\}\}(p-q)^2 \] These results are instrumental in regret analysis when means are small [1703.06513]. From a computational perspective, the required KL bounds can always be computed via cheap, one-dimensional convex search, and their complexity is negligible relative to typical algorithmic costs [1703.06513]. ## 7. Asymptotics, Information Geometry, and Limit Behavior In the infinitesimal regime, the Bernoulli-KL certificate aligns with classical information-theoretic limits. If $Q$ is an infinitesimal perturbation of $P$, Taylor expansions yield \[ \mathrm{KL}(P\|Q) \approx \frac{1}{2} I(\theta) \delta\theta^2 \] where the bound via the HCRB matches the Cramér-Rao lower bound. For Bernoulli, the equivalence holds exactly, tying the Bernoulli-KL certificate to foundational limits in asymptotic parametric inference [1907.00288]. --- **Summary Table: Key Bernoulli–KL Results** | Concept | Formula/Result | Source | |------------------------|---------------------------------------------------------------------------------------------------------------------|-------------------| | Bernoulli KL-divergence| $d(p, q) = p\ln\frac{p}{q} + (1-p)\ln\frac{1-p}{1-q}$ | [1102.2490] | | KL-UCB index | $U_a(t) = \max\{q \ge \hat\mu_a(t): N_a(t)\,d(\hat\mu_a(t), q) \le \log t + c\log\log t\}$ | [1102.2490] | | High-prob. certificate | $\Pr(U_a(t) < \mu_a) \le 1/t$ | [1102.2490] | | Confidence intervals | $\mathrm{UCB} = \sup\{q: n\,d(\hat p, q) \le \ln(1/\delta')\}$ | [1703.06513] | | KL lower bound (exact, Bern.) | $\mathrm{KL}(\mathrm{Bern}(p)\|\mathrm{Bern}(q)) = p\ln\frac{p}{q} + (1-p)\ln\frac{1-p}{1-q}$ | [1907.00288] | | Regret bound | $\mathbb{E}[R_T] \le \sum_{a: \Delta_a>0} \frac{\Delta_a}{d(\mu_a, \mu^*)}\log T + O(\log\log T)$ | [1102.2490] | These results collectively establish the Bernoulli-KL certificate as a keystone of non-asymptotic statistical inference and online learning with binary data, ensuring exactness, computational practicality, and rigorous high-probability control. Source: https://www.emergentmind.com/topics/bernoulli-kl-certificate