Algorithms, Complexity, and Entropy of the Bernard-Letac Fair-Sampling Construction

Abstract: Bernard and Letac (1971) introduced a method for uniform random sampling among m outcomes from an unknown biased source of independent and identically distributed symbols. The process terminates when the multinomial coefficient of the cumulative symbol counts equals zero modulo m. This study extends the computational and information-theoretic analysis of their construction by presenting five algorithms with formal correctness guarantees and comprehensive complexity analyses. For prime m = p, the Bernard-Letac framework is analyzed in greater detail. The Rényi entropies of the source yield an exact product formula for the expected number of draws. A first-order approximation consistently overestimates this value, and the entropy lower bound is never attained. As p approaches 1, the expected cost converges to a constant greater than 1, determined by the entire source distribution. Furthermore, a seven-state automaton computes the mod-2 first-passage kernel of the binary walk, reducing the fair assignment cost from quadratic to nearly linear.

• The paper formalizes five Bernard–Letac sampling algorithms with correctness and complexity guarantees, including incremental p-adic valuation checks and near-linear exact binary assignment using a seven-state automaton.
• The paper shows that, for prime moduli, expected stopping time is governed by Rényi entropies of orders p, p², and higher, while its continuous small-modulus limit depends on the full source distribution rather than Shannon entropy alone.
• The paper proves the information-theoretic lower bound is never attained, finds direct sampling can use about 40% fewer biased draws than strong von Neumann baselines for composite moduli m≥6, and identifies open problems in adaptive and general-prime constructions.

Background and scope

Bernard and Letac's 1971 construction solves the fair-sampling problem: given i.i.d. draws from an unknown non-degenerate distribution $\pi$ on an alphabet $I$, produce an exactly uniform outcome among $m$ possibilities using a finite random number of draws. The sampler tracks the empirical count vector $S_t$ in the free abelian monoid $M$ and stops at the first time the multinomial coefficient $c(S_t)$ vanishes modulo $m$; the set $H_m = \{x : c(x) \equiv 0 \pmod m\}$ then supports a partition of the stopping trajectories into $m$ equiprobable classes for every admissible $\pi$. The original work was combinatorial and number-theoretic, with no computational or information-theoretic analysis. The paper under review, "Algorithms, Complexity, and Entropy of the Bernard–Letac Fair-Sampling Construction" (2608.20234), fills this gap in three ways: it extracts five algorithms with correctness guarantees and complexity bounds from the original theorems; it derives exact entropy-based formulas for the expected stopping time at prime moduli; and it establishes a transfer theorem yielding a seven-state automaton for the binary first-passage kernel.

Five algorithms with formal guarantees

The paper formalizes five procedures. The first computes the $I$0-adic valuation $I$1 via Kummer carries in $I$2 time, with an incremental variant costing $I$3 per step since only one coordinate changes. The second is the full fair $I$4-way sampler itself, proved to halt almost surely and to output exactly uniform outcomes independent of $I$5; its group-assignment step ranks the observed path among the $I$6 avoiding paths via a backward dynamic program over the box $I$7. A notable negative result appears here: partitioning all paths lexicographically (which would cost only $I$8) is not equiprobable — for $I$9 on the binary alphabet with $m$0, the block rule yields $m$1, essentially reproducing the source bias, because observable paths are not evenly distributed across lexicographic blocks. The remaining three algorithms are a Triangle-Theorem membership oracle for $m$2 (with $m$3 queries for shallow perturbations of precomputed deep generators), a periodic lookup table for $m$4 requiring period $m$5 rather than the naive $m$6 — the paper gives an explicit counterexample where the shorter table fails ($m$7 against a table entry of $m$8) — and an algorithm computing the $m$9-adic limit $S_t$0 modulo $S_t$1 in $S_t$2 iterations.

Two worked examples illustrate the machinery. For prime modulus $S_t$3 on a binary alphabet, stopping occurs at level $S_t$4, every path is observable, and groups are formed by pairing consecutive paths. For composite modulus $S_t$5 on a ternary alphabet, the structure degrades markedly: some $S_t$6-states such as $S_t$7 and $S_t$8 are unreachable ($S_t$9), group sizes vary, and no stopping occurs at level $M$0. These irregularities foreshadow why composite moduli resist closed-form analysis.

Entropy structure of the expected stopping time

For prime modulus $M$1, with $M$2, the exact identity $M$3 holds, and each $M$4 factorizes through the Rényi entropy of order $M$5: $M$6. The complexity of the sampler is thus governed by the Rényi entropies at orders $M$7, with the dominant contribution from order $M$8 — the paper states this is the first appearance of Rényi entropy as a natural parameter within fair sampling. The renewal approximation $M$9 is shown to strictly overestimate $c(S_t)$0 for every non-degenerate $c(S_t)$1: writing the ratio out, every correction factor $c(S_t)$2 lies strictly in $c(S_t)$3 because non-degeneracy forces $c(S_t)$4. Empirically the estimate is within 1% once $c(S_t)$5, though this accuracy claim is numerical and unproven. Monotonicity properties include strict inequality $c(S_t)$6, Schur-convexity in $c(S_t)$7 (so $c(S_t)$8 decreases toward uniform sources along majorization), divergence under near-degeneracy, and $c(S_t)$9 as $m$0. Notably, $m$1 cannot be expressed as a function of Shannon entropy alone: $m$2 has smaller entropy than $m$3 yet also smaller $m$4 at $m$5.

The sharpest structural finding concerns the small-$m$6 limit. Treating $m$7 continuously, $m$8 converges to a constant

$m$9

proved via a Riemann-sum argument using a uniform expansion of $H_m = \{x : c(x) \equiv 0 \pmod m\}$0 on all of $H_m = \{x : c(x) \equiv 0 \pmod m\}$1. This constant depends on the full law $H_m = \{x : c(x) \equiv 0 \pmod m\}$2, not on Shannon entropy alone, and is not equal to $H_m = \{x : c(x) \equiv 0 \pmod m\}$3: for the symmetric binary source, $H_m = \{x : c(x) \equiv 0 \pmod m\}$4 against $H_m = \{x : c(x) \equiv 0 \pmod m\}$5. The consequence is that the Bernard–Letac sampler does not approach Shannon optimality anywhere on the modulus scale.

Lower bound, strictness, and efficiency

A Wald-type argument yields the universal lower bound $H_m = \{x : c(x) \equiv 0 \pmod m\}$6 for any admissible procedure, matching the Knuth–Yao framework. More strikingly, the bound is never attained: if equality held, each output event would correspond to a single word $H_m = \{x : c(x) \equiv 0 \pmod m\}$7 whose cylinder must be contained in $H_m = \{x : c(x) \equiv 0 \pmod m\}$8 for all admissible $H_m = \{x : c(x) \equiv 0 \pmod m\}$9; maximizing the resulting monomial over the simplex forces $m$0 to equal the empirical type $m$1 of all words simultaneously, which contradicts the count constraint $m$2. Consequently the efficiency $m$3 satisfies $m$4 always, tends to zero both as $m$5 (since $m$6 while $m$7) and as $m$8 (since $m$9), and attains an interior maximum. Numerically, for $\pi$0 the maximum efficiency is $\pi$1 near $\pi$2; for $\pi$3, $\pi$4 near $\pi$5. Unimodality of the efficiency curve is observed in all computed cases but not proven. No closed form for $\pi$6 exists for composite $\pi$7: the generating-function proof relies on a single prime base, and the intersection $\pi$8 couples incompatible carry structures. The conjectured $\pi$9 scaling for fixed $I$00 remains unproven.

The transfer theorem and fast binary assignment

For $I$01 on a binary alphabet, the stopping test reduces by Kummer's theorem to the constant-time bitwise check $I$02, and the bottleneck becomes ranking the observed path among avoiding paths — $I$03 bit operations via dynamic programming modulo 2. The paper proves that the parity kernel $I$04 is 2-automatic: seven explicit section kernels $I$05 (identity, dead state, shifted-start kernels, and delta-type certificates) are closed under all sixteen digit-column sections, giving a deterministic finite automaton that reads base-2 columns least-significant-first and accepts exactly in states evaluating to 1 at the origin. Each identity in the proof is an exact path bijection except one step: a fixed-point-free involution swapping the two orders of a terminal mixed block, which introduces the reduction modulo 2. Combining the automaton with a residue rule (replacing the rank by its residue mod $I$06 preserves exact fairness) yields exact fair bit extraction in $I$07 bit operations instead of $I$08, with a truncation remark verifying the digit-width edge case is handled exactly. The paper conjectures a corresponding finite transfer system for general prime $I$09 and alphabets, based on cyclic rotations of terminal full blocks contributing zero modulo $I$10, but leaves the state-count question open.

Numerical benchmarking for composite moduli

Since no closed form exists for composite $I$11, the paper compares the direct Bernard–Letac sampler against two von Neumann baselines (with rejection sampling, and with Lumbroso's optimal Fast Dice Roller) using two million samples per configuration:

$I$12	$I$13	VN+Reject	VN+Lumbroso	Bernard–Letac	BL/Lum
4	0.50	8.00	8.00	5.86	0.73
4	0.70	9.52	9.52	6.30	0.66
6	0.50	16.00	14.67	6.65	0.45
6	0.70	19.05	17.46	7.18	0.41
10	0.50	25.60	18.40	7.37	0.40
10	0.70	30.48	21.90	8.97	0.41

The direct sampler consumes roughly 40% fewer biased draws than the strongest debiasing pipeline for $I$14, an advantage attributed structurally to skipping intermediate debiasing. All methods remain far above the lower bound $I$15, so none is absolutely optimal. The paper also corrects an error in the original literature: Remarque 3 of Bernard–Letac prints $I$16 for the fair coin, whereas the exact value is $I$17; their companion figure $I$18 is confirmed correct.

Limitations and open questions

Several limitations are stated plainly. Composite moduli lack any closed form for $I$19, and the derivation provably does not extend due to carry coupling between bases. The unimodality of efficiency and the $I$20-accuracy claim for the first-order approximation are numerical observations without proofs. Whether an adaptive stopping boundary can reduce $I$21 while preserving exact fairness is unresolved. The information-theoretic lower bound becomes vacuous for sources of infinite Shannon entropy, and no replacement is known. Open problems enumerated include transfer systems for general primes and alphabets (P1), closed-form composite-$I$22 possibly via Möbius inversion over prime-power entropies (P2), optimal modulus decomposition including rounding up to the nearest prime (P3), adaptive boundaries (P4), an entropy interpretation of the $I$23-adic limit $I$24 whose product formula resembles a convolution connected to $I$25-adic integration (P5), infinite-entropy lower bounds (P6), and improved zero-one matrix counts via Kummer carries (P7).

Conclusion

This paper converts the Bernard–Letac construction from a purely combinatorial object into a computationally specified and information-theoretically characterized sampler. Its central contributions are the strict-overestimation theorem for the Rényi-entropy approximation, the identification of the non-Shannon limit constant $I$26 showing optimality is never achieved, the never-attained lower bound, and the seven-state automaton reducing exact binary assignment to nearly linear time. The gap between these results and the open composite-modulus and general-prime questions delineates precisely what remains unknown about this classical construction.