Random variate generation using only finitely many unbiased, independently and identically distributed random bits

Abstract: For any discrete probability distributions with bounded entropy, we can generate exactly a random variate using only a finite expected number of perfect coin flips. A perfect coin flip is the outcome of an unbiased Bernoulli random variable. Coin flips are unbiased, independently and identically distributed in all our work. We survey well-known algorithms for the discrete case such as the one from Knuth and Yao as well as the one from Han and Hoshi. We also discuss briefly about a practical implementation for the algorithm proposed by Knuth and Yao. For the continuous case, only approximations can be hoped for. The freedom to choose the accuracy for the approximations matters, and, for that, we propose to measure accuracy in terms of the Wasserstein $L_\infty$-metric. We derive a universal lower bound for the expected number of perfect coin flips required to reach a desired accuracy. We also provide several algorithms for absolutely continuous distributions that come within our universal lower bound.