Guaranteed avoidance of the initial bits problem in BB-ANS

Develop a provably guaranteed method to avoid the initial bits problem (state depletion) when using bits-back with Asymmetric Numeral Systems (BB-ANS) to encode latent-variable models, ensuring the ANS state remains sufficiently large throughout encoding so that sampling from the approximate posterior is always valid without requiring ad hoc workarounds.

Background

In bits-back coding with ANS, decoding the latent variable first consumes entropy from the shared ANS state. If the state is too small at any step, sampling the latent correctly can fail, a phenomenon referred to as the “initial bits problem” or ANS state depletion. The thesis presents practical mitigations such as decomposing latents and observations into sub-variables but notes none provide a formal guarantee against depletion.

The authors explicitly state that, at the time of writing, no method is known that guarantees avoidance of this issue. A general solution would have significant impact on BB-ANS and related bits-back schemes, providing stronger correctness guarantees and potentially simplifying implementations across applications.