Next-Bit Pseudoentropy: Definition & Use
- Next-bit pseudoentropy is a computational measure where a bit sequence's next bit, given the prefix, is computationally indistinguishable from a distribution with higher Shannon entropy.
- It is used to analyze the unpredictability of sequences more precisely than Shannon entropy, as it considers computational limits rather than statistical closeness.
- Applications include pseudorandom generators, language modeling, and assessing the security of cryptographic functions like one-way functions.
Next-bit pseudoentropy is a computational analogue of conditional entropy for a sequence of bits. Informally, a distribution has next-bit pseudoentropy when each next bit, conditioned on the preceding prefix, can be computationally replaced by a bit distribution with greater Shannon entropy without being efficiently distinguishable from the original distribution. The notion separates information-theoretic predictability from computational unpredictability: the actual sequence may have limited entropy, while an efficient observer cannot distinguish it from one with a larger aggregate conditional entropy. In its standard formulation, next-bit pseudoentropy is the one-bit-block specialization of next-block pseudoentropy and is closely related to pseudorandom generators, one-way functions, conditional pseudoentropy, KL divergence, and monotone incompressibility.
1. Definition and formal framework
Let be a family of binary random variables, and let be uniformly distributed over . The prefix and selected next bit are
The family has non-uniform next-bit pseudoentropy at least if the conditional pseudoentropy of the selected bit given its prefix is at least . Equivalently, the total pseudoentropy summed over the conditional next-bit distributions is at least (Mazor et al., 8 Oct 2026).
More explicitly, let and let the selected bit be 0. There must exist a binary-valued family 1, jointly distributed with 2, such that, for every polynomial 3,
4
for all sufficiently large 5(X,B)6(X,C)7CI_n$8, there is a negligible function $I_n$9 such that
$[m(n)]$0
where $[m(n)]$1 is sampled from $[m(n)]$2 and $[m(n)]$3 from $[m(n)]$4.
The definition is commonly expressed through next-block pseudoentropy. If $[m(n)]$5 is a block decomposition, next-block pseudoentropy at least $[m(n)]$6 means that there exists $[m(n)]$7 such that, for every $[m(n)]$8,
$[m(n)]$9
and
$(B_n)_{ Next-bit pseudoentropy is the special case in which every block $(B_n)_{ Next-bit pseudoentropy differs from ordinary Shannon entropy and conditional min-entropy. Shannon entropy measures the actual average uncertainty of the distribution. For a binary process, its entropy rate can be below one even when individual marginals or fixed-length block frequencies appear uniform. Conditional min-entropy instead measures worst-case prediction probability. A standard information-theoretic next-bit condition would require the conditional distribution of $(B_n)_{ The distinction is illustrated by two-faced processes. For an order-$(B_n)_{ $(B_n)_{ which is less than one whenever $B=\{B_n\}$0. Once the relevant context is known, an optimal predictor succeeds with probability $B=\{B_n\}$1 and the corresponding conditional min-entropy is $B=\{B_n\}$2 Consequently, uniform block frequencies do not imply next-bit unpredictability. A predictor that knows the last $B=\{B_n\}$3 bits and the transition rule can distinguish the next bit from an independent fair bit with advantage proportional to $B=\{B_n\}$4 (Ryabko, 2015). This example establishes a central conceptual separation: The twice two-faced construction further shows that asymptotically fair frequencies for every fixed finite word length can coexist with low Shannon entropy. However, such pointwise asymptotic frequency statements do not establish computational next-bit indistinguishability, quantitative finite-length security, or unpredictability given the complete prefix (Ryabko, 2015). KL divergence provides a quantitative bridge between efficient prediction and pseudoentropy. For distributions $B=\{B_n\}$5 and $B=\{B_n\}$6, $B=\{B_n\}$7 For a sequence distribution $B=\{B_n\}$8 and an autoregressive model $B=\{B_n\}$9, let $h(\cdot)$0 denote the complete-output distribution induced by the model. The model’s expected negative log-likelihood is $h(\cdot)$1 Thus the excess prediction loss above the optimal entropy is exactly the KL divergence from the true distribution to the model distribution. An efficiently computable model is $h(\cdot)$2-optimal when $h(\cdot)$3 Vadhan and Zheng’s characterization, as used in the later work, states that $h(\cdot)$4 has next-bit pseudoentropy at least $h(\cdot)$5 if and only if every efficiently computable autoregressive model fails to be $h(\cdot)$6-optimal for every polynomial $h(\cdot)$7 and all sufficiently large $h(\cdot)$8 (Mazor et al., 8 Oct 2026). This characterization gives an operational interpretation: Next-bit pseudoentropy is the computational hardness of constructing an accurate autoregressive approximation to the sequence distribution. The relationship is also expressed through relative pseudoentropy. If $h(\cdot)$9 is jointly distributed, $h(n)/m(n)$0 has relative pseudoentropy at least $h(n)/m(n)$1 given $h(n)/m(n)$2 when every PPT sampler $h(n)/m(n)$3 satisfies $h(n)/m(n)$4 For a block decomposition $h(n)/m(n)$5, next-block relative pseudoentropy requires that every online simulator incur a total conditional KL divergence of at least $h(n)/m(n)$6: $h(n)/m(n)$7 The KL chain rule distributes the global hardness across positions. For blocks of length $h(n)/m(n)$8, next-block relative pseudoentropy can be converted into a computationally indistinguishable distribution with a larger conditional Shannon entropy. Taking one-bit blocks yields next-bit pseudoentropy (Agrawal et al., 2019). One-way functions provide a canonical source of next-bit pseudoentropy. Let $h(n)/m(n)$9 be a one-way function and $m(n)$0. Inversion hardness implies that, for every efficient simulator $m(n)$1, $m(n)$2 where $m(n)$3 is the inversion success bound. Hence $m(n)$4 has relative pseudoentropy $m(n)$5 given $m(n)$6. For strongly one-way functions, this quantity is $m(n)$7. Partitioning $m(n)$8 into blocks of length $m(n)$9 and applying the short-block conversion gives next-block pseudoentropy exceeding the real entropy by the relative-entropy hardness parameter. With one-bit blocks, the resulting variable $h(n)$0 has next-bit pseudoentropy at least $h(n)$1 The KL framework therefore decomposes the standard implication from one-wayness to next-bit pseudoentropy into three stages: The framework also unifies pseudoentropy with inaccessible entropy. In pseudoentropy, an efficient simulator cannot sample the hidden value with small KL divergence. In inaccessible entropy, an online generator cannot reproduce the target distribution while retaining equally accessible entropy. For flat distributions, KL divergence equals the entropy difference: $h(n)$2 These notions share a KL-based foundation but are not identical: next-bit pseudoentropy concerns the existence of a computationally indistinguishable higher-entropy distribution, whereas inaccessible entropy concerns the inability of an online generator to access or reproduce entropy (Agrawal et al., 2019). Generalized pseudoentropy extends this framework beyond Shannon and min-entropy. For a conditional distribution $h(n)$3 represented by $h(n)$4 and a convex function $h(n)$5 on a probability simplex, define $h(n)$6 Negative Shannon entropy and the maximum-coordinate function recover conditional Shannon entropy and average conditional min-entropy, respectively. The associated generalized Bregman divergence measures the hardness of efficiently approximating the conditional distribution. A universal simulator can simultaneously satisfy pseudoentropy guarantees for an entire family of convex entropy functions, using computational indistinguishability together with weight-restricted calibration (Hu et al., 8 Jul 2025). That framework does not explicitly define next-bit pseudoentropy or analyze a sequential predictor receiving $h(n)$7. It concerns conditional pseudoentropy of a label $h(n)$8 given side information $h(n)$9, where $X_n=(B_n)_{ Next-bit pseudoentropy has an exact computational relationship with monotone compression. A prefix-free encoding is monotone when it preserves lexicographic order: for inputs $X_n=(B_n)_{ $X_n=(B_n)_{ A distribution is monotonically incompressible to length $X_n=(B_n)_{ An efficient autoregressive model can be converted into a monotone encoding using arithmetic coding. If $X_n=(B_n)_{ $X_n=(B_n)_{ Averaging over $X_n=(B_n)_{ $I_n$00 Thus a model with excess loss at most $I_n$01 yields a monotone encoding of expected length at most $I_n$02 The reverse transformation also holds. An efficient prefix-free monotone encoding can be converted into an autoregressive model whose complete-output distribution has KL divergence no greater than the encoding’s excess expected length. Monotonicity permits efficient recovery of cumulative probabilities by binary search, after which prefix probabilities and conditional next-token probabilities can be computed. The resulting equivalence is: $I_n$03 up to a constant additive gap of two bits in the direction from prediction to compression (Mazor et al., 8 Oct 2026). More precisely: The equivalence is unconditional. Cryptographic one-way functions are needed only for a separate separation result involving unrestricted, order-scrambling compression. Under one-way functions, there are distributions that admit efficient non-monotone compression at optimal expected length while every efficient autoregressive model incurs large KL divergence. Thus arbitrary compression is strictly more permissive than monotone compression for this purpose. The distinction explains why ordinary incompressibility implies next-bit pseudoentropy but the converse can fail. Non-monotone encodings may assign short codewords in an order that cannot be converted into a coherent conditional distribution. Monotonicity preserves the ordering required to reconstruct such a distribution. Next-bit pseudoentropy is used in the analysis of pseudorandom generators, one-way functions, conditional pseudoentropy, compression, language modeling, and computational entropy. Its relevance to pseudorandomness is structural: if a sequence has high next-bit pseudoentropy, then no efficient autoregressive process can closely reproduce its distribution while eliminating the associated computational entropy gap. The notion should not be confused with several weaker or different properties: The 2025 generalized characterization results also do not constitute a direct next-bit theorem. They characterize conditional pseudo-min-entropy, conditional pseudo-Shannon entropy, and generalized conditional $I_n$12-pseudoentropy through computational approximation hardness, Bregman divergence, multiaccuracy, and weight-restricted calibration. Their predictors receive side information $I_n$13 and predict an entire label $I_n$14, rather than receiving a prefix and predicting one subsequent bit (Hu et al., 8 Jul 2025). The connection with language modeling follows from the autoregressive representation. A LLM is modeled as a next-token predictor, while an autoregressive model induces a complete-output distribution. Arithmetic coding transforms the model’s conditional probabilities into an order-preserving code, and a monotone code can be transformed back into a coherent autoregressive distribution. This establishes the prediction–compression correspondence underlying the equivalence between next-bit pseudoentropy and monotone incompressibility (Mazor et al., 8 Oct 2026). The development of next-bit pseudoentropy reflects a progression from information-theoretic randomness to computational entropy. Two-faced processes demonstrate that low entropy can coexist with fair-looking finite-block statistics, while also showing why frequency tests do not establish conditional unpredictability (Ryabko, 2015). KL-based formulations then connect hardness of sampling and inversion to computational entropy gaps, providing a modular route from one-way functions to next-block and next-bit pseudoentropy (Agrawal et al., 2019). Generalized entropy characterizations broaden this perspective by treating Shannon entropy, min-entropy, collision measures, and related quantities within a common convex-analytic framework. Their universal-simulator results show that a single computationally indistinguishable distribution can simultaneously witness pseudoentropy properties for multiple entropy notions, although the framework is not itself sequential (Hu et al., 8 Jul 2025). The compression characterization supplies a complementary interpretation. Next-bit pseudoentropy is not merely a statement that a distribution resists short descriptions. It corresponds specifically to resistance to efficient monotone compression, because monotonicity is what permits a code to define consistent conditional probabilities. The resulting equivalence identifies the precise compression notion matching next-bit prediction: 15 with the two-bit arithmetic-coding gap and the stated asymptotic qualifications (Mazor et al., 8 Oct 2026).2. Entropy, prediction, and indistinguishability
3. KL divergence and computational entropy
4. Relation to one-way functions and generalized pseudoentropy
5. Compression and monotone incompressibility
6. Applications, limitations, and related notions
7. Historical and conceptual significance