Monotone Compression

• Monotone compression refers to preserving or exploiting ordered structures during encoding and optimization to reduce description complexity and achieve efficient decoding.
• Applications of monotone compression include universal source coding, streaming compression, combinatorial optimization, and computational learning theory, each leveraging different aspects of monotonicity to improve efficiency and performance.
• This term encompasses diverse methods, such as ordered probability vectors in universal source coding for lower parameter-description costs, and prefix-preserving codes in streaming for coherent data compression, showing its versatile applicability across domains

Monotone compression denotes a family of compression principles in which an order, prefix relation, semantic utility, geometric filtration, or other structured relation is preserved or exploited during encoding and optimization. The term is not associated with a single formalism. In universal source coding, it concerns distributions whose probabilities are known to be ordered; in streaming and algorithmic information theory, it concerns prefix-preserving descriptions; in entropy-compression proofs, it describes backward encodings of monotone optimization trajectories; in scheduling and prompt selection, it refers to monotone structural objectives that permit representative-set reduction; and in computational learning theory, it denotes order-preserving prefix-free compression. Across these settings, monotonicity can reduce description complexity, enable efficient decoding, control exceptional events, or produce nested optimization guarantees. It does not, in general, imply that every individual code length, geometric quantity, algorithmic trajectory, or regularized objective is itself monotone.

1. Terminological scope and formal distinctions

The principal ambiguity in monotone compression arises from the different objects to which monotonicity may apply. A distribution may be monotonic because its probability parameters satisfy an ordering constraint; a code may be monotonic because ordered inputs receive ordered codewords; a stream code may be prefix-preserving because descriptions of longer prefixes extend descriptions of shorter prefixes; and a set function may be monotone because adding an element never decreases utility.

These notions should be distinguished from monotonicity of the final compression cost. For example, a monotone submodular semantic utility can be combined with a modular token penalty to produce a regularized objective that is non-monotone. Similarly, an order-preserving compressor provides simultaneous bounds indexed by empirical-entropy order, but its actual output length is not asserted to decrease as more context is supplied.

Setting	Monotone object	Principal consequence
Universal source coding	Ordered probability vector	Lower parameter-description cost
Streaming compression	Simultaneous empirical-entropy guarantees	One-pass compression with bounded-memory redundancy
Algorithmic information	Nested code prefixes	Coherent compression of infinite streams
Entropy compression	Fitness-constrained update trajectories	Control of exceptional optimization steps
Scheduling and prompt selection	Work or semantic utility	Representative-set reduction or diminishing returns
Computational learning theory	Input-to-codeword order	Equivalence with next-token prediction

A useful common abstraction is that monotonicity restricts the admissible descriptions. The restriction may reduce the number of distinguishable parameter vectors, make cumulative probabilities efficiently computable, constrain feasible transitions, or guarantee that a larger feasible set retains all previously available candidates. The resulting gain depends on the formal object and cannot be transferred between settings without qualification.

2. Ordered distributions and universal source coding

For an i.i.d. sequence $X^n=(X_1,\ldots,X_n)$ over $\mathcal X=\{1,\ldots,k\}$, the unknown source parameter is $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$, with sequence probability

$P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$

The monotone class is

$$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$

The ordering is known in advance, although the probability values remain unknown. Consequently, the code describes an ordered probability vector rather than an arbitrary permutation of estimated probabilities. Examples include distributions over integer-valued observations, geometric distributions, frequency-ranked names, residual values, and species or symbols whose relative frequencies are known to be ordered (0704.0838).

For a code with length function $L$, the average per-symbol redundancy is

$$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$

and the corresponding total redundancy is $nR_n(L,\theta)$ bits. Universal probability assignments satisfy $L(x^n)=-\log Q(x^n)$ up to integer-length effects. The individual-sequence analogue compares the code length with the monotone maximum-likelihood description length,

$$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$

where

$\mathcal X=\{1,\ldots,k\}$0

For sufficiently small finite alphabets, each free probability parameter costs essentially

$\mathcal X=\{1,\ldots,k\}$1

bits, provided $\mathcal X=\{1,\ldots,k\}$2. In total form, the lower bound is of order

$\mathcal X=\{1,\ldots,k\}$3

up to lower-order terms. The ordered-probability side information therefore changes the usual parameter-description cost. When $\mathcal X=\{1,\ldots,k\}$4 is larger, the parametric characterization ceases to govern the redundancy; for $\mathcal X=\{1,\ldots,k\}$5, the total redundancy is instead of order $\mathcal X=\{1,\ldots,k\}$6 in the lower-bound regime and $\mathcal X=\{1,\ldots,k\}$7 bits is achievable.

The corresponding construction is a two-part code. It quantizes a monotone probability vector, describes the quantized vector, and encodes the data using the quantized distribution. Quantization is nonuniform: small probabilities are quantized more finely because relative errors in small probabilities can otherwise impose a large likelihood penalty. Exponentially expanding intervals and Elias coding describe grid indices and index differences. The ordered structure eliminates the need to transmit permutation information.

For infinite alphabets, the method introduces an effective alphabet $\mathcal X=\{1,\ldots,k\}$8. The first $\mathcal X=\{1,\ldots,k\}$9 symbols are encoded using a finite monotone model, while symbols beyond $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$0 are grouped into a residual class and explicitly identified when they occur. The tail cost is controlled by quantities such as

$\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$1

Fast-decaying sources can therefore behave, from the coding perspective, like finite alphabets. Geometric distributions achieve total redundancy $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$2 bits, while broader fast-decaying classes achieve $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$3-type bounds. For slower tails, distribution-dependent rates arise from balancing quantization, model-description, and rare-symbol costs. Finite entropy guarantees diminishing per-symbol redundancy:

$\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$4

This is a statement about total redundancy being $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$5, not necessarily about bounded or subpolynomial total redundancy.

3. Prefix-preserving compression of streams and empirical entropy

A separate use of monotone compression concerns coherent descriptions of prefixes. For an infinite binary stream $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$6, independently choosing shortest descriptions of each prefix does not yield a single progressively decodable code: the descriptions need not be nested, and their lengths may fluctuate. A prefix-preserving code instead produces an infinite code stream $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$7 such that code prefixes used to recover shorter source prefixes are contained in those used for longer prefixes.

For a partial computable information content measure $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$8, defined along every prefix of $\boldsymbol\theta=(\theta_1,\ldots,\theta_k)$9, there exists a Martin-Löf random stream $P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$0 from which $P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$1 is recoverable with oracle use

$P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$2

up to an additive constant. For arbitrary streams, the bound becomes

$P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$3

where $P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$4 is prefix-free Kolmogorov complexity. The future minimum is required because information values need not be monotone as functions of prefix length. It creates a monotone envelope for the decoding process (Barmpalias et al., 2017).

The construction uses a layered Kraft–Chaitin method. Requests form a tree: a code for a longer source description must extend the code assigned to its predecessor. Local minima of the information measure determine nested code lengths, and the construction avoids effectively enumerable sets of compressible strings so that every infinite path through the code tree is algorithmically random. The result differs from standard monotone Kolmogorov complexity: its monotonicity is in the coding relation between source prefixes and code prefixes, while the principal complexity measure remains prefix-free complexity or a general information content measure.

A related finite-string setting concerns one-pass compression under bounded memory. For a string $P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$5 of length $P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$6 over an alphabet of constant size $P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$7, the empirical entropy

$P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$8

measures the average uncertainty of a character given its preceding $P_\theta(x^n)=\prod_{i=1}^k\theta_i^{n_x(i)}.$9 characters. With $$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$0 bits of memory and one pass, one encoding simultaneously satisfies

$$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$1

for every $$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$2. A matching lower-bound construction uses de Bruijn sequences, for which $$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$3 but a one-pass compressor with limited memory must still emit substantial information (0708.1877).

The upper bound partitions the input into blocks, applies a non-streaming empirical-entropy compressor to each block, and uses the superadditivity relation

$$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$4

The redundancy is therefore proportional to the number of blocks and scales approximately as

$$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$5

This is simultaneous compression across all context orders, not a claim that the numerical output length is monotone in $$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$6. The redundancy term itself grows with $$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$7, even though the ideal empirical-entropy targets satisfy $$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$8.

4. Entropy compression in monotone optimization

In entropy-compression analyses, monotonicity constrains an optimization trajectory rather than a source distribution or codeword order. For the $$\mathcal M_k=\left\{\boldsymbol\theta:\theta_1\geq\theta_2\geq\cdots\geq\theta_k\geq0,\ \sum_i\theta_i=1\right\}.$$9-Evolutionary Algorithm on strictly monotone pseudo-Boolean functions, each bit is independently flipped with probability $L$0, and accepted offspring satisfy the fitness comparison. The unique optimum is $L$1.

Ordinary drift arguments give positive expected progress in the number of one-bits only for $L$2. For $L$3, accepted moves can lose more one-bits than they gain because a monotone function may assign exceptionally high marginal value to particular coordinates. The analysis separates updates into good and bad updates. A bad update flips exactly one zero-bit to one and introduces a bit whose marginal value exceeds that of at least $L$4 existing one-bits.

The proof compresses the sequence of accepted updates. Starting from the terminal state $L$5, it encodes the trajectory backward using the numbers of upflips and downflips and the coordinates involved. For bad updates, the exceptional upflipped bit belongs to a small reconstructible subset, saving approximately $L$6 bits. If bad updates occurred too frequently, the backward description would be shorter than the entropy of the mutation process.

The entropy comparison yields

$L$7

where $L$8 is the number of accepted updates and $L$9 is the number of initially set bits. The factorial ratio arising in the encoding telescopes because the number of zero-bits evolves according to

$$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$0

Combining the bound on bad updates with positive drift on good updates gives an expected $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$1 number of updates for every $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$2, for some $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$3. Accounting for rejected mutations through phase decomposition yields an expected runtime of

$$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$4

Thus the phase transition does not occur exactly at $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$5, although its precise location remains open. Earlier constructions give exponential lower bounds for sufficiently large $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$6, summarized as $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$7 in the cited discussion (Lengler et al., 2018).

The compression argument does not assert that every transition improves a scalar potential. Instead, it shows that structurally exceptional transitions are collectively compressible. Monotonicity is essential because it guarantees that accepted nontrivial moves contain an upflip, controls downflip probabilities, identifies exceptional marginal values, and permits backward reconstruction from a known optimum.

5. Monotone compression in combinatorial optimization and geometry

In moldable-job scheduling, each job $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$8 can use an allotment $$R_n(L,\theta)=\frac{1}{n}\mathbb E_\theta[L(X^n)]-H_\theta(X),$$9 of machines, with processing time $nR_n(L,\theta)$0 and work

$nR_n(L,\theta)$1

The work function is monotone when

$nR_n(L,\theta)$2

This permits geometric compression of the $nR_n(L,\theta)$3 possible allotments. For $nR_n(L,\theta)$4 and $nR_n(L,\theta)$5,

$nR_n(L,\theta)$6

Taking $nR_n(L,\theta)$7, the allotments can be restricted to

$nR_n(L,\theta)$8

whose cardinality is

$nR_n(L,\theta)$9

Rounding each allotment downward reduces work by monotonicity and increases processing time by at most a multiplicative $L(x^n)=-\log Q(x^n)$0 factor. Thus the algorithm replaces a linear number of possible machine counts with a logarithmic-size representative set. This reduction supports an improved FPTAS for $L(x^n)=-\log Q(x^n)$1 and a $L(x^n)=-\log Q(x^n)$2-approximation for $L(x^n)=-\log Q(x^n)$3 through a compressed knapsack instance and max-plus convolution (Grage et al., 2023).

The same structural distinction appears in lattice-knot profiles, but with a different conclusion. For a lattice polygon $L(x^n)=-\log Q(x^n)$4, raw density and compression radius are defined by

$L(x^n)=-\log Q(x^n)$5

and

$L(x^n)=-\log Q(x^n)$6

Under the raw simple-cubic convention $L(x^n)=-\log Q(x^n)$7, the compression radius becomes $L(x^n)=-\log Q(x^n)$8. Length-filtered minima are monotone because feasible representative sets are nested:

$L(x^n)=-\log Q(x^n)$9

and likewise for compression-radius profiles.

This set-theoretic monotonicity does not extend to individual polygons or BFACF paths. Exact-level values can worsen as length increases, and individual moves can increase or decrease density and compression radius. In the figure-eight example, a 32-edge connecting state has larger raw density and slightly larger $$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$0 than the 30-edge endpoint seeds, while an intermediate 30-edge state has smaller compression radius but larger density. The raw ropelength proxy can also increase and then decrease along the path (Ozawa, 27 May 2026).

These examples establish a general limitation: monotonicity of minima over nested feasible sets is not monotonicity of every trajectory, state, or objective value.

6. Monotone threshold codes, submodular selection, and language modeling

For one-bit unbiased scalar compression, monotonicity emerges as a consequence of exact second-moment optimality. Given a source distribution $$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$1 with CDF $$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$2, mean $$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$3, and

$$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$4

the optimal source-averaged reconstruction second moment is

$$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$5

For regular full-support sources, every attaining code is, up to null sets, bit relabeling, and public-seed refinement, a random-threshold code. The threshold density is

$$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$6

the encoder sends $$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$7, and the decoder uses threshold-dependent reconstruction levels. Thus exact optimality forces a monotone encoder, although arbitrary unbiased codes need not be monotone (Zhang et al., 26 Sep 2026).

This result also exposes a stability trade-off. For Gaussian location families, exact minimaxity has an infinite third absolute moment at the endpoint means, while a Cauchy-mixture threshold density preserves pointwise unbiasedness and monotonicity, gives finite moments of every positive order, and increases the second-moment bound by at most $$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$8. Finite public randomness and bounded outputs cannot preserve exact full-line unbiasedness; finite-grid threshold codes therefore introduce explicit bias.

In selective prompt compression, monotonicity applies to semantic utility rather than to codewords. PC-SubMax defines a normalized nonnegative monotone submodular utility

$$-\log P_{\mathrm{ML}(x^n)} = -\log P_{\widehat{\theta}_M}(x^n),$$9

under a token-budget constraint $\mathcal X=\{1,\ldots,k\}$00. Coverage, log-determinant diversity, and nonnegative relevance produce diminishing returns. The regularized objective

$\mathcal X=\{1,\ldots,k\}$01

remains submodular but need not be monotone because token costs can exceed semantic marginal gains. Regularized Greedy+Max returns a feasible set $\mathcal X=\{1,\ldots,k\}$02 satisfying

$\mathcal X=\{1,\ldots,k\}$03

with $\mathcal X=\{1,\ldots,k\}$04 value-oracle queries, where $\mathcal X=\{1,\ldots,k\}$05 is the maximum feasible cardinality (Zhang et al., 26 Sep 2026).

Finally, in computational language modeling, monotonicity is formalized as order preservation. A prefix-free encoding $\mathcal X=\{1,\ldots,k\}$06 is monotone if

$\mathcal X=\{1,\ldots,k\}$07

An efficiently computable next-token predictor can be converted by arithmetic coding into an efficiently computable monotone compressor with at most an additive two-bit loss. Conversely, an efficient prefix-free monotone compressor induces an efficient autoregressive predictor with the same expected cross-entropy redundancy. The monotonicity condition enables binary search over codewords and efficient computation of cumulative probabilities (Mazor et al., 8 Oct 2026).

The equivalence fails for arbitrary compression under cryptographic assumptions. If infinitely-often one-way functions exist, there are efficiently samplable distributions with optimal non-monotone compression but for which every efficient autoregressive model incurs linear KL loss. The same framework identifies monotone incompressibility with next-bit pseudoentropy, up to the stated additive gap. In this setting, order preservation is not merely an implementation preference: it is the structural condition connecting short descriptions with efficiently computable sequential prediction.

Across these applications, monotone compression therefore has a precise but plural significance. It may mean exploiting ordered source parameters, preserving prefix or lexicographic order, encoding exceptional trajectories more economically, reducing a structured choice space to geometric representatives, optimizing a monotone semantic surrogate, or enforcing the order structure required for compression–prediction equivalence. The common principle is preservation or exploitation of structure; the particular guarantee depends on whether monotonicity concerns distributions, descriptions, trajectories, feasible sets, utilities, or input–code order.