Finite-Precision Kolmogorov Complexity

• Finite-precision Kolmogorov complexity is a framework that quantifies description lengths when objects, representations, or mechanisms are discretized.
• It unifies computable approximations, finite-state models, and fixed-precision neural formulations, yielding tight bounds that align with classical Kolmogorov measures.
• The approach underpins practical insights such as MDL generalization bounds and Bayesian interpretations, linking fixed-precision network norms to universal priors.

Searching arXiv for recent and foundational papers relevant to finite-precision Kolmogorov complexity. Finite-precision Kolmogorov complexity concerns description length when either the object being described, the representation used for that object, or the descriptive mechanism itself is discretized, truncated, or otherwise restricted. In the works surveyed here, the theme appears in several technically distinct forms: fixed-precision neural parameterizations, computable upper bounds on noncomputable complexity, finite-prefix comparisons between numeral systems, oracle-relative bounds on finite strings, and restricted decompressor models. A central recent result shows that, for looped neural networks with fixed-precision weights, the minimum norm required to emit a binary string coincides with its Kolmogorov complexity up to a logarithmic factor (Musat, 11 May 2026).

1. Formal scope and basic distinctions

The starting point is the standard definition of prefix-free Kolmogorov complexity for a universal prefix-free machine $U$,

$K(x)=\min\{\,|p|:U(p)=x\,\}.$

Because $K$ is only upper semicomputable, several papers treat finite-precision or computable surrogates rather than exact values. One canonical characterization states that a computable function $f$ is an upper bound on $K$ up to an additive constant exactly when its Kraft sum is finite: $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$ This places computable approximations to $K$ inside the usual Kraft–Chaitin framework and makes explicit that finite-precision estimates of complexity are necessarily upper bounds or restricted-model analogues rather than exact evaluations of the noncomputable invariant (0902.1041).

A different, more literal notion of finite precision arises when descriptions are stored in a discrete parameter alphabet. In the neural setting, the relevant regime is

$${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$

so each nonzero parameter is drawn from a constant-size set of admissible values. In that regime each nonzero weight carries only $O(1)$ bits, and norm, sparsity, and description length become comparable. The same paper stresses that this comparability fails outside fixed precision: real-valued weights can be super-Turing, and bounded rational weights can encode arbitrarily many bits in their numerators and denominators, so norm no longer reflects stored information (Musat, 11 May 2026).

These two viewpoints already delineate the field. One concerns computable approximations to a noncomputable quantity; the other concerns exact description length after an explicit discretization of the representational medium.

2. Fixed-precision neural complexity as a description-length measure

The most explicit finite-precision theorem in the surveyed literature defines the neural complexity of a computable binary string $s$ as the minimum number of nonzero parameters in any fixed-precision looped network that outputs $K(x)=\min\{\,|p|:U(p)=x\,\}.$0: $K(x)=\min\{\,|p|:U(p)=x\,\}.$1 where $K(x)=\min\{\,|p|:U(p)=x\,\}.$2 denotes the nonzero-parameter count. The main theorem gives a two-sided sandwich bound: $K(x)=\min\{\,|p|:U(p)=x\,\}.$3 This makes the minimum fixed-precision network size a bona fide description-length measure, equal to Kolmogorov complexity up to a logarithmic factor (Musat, 11 May 2026).

The upper bound is obtained by encoding a shortest universal Turing machine program $K(x)=\min\{\,|p|:U(p)=x\,\}.$4 for $K(x)=\min\{\,|p|:U(p)=x\,\}.$5 directly into the network weights. The construction uses a universal looped network $K(x)=\min\{\,|p|:U(p)=x\,\}.$6, an iteration-1 gate that fires only at the first time step, and a routing layer with one ternary parameter per bit of $K(x)=\min\{\,|p|:U(p)=x\,\}.$7. In the paper’s notation,

$K(x)=\min\{\,|p|:U(p)=x\,\}.$8

so program bits are written into weights at unit cost per bit.

The lower bound treats a fixed-precision network as a finite discrete object. If the network has $K(x)=\min\{\,|p|:U(p)=x\,\}.$9 nonzero parameters, each can be encoded as a tuple

$K$0

consisting of layer index, source neuron, target neuron, and weight value. After pruning unused neurons and collapsing empty layers, one may assume at most $K$1 neurons and $K$2 layers. Each nonzero parameter then needs at most

$K$3

bits, where the value alphabet $K$4 has constant size. Hence the entire network has a description of length

$K$5

and a constant-size simulator reconstructs the network and runs it to produce $K$6.

The logarithmic term comes from addressing overhead: sparse parameters must specify not only their values but also their locations. The same paper shows that this is not proof slack. For a permutation $K$7, if $K$8 is the row-major serialization of the permutation matrix $K$9, then for typical $f$0,

$f$1

while a looped network with only $f$2 nonzero ternary parameters outputs $f$3. Thus

$f$4

which realizes the lower-bound factor.

3. Norm collapse, Solomonoff priors, and regularization

Within fixed precision, the same neural result is norm-agnostic because all $f$5 norms collapse to the nonzero-parameter count up to constants: $f$6 Consequently,

$f$7

and the theorem immediately transfers to any $f$8 regularizer. The appendix summarizes this as

$f$9

In precise form,

$K$0

(Musat, 11 May 2026).

This has a direct Bayesian interpretation. If $K$1 decay is identified with the Gaussian prior

$K$2

then the induced prior on outputs,

$K$3

is comparable to Solomonoff’s universal prior. The corollary states that there exist constants $K$4 such that

$K$5

Since Solomonoff’s prior satisfies

$K$6

the induced neural prior agrees with the universal prior up to a logarithmic factor in the exponent.

The same encoding argument yields an MDL-style generalization bound,

$K$7

Under fixed precision, weight decay therefore ceases to be only a smoothness or optimization heuristic and becomes a discrete description-length prior over computable outputs.

4. Computable approximations and bounded-overhead variants

A central computable approximation to prefix-free complexity is the Solovay function: a computable $K$8 such that $K$9 for all $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$0, and $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$1 for infinitely many $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$2. Equivalently,

$$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$3

The structural characterization is especially strong: $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$4 Among computable upper bounds on $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$5, these are precisely the ones that are globally valid and infinitely often sharp, and they function as “best possible” computable surrogates of a noncomputable complexity measure (0902.1041).

Another finite-resolution viewpoint studies the complexity of the complexity value itself. For every $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$6, there exists a string $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$7 of length $$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$8 such that

$$K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty.$$9

and the prefix-free analogue satisfies

$K$0

for suitable $K$1. The same paper shows that the discrepancy between maximal plain and maximal prefix complexity at length $K$2 is governed by the iterated quantity $K$3. This frames $K$4 as the natural finite-resolution scale on which the value of complexity itself can be maximally hard to recover (Bauwens et al., 2012).

Classical lowness notions can also be weakened by replacing constant additive slack with slowly growing computable bounds. For a real $K$5, the classes

$K$6

generalize low-for-$K$7 and $K$8-triviality. When the bounds range over all $K$9 orders, equivalently over all finite-to-one approximable functions, the classical equivalence $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$0 breaks down: $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$1, $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$2 is cofinal in the Turing degrees, $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$3 remains downward closed under $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$4, and $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$5 is instead shown to be downward closed under $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$6 and closed under $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$7 (Herbert, 2014). The approximation budget is therefore not a benign technicality; it changes the structure of the induced lowness classes.

5. Finite-prefix, oracle-relative, and finite-state formulations

Finite-precision questions also arise when one compares representations of the same real number at matched approximation scales. For computable $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$8, the $${}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\},$$9-expansion paper proves a two-sided complexity comparison between the first $O(1)$0 digits of a suitable $O(1)$1-expansion and the first $O(1)$2 bits of the binary expansion of the same real: $O(1)$3 It also proves the lower bound

$O(1)$4

for every $O(1)$5-expansion $O(1)$6, and the converse upper bound for a specially constructed $O(1)$7. Thus, at matched precision, $O(1)$8-based representations need not be more algorithmically complex than binary ones (Abadie et al., 2024).

Oracle-relative finite-string complexity leads to another notion of finite informational advantage. Levin mutual information of reals $O(1)$9 and $s$0 is defined by

$s$1

and $s$2 has finite self-information if $s$3. A key sufficient condition is that if

$s$4

for a suitable $s$5, then $s$6 has finite self-information. The paper constructs a perfect $s$7 class all of whose members satisfy such inequalities, hence a perfect set of reals with finite self-information. With $s$8, the same construction yields a perfect set of reals low for both effective Hausdorff dimension and effective packing dimension (Herbert, 2012).

Restricted-model analogues of Kolmogorov complexity provide a still more explicit finite-state notion. An automatic description mode is an automatic relation $s$9 that is the graph of an $K(x)=\min\{\,|p|:U(p)=x\,\}.$00-valued function, and its induced complexity is

$K(x)=\min\{\,|p|:U(p)=x\,\}.$01

This automatic Kolmogorov complexity has no optimal description mode, but it still supports sharp asymptotic characterizations: a binary sequence is normal iff

$K(x)=\min\{\,|p|:U(p)=x\,\}.$02

for every automatic description mode $K(x)=\min\{\,|p|:U(p)=x\,\}.$03, and finite-state dimension satisfies

$K(x)=\min\{\,|p|:U(p)=x\,\}.$04

The finite-state theory therefore replaces universal computable decompressors by automata and full algorithmic complexity by asymptotic incompressibility under finite-state decoding (Kozachinskiy et al., 2017).

6. Complexity under transformations, extraction, and adjacent extensions

Finite-precision complexity can be analyzed under stochastic transformations. If a binary string $K(x)=\min\{\,|p|:U(p)=x\,\}.$05 of length $K(x)=\min\{\,|p|:U(p)=x\,\}.$06 has complexity rate $K(x)=\min\{\,|p|:U(p)=x\,\}.$07, and each bit is independently flipped with probability $K(x)=\min\{\,|p|:U(p)=x\,\}.$08, then with

$K(x)=\min\{\,|p|:U(p)=x\,\}.$09

the noisy string satisfies

$K(x)=\min\{\,|p|:U(p)=x\,\}.$10

where the explicit loss can be taken as

$K(x)=\min\{\,|p|:U(p)=x\,\}.$11

The paper also proves that this lower bound is optimal in general and extends the result to effective Hausdorff dimension for infinite sequences (Posobin et al., 2018).

A complementary line studies extraction of algorithmic information from dependent sources. For two length-$K(x)=\min\{\,|p|:U(p)=x\,\}.$12 strings $K(x)=\min\{\,|p|:U(p)=x\,\}.$13 with

$K(x)=\min\{\,|p|:U(p)=x\,\}.$14

there is a computable $K(x)=\min\{\,|p|:U(p)=x\,\}.$15 such that $K(x)=\min\{\,|p|:U(p)=x\,\}.$16 has length

$K(x)=\min\{\,|p|:U(p)=x\,\}.$17

and complexity

$K(x)=\min\{\,|p|:U(p)=x\,\}.$18

A stronger theorem produces an output $K(x)=\min\{\,|p|:U(p)=x\,\}.$19 of length $K(x)=\min\{\,|p|:U(p)=x\,\}.$20 such that

$K(x)=\min\{\,|p|:U(p)=x\,\}.$21

For infinite sequences, a truth-table reduction raises randomness rate from $K(x)=\min\{\,|p|:U(p)=x\,\}.$22 to $K(x)=\min\{\,|p|:U(p)=x\,\}.$23 even under linear dependency $K(x)=\min\{\,|p|:U(p)=x\,\}.$24, provided $K(x)=\min\{\,|p|:U(p)=x\,\}.$25 is sufficiently small (0902.2141).

Several adjacent notions broaden the finite-precision theme beyond classical string complexity. Compression complexity studies the shortest compressor rather than the shortest decompressor: for every $K(x)=\min\{\,|p|:U(p)=x\,\}.$26, there exists a compression function $K(x)=\min\{\,|p|:U(p)=x\,\}.$27 with $K(x)=\min\{\,|p|:U(p)=x\,\}.$28 such that $K(x)=\min\{\,|p|:U(p)=x\,\}.$29 whenever $K(x)=\min\{\,|p|:U(p)=x\,\}.$30, and otherwise $K(x)=\min\{\,|p|:U(p)=x\,\}.$31 (Fenner et al., 2017). In quantum information, the Kolmogorov complexity of a finite unitary transformation is defined by mapping $K(x)=\min\{\,|p|:U(p)=x\,\}.$32 to the normalized positive operator $K(x)=\min\{\,|p|:U(p)=x\,\}.$33, where $K(x)=\min\{\,|p|:U(p)=x\,\}.$34, purifying it, and measuring the shortest quantum description of the purification; the resulting bound is

$K(x)=\min\{\,|p|:U(p)=x\,\}.$35

for an $K(x)=\min\{\,|p|:U(p)=x\,\}.$36-qubit unitary (Kaltchenko, 2021).

The literature also makes clear that not every computable proxy deserves to be identified with Kolmogorov complexity itself. The braid-group study uses

$K(x)=\min\{\,|p|:U(p)=x\,\}.$37

as a computable estimator, reports banded reductions under braid relations, and explicitly notes that its results are statements about empirical compressibility rather than exact $K(x)=\min\{\,|p|:U(p)=x\,\}.$38; it further describes its universality claim as empirical or generalized rather than rigorously established (Shayda, 2013). That distinction is fundamental across the area: exact finite-precision theorems depend on explicit discretization or restricted models, whereas purely empirical compression statistics remain proxy-based.

Across these lines of work, finite-precision Kolmogorov complexity is best understood not as a single invariant but as a family of exact and approximate description-length formalisms. The unifying idea is that once a representational scale, a computable upper bound, or a restricted descriptive mechanism is fixed, complexity becomes analyzable in ways that the full noncomputable quantity does not permit.