--- title: Finite-Precision Kolmogorov Complexity url: https://www.emergentmind.com/topics/finite-precision-kolmogorov-complexity type: topic --- # Finite-Precision Kolmogorov Complexity Searching arXiv for recent and foundational papers relevant to finite-precision Kolmogorov complexity. Finite-precision Kolmogorov complexity concerns description length when either the object being described, the representation used for that object, or the descriptive mechanism itself is discretized, truncated, or otherwise restricted. In the works surveyed here, the theme appears in several technically distinct forms: fixed-precision neural parameterizations, computable upper bounds on noncomputable complexity, finite-prefix comparisons between numeral systems, oracle-relative bounds on finite strings, and restricted decompressor models. A central recent result shows that, for looped neural networks with fixed-precision weights, the minimum norm required to emit a binary string coincides with its Kolmogorov complexity up to a logarithmic factor [2605.10878]. ## 1. Formal scope and basic distinctions The starting point is the standard definition of prefix-free Kolmogorov complexity for a universal prefix-free machine \(U\), \[ K(x)=\min\{\,|p|:U(p)=x\,\}. \] Because \(K\) is only upper semicomputable, several papers treat finite-precision or computable surrogates rather than exact values. One canonical characterization states that a computable function \(f\) is an upper bound on \(K\) up to an additive constant exactly when its Kraft sum is finite: \[ K \le f + O(1) \quad\Longleftrightarrow\quad \sum_x 2^{-f(x)}<\infty. \] This places computable approximations to \(K\) inside the usual Kraft–Chaitin framework and makes explicit that finite-precision estimates of complexity are necessarily upper bounds or restricted-model analogues rather than exact evaluations of the noncomputable invariant [0902.1041]. A different, more literal notion of finite precision arises when descriptions are stored in a discrete parameter alphabet. In the neural setting, the relevant regime is \[ {}_{\delta,M}=\bigl\{\theta\in(\delta\cdot\mathbb Z\cap[-M,M])^d:d\in\mathbb N\bigr\}, \] so each nonzero parameter is drawn from a constant-size set of admissible values. In that regime each nonzero weight carries only \(O(1)\) bits, and norm, sparsity, and description length become comparable. The same paper stresses that this comparability fails outside fixed precision: real-valued weights can be super-Turing, and bounded rational weights can encode arbitrarily many bits in their numerators and denominators, so norm no longer reflects stored information [2605.10878]. These two viewpoints already delineate the field. One concerns computable approximations to a noncomputable quantity; the other concerns exact description length after an explicit discretization of the representational medium. ## 2. Fixed-precision neural complexity as a description-length measure The most explicit finite-precision theorem in the surveyed literature defines the neural complexity of a computable binary string \(s\) as the minimum number of nonzero parameters in any fixed-precision looped network that outputs \(s\): \[ N(s)=\inf_\theta\{\,\theta_0:\theta\in{}_{\delta,M},\ \theta\text{ outputs }s\,\}, \] where \(\theta_0\) denotes the nonzero-parameter count. The main theorem gives a two-sided sandwich bound: \[ N(s)\le K(s)+c_U,\qquad K(s)\le c_d\,N(s)\log_2N(s)+c_d. \] This makes the minimum fixed-precision network size a bona fide description-length measure, equal to Kolmogorov complexity up to a logarithmic factor [2605.10878]. The upper bound is obtained by encoding a shortest universal Turing machine program \(p\) for \(s\) directly into the network weights. The construction uses a universal looped network \(T_U\), an iteration-1 gate that fires only at the first time step, and a routing layer with one ternary parameter per bit of \(p\). In the paper’s notation, \[ {\theta_p}_0=c_U'+|p|+O(1)=|p|+c_U, \] so program bits are written into weights at unit cost per bit. The lower bound treats a fixed-precision network as a finite discrete object. If the network has \(W=\theta_0\) nonzero parameters, each can be encoded as a tuple \[ (\ell_j,u_j,v_j,w_j), \] consisting of layer index, source neuron, target neuron, and weight value. After pruning unused neurons and collapsing empty layers, one may assume at most \(n\le 2W\) neurons and \(L'\le W\) layers. Each nonzero parameter then needs at most \[ \lceil\log_2L'\rceil+2\lceil\log_2n\rceil+\lceil\log_2|\mathcal V|\rceil \le 3\log_2W+O(1) \] bits, where the value alphabet \(\mathcal V=(\delta\mathbb Z\cap[-M,M])\setminus\{0\}\) has constant size. Hence the entire network has a description of length \[ |\hat p(\theta)|\le c_dW\log_2W+c_d, \] and a constant-size simulator reconstructs the network and runs it to produce \(s\). The logarithmic term comes from addressing overhead: sparse parameters must specify not only their values but also their locations. The same paper shows that this is not proof slack. For a permutation \(\pi:[N]\to[N]\), if \(s_\pi\) is the row-major serialization of the permutation matrix \(P_\pi\), then for typical \(\pi\), \[ K(s_\pi)=\log_2N!+O(\log N)=\Theta(N\log N), \] while a looped network with only \(\Theta(N)\) nonzero ternary parameters outputs \(s_\pi\). Thus \[ K(s_\pi)=\Theta\!\bigl(N(s_\pi)\log N(s_\pi)\bigr), \] which realizes the lower-bound factor. ## 3. Norm collapse, Solomonoff priors, and regularization Within fixed precision, the same neural result is norm-agnostic because all \(L_p\) norms collapse to the nonzero-parameter count up to constants: \[ \delta^p\theta_0\le \theta_p^p=\sum_{i=1}^d|\theta_i|^p\le M^p\theta_0. \] Consequently, \[ \theta_p^p=\Theta(\theta_0), \] and the theorem immediately transfers to any \(L_p\) regularizer. The appendix summarizes this as \[ L_1:\ N_1(s)=\tilde\Theta(K(s)),\qquad L_2:\ N_2(s)^2=\tilde\Theta(K(s)),\qquad L_p:\ N_p(s)^p=\tilde\Theta(K(s)). \] In precise form, \[ N_p(s)^p\le M^p(K(s)+c_U),\qquad K(s)\le \tfrac{c_d}{\delta^p}N_p(s)^p\log\!\bigl(N_p(s)^p/\delta^p\bigr)+c_d \] [2605.10878]. This has a direct Bayesian interpretation. If \(L_2\) decay is identified with the Gaussian prior \[ \pi(\theta)\propto \exp\!\left(-\tfrac{\lambda}{2}\theta_2^2\right), \] then the induced prior on outputs, \[ Q(s)=\sum_{\theta:\theta\text{ outputs }s}\pi(\theta), \] is comparable to Solomonoff’s universal prior. The corollary states that there exist constants \(\alpha,\beta>0\) such that \[ 2^{-K(s)-\alpha}\le Q(s)\le 2^{-K(s)/(\beta\log K(s))}. \] Since Solomonoff’s prior satisfies \[ -\log M(s)=K(s)+O(1), \] the induced neural prior agrees with the universal prior up to a logarithmic factor in the exponent. The same encoding argument yields an MDL-style generalization bound, \[ L(\theta)\le \hat L(\theta)+\widetilde O\!\left(\sqrt{\frac{c_d\,\theta_2^2\log \theta_2^2+\log(1/\eta)}{m}}\right). \] Under fixed precision, weight decay therefore ceases to be only a smoothness or optimization heuristic and becomes a discrete description-length prior over computable outputs. ## 4. Computable approximations and bounded-overhead variants A central computable approximation to prefix-free complexity is the Solovay function: a computable \(f\) such that \(K(x)\le f(x)+O(1)\) for all \(x\), and \(f(x)\le K(x)+O(1)\) for infinitely many \(x\). Equivalently, \[ \liminf_{x\to\infty}(f(x)-K(x))<\infty. \] The structural characterization is especially strong: \[ f\text{ is a Solovay function} \quad\Longleftrightarrow\quad \sum_x2^{-f(x)}\text{ is finite and Martin-Löf random.} \] Among computable upper bounds on \(K\), these are precisely the ones that are globally valid and infinitely often sharp, and they function as “best possible” computable surrogates of a noncomputable complexity measure [0902.1041]. Another finite-resolution viewpoint studies the complexity of the complexity value itself. For every \(n\), there exists a string \(x\) of length \(n\) such that \[ C(C(x)\mid x)\ge \log n-O(1), \] and the prefix-free analogue satisfies \[ K(K(x)\mid x)=\log n+O(1) \] for suitable \(x\). The same paper shows that the discrepancy between maximal plain and maximal prefix complexity at length \(n\) is governed by the iterated quantity \(K(K(n)\mid n)\). This frames \(\log n\) as the natural finite-resolution scale on which the value of complexity itself can be maximally hard to recover [1202.6668]. Classical lowness notions can also be weakened by replacing constant additive slack with slowly growing computable bounds. For a real \(A\), the classes \[ LK(f):\ K(\sigma)\leq^+ K^A(\sigma)+f(\sigma), \qquad KT(g):\ K(A{\upharpoonright}n)\leq^+ K(n)+g(n) \] generalize low-for-\(K\) and \(K\)-triviality. When the bounds range over all \(\Delta^0_2\) orders, equivalently over all finite-to-one approximable functions, the classical equivalence \(KT(0)=LK(0)\) breaks down: \(LK()\subsetneq KT()\), \(KT()\) is cofinal in the Turing degrees, \(LK(f)\) remains downward closed under \(\le_T\), and \(KT()\) is instead shown to be downward closed under \(\le_{wtt}\) and closed under \(\oplus\) [1410.3615]. The approximation budget is therefore not a benign technicality; it changes the structure of the induced lowness classes. ## 5. Finite-prefix, oracle-relative, and finite-state formulations Finite-precision questions also arise when one compares representations of the same real number at matched approximation scales. For computable \(\beta\in(1,2]\), the \(\beta\)-expansion paper proves a two-sided complexity comparison between the first \(k\) digits of a suitable \(\beta\)-expansion and the first \(\lceil k\log_2\beta\rceil\) bits of the binary expansion of the same real: \[ \left|K\!\left[\xf'_{1:k}\right]-K\!\left[\xf_{1:\lceil k\log_2(\beta)\rceil}\right]\right|