---
title: Kolmogorov-Optimal Approximants
url: https://www.emergentmind.com/topics/kolmogorov-optimal-approximants
type: topic
---

# Kolmogorov-Optimal Approximants

A Kolmogorov-optimal approximant is a constructed object—function, operator, distribution, or matrix—that achieves or attains the minimal possible worst-case approximation error with respect to the Kolmogorov n-width for a given class and norm. This concept permeates nonlinear, adaptive, and probabilistic approximation theory, and connects to explicit algorithms both in function spaces and for discrete structures. Kolmogorov-optimality is quantified asymptotically by the Kolmogorov n-width, which measures the best accuracy achievable by any $n$-dimensional (possibly nonlinear) approximant, and is intimately linked to partitions, multifractal analysis, and, in finite-dimensional settings, to best low-rank approximations.

## 1. Kolmogorov Widths and the Optimization Principle

Given a compact subset $K$ of a normed space $X$, the Kolmogorov n-width $d_n(K;X)$ is defined as
$$
d_n(K;X) = \inf_{\dim S = n} \sup_{f \in K} \inf_{g\in S} \|f - g\|_X,
$$
where the infimum is taken over all $n$-dimensional subspaces $S\subset X$. The n-width quantifies the minimal maximal deviation achievable by any $n$-dimensional method, linear or nonlinear, for approximating $K$. 

An approximant or family of approximants is called "Kolmogorov-optimal" (or more precisely "Kolmogorov-optimal of order $n$") if it achieves error $d_n(K;X)$; that is, the worst-case error of the method equals the Kolmogorov width. For a given class, this is the gold standard for best-possible approximation rates.

In the matrix case, it is explicitly shown that the sequence of Kolmogorov n-widths for the image of the unit ball under a matrix $A$ coincides with the singular values:
$$
d_n(A(B_2^m)) = \sigma_{n+1},
$$
and any best rank-$n$ approximation in the spectral norm is Kolmogorov-optimal [2007.13196].

## 2. Adaptive Partitions and Asymptotic Rates

For nonlinear and nonuniform approximation, Kolmogorov-optimality is achieved via adaptive partition schemes. Consider the unit cube $Q \subset \mathbb{R}^d$ and a monotone set function $J: \mathcal{D} \to [0,\infty)$ defined on dyadic subcubes $\mathcal{D}$, representing a "badness" or local approximation fidelity measure.

The construction uses an adaptive subdivision: for $x > 1/J(Q_0)$, define the minimal "x-good" partition $G_x$ by subdividing such that, for each cube $Q\in G_x$,
$$
J(Q) < 1/x \quad \text{but} \quad J(Q^-) \geq 1/x,
$$
with $Q^-$ the parent of $Q$. This produces a partition $G_x$ of minimal cardinality $M(x)=|G_x|$ with the property $\sup_{Q\in G_x} J(Q)<1/x$ [2312.16644].

The rate of growth $M(x)$ is governed by the critical zero $s^*$ of the partition function
$$
\tau_J(q) = \limsup_{n\to\infty} \frac{1}{n\log2} \log\left(\sum_{Q\in\mathcal{D}_n} J(Q)^q\right),
$$
and the associated zeta sum
$$
Z_J(s) = \sum_{Q\in\mathcal{D}} J(Q)^s.
$$
Here, $s^* = \inf\{s\ge0: Z_J(s)<\infty\}$, which is equivalently the unique solution to $\tau_J(q)=0$.

Under this scheme, the exact asymptotics are
$$
\lim_{x\to\infty} \frac{\log M(x)}{\log x} = s^*,
$$
and, dually, optimal $n$-term partitioning achieves local error decay
$$
\gamma_n \sim C n^{-1/s^*}, \quad n\to\infty,
$$
where $\gamma_n = \min\,\{\max_{Q\in P} J(Q):\, P \text{ partition, } |P|\le n\}$ [2312.16644].

## 3. Explicit Construction of Kolmogorov-Optimal Approximants

The adaptive partition $G_x$ serves as the geometric scaffold for constructing Kolmogorov-optimal approximants:
- On each cube $Q\in G_x$, take the best local approximation of the target function $f$—for instance, by local averaging or polynomial projection.
- The global approximant $A_n f$ is assembled by superposing these local approximants.

Formally, letting $n \approx |G_{n^{1/s^*}}|$, one obtains
$$
\|f - A_n f\| \leq n^{-1/s^*},
$$
matching the order of the Kolmogorov width $d_n \sim n^{-1/s^*}$ [2312.16644].

In the low-rank matrix context, every orthonormal basis in an $n$-dimensional Kolmogorov-optimal subspace $S$ yields a best rank-$n$ approximation $A_n = P_S A$, with extremal error
$$
\|A-A_n\|_2 = \sigma_{n+1},
$$
and the manifold of such optimal subspaces is characterized by spectral-positivity and orthogonality constraints [2007.13196].

For discrete random variables, the Kolmogorov-optimal $m$-approximant to a variable $X$ is the one minimizing Kolmogorov distance $d_K(X,Y)$ over all $Y$ with support size at most $m$. The construction uses a min-max path algorithm in a DAG built from the support of $X$, ensuring provable optimality with respect to the Kolmogorov metric [1805.07535].

## 4. Applications and Variants

Kolmogorov-optimal approximants appear in numerous domains:

- **Spectral theory of differential operators**: For singular Sturm–Liouville (Krein–Feller) problems, the set function $J(Q) = \mu(Q)^s |Q|^a$ relates to eigenvalue asymptotics via Weyl-type laws $\lambda_k \sim C k^{s^*}$ [2312.16644].
- **Quantization of measures**: Setting $J(Q)=\mu(Q)^r$ yields the $r$th-order quantization dimension $D_r = s^*$, governing rates of optimal finite-support approximations [2312.16644].
- **Sobolev and Besov embeddings**: For $J_{\mu,r,p}(Q) = [\mu(Q)]^{2/p} |Q|^{(2r/d)-1}$, the critical exponent $s^*$ controls the decay $d_n \sim n^{-1/s^*}$ for nonlinear widths [2312.16644].
- **Low-rank matrix approximations**: The best rank-$n$ approximations with respect to the spectral norm coincide with the Kolmogorov widths and allow flexibility beyond classical SVD, including structured or problem-oriented subspaces [2007.13196].
- **Discretized distributions**: In probabilistic modeling and simulation, Kolmogorov-optimal $m$-approximants offer minimal worst-case cdf deviation while reducing support cardinality [1805.07535].

## 5. Smooth Kolmogorov-Type Approximants

The classical Kolmogorov-Arnold representation is exact but exhibits pathological lack of smoothness in its single-variable "inner" functions. Recent work constructs approximate Kolmogorov-Arnold superpositions where all inner and outer functions are $C^2$, resulting in explicit, parallelizable, and fully smooth approximants that nonetheless achieve the Kolmogorov-optimal approximation rate for $\alpha$-Hölder continuous targets:
$$
\|f-\hat{f}\|_\infty \leq C N^{-\alpha/d},
$$
matching the Kolmogorov width $w_N(\mathcal{H}^\alpha) \sim N^{-\alpha/d}$. This construction leverages translated/dilated $C^2$ shape functions and row-wise $C^2$ interpolation to maintain smoothness throughout [2508.04392].

## 6. Fractal and Multifractal Considerations

The asymptotic order of Kolmogorov-optimal approximants is determined by multifractal parameters of the underlying set function $J$. Fractal-geometric quantities, such as $\tau_J(0)$ and $\dim_\infty(J)$, provide explicit upper and lower bounds on the critical exponent $s^*$:
$$
\frac{\tau_J(0)}{\tau_J(0)-\tau_J(1)} \leq s^* \leq \frac{[\dim_\infty(J) + \tau_J(1)]}{\dim_\infty(J)} \leq \frac{\tau_J(0)}{\dim_\infty(J)} \leq \frac{d}{\dim_\infty(J)}.
$$
This relates the performance of Kolmogorov-optimal approximants to the fine geometric structure of the measure or function being approximated, and connects the theory to entropy and partition zeta functions [2312.16644].

## 7. Algorithmic and Computational Aspects

Kolmogorov-optimal approximants admit constructive algorithms in both finite and infinite-dimensional settings:
- **Discrete random variables**: A polynomial-time $O(n^2m)$ algorithm constructs the Kolmogorov-optimal $m$-approximant, outperforming linear programming and heuristic binning [1805.07535].
- **Low-rank matrix approximations**: Iterated projection/orthonormalization algorithms converge to the SVD-optimal subspaces, while the entire manifold of optimal subspaces may be efficiently explored, allowing exploitation of structure and flexibility [2007.13196].
- **Adaptive partitions**: The partition algorithms generate near-optimal support and storage cost for piecewise polynomial approximations, applicable in high dimensions and for measures with fractal support [2312.16644].
- **Smooth superpositions**: Explicit recipes using $C^2$ shape functions yield directly implementable, parallelizable two-layer architectures [2508.04392].

These computational strategies ensure that Kolmogorov-optimality is not a purely existential property but a practical one across diverse settings.

Source: https://www.emergentmind.com/topics/kolmogorov-optimal-approximants