---
title: Kolmogorov-Arnold Theorem
url: https://www.emergentmind.com/topics/kolmogorov-arnold-theorem-eaf2a743-dbb4-404b-afba-4a3fc6236cdf
type: topic
---

# Kolmogorov-Arnold Theorem

The Kolmogorov-Arnold Theorem—often called the Kolmogorov-Arnold Superposition Theorem or Kolmogorov Superposition Theorem (KST)—is a foundational result in real analysis and approximation theory. It establishes that every multivariate continuous function can be exactly represented as a finite sum of compositions of continuous univariate functions. This theorem provides a constructive alternative to the Universal Approximation Theorem and underpins the development of Kolmogorov-Arnold Networks (KANs), with significant implications for both pure mathematics and modern machine learning, particularly regarding the curse of dimensionality [2508.00247, 2411.10622, 2502.16664].

## 1. Historical Background and Motivation

The theorem originates from the resolution of Hilbert’s 13th problem, which questioned whether solutions to the general seventh-degree polynomial could be expressed as superpositions of functions of only two variables. The prevailing belief in the early 20th century was that continuous multivariate functions intrinsically required higher-arity functional representations. Kolmogorov’s 1957 result, and Arnold’s subsequent refinements, overturned this assumption by proving that for any continuous function $f:[0,1]^n\rightarrow\mathbb{R}$, there exists an explicit, finite, and exact decomposition into sums and compositions of continuous univariate maps [2411.10622].

This result demonstrates a deep structural property of function spaces and provides a constructive blueprint for reducing the effective complexity of high-dimensional function approximation to tractable univariate operations, a principle increasingly central to the design of scalable machine learning models [2502.16664].

## 2. Formal Statement and Canonical Representations

For $n\geq 1$ and any continuous function $f:[0,1]^n\to\mathbb{R}$, there exist continuous “inner” functions $\{\phi_{q, p}\}$ and continuous “outer” functions $\{\Phi_q\}$ such that
\[
f(x_1, ..., x_n) = \sum_{q=1}^{2n+1} \Phi_q\left(\sum_{p=1}^n \phi_{q,p}(x_p)\right)
\]
for all $(x_1, ..., x_n)\in[0,1]^n$ [2502.16664, 2411.10622, 2508.00247]. The inner functions act independently on each coordinate, and their sum serves as the argument for each outer function.

Further simplifications by Lorentz and Sprecher show that it is possible to reduce the set of outer and inner functions—up to specific shifts and scalings—to a single monotonic inner function and a single continuous outer function:
\[
f(x) = \sum_{q=0}^{2n} g\left(\sum_{p=1}^n \lambda_p\,\psi(x_p + q\,a) + c_q\right)
\]
with constants $\lambda_p$, $a$, $c_q$ and continuous, monotonic $\psi$, and continuous $g$ [2508.00247].

## 3. Constructive Proofs and Smoothness Properties

The original proofs by Kolmogorov and Arnold are constructive, relying on space-filling or Cantor set embeddings to reduce the multivariate domain to a single univariate argument and then reconstruct the original function through the compositional structure [2007.15884, 2502.16664]. However, the constructed inner functions are generally only continuous and can exhibit highly irregular (non-smooth, even Cantor staircase-like) behavior.

Recent work addresses the inherent irregularity of the outer and inner functions in this decomposition. For instance, a smoothness-preserving variant based on Cantor-set embeddings ensures that if $f$ is Hölder-$\beta$ smooth, the induced outer function $g$ retains controlled Hölder smoothness, albeit with an exponent reduced as a function of the input dimension. Specifically, for $f$ Hölder-$\beta$, the induced $g$ is Hölder with exponent $\alpha=\frac{\beta\log2}{d\log3}$ on the Cantor set [2007.15884].

## 4. Implications for Approximation, Neural Networks, and the Curse of Dimensionality

The Kolmogorov-Arnold Theorem provides an *exact* representation for all continuous multivariate functions using a fixed number of univariate continuous functions—critically, this number is finite and scales only linearly in input dimension. In contrast to grid-based approximation, which is exponential in $n$ (the curse of dimensionality), the KST-based constructions reduce the effective parameterization to $O(n G)$ for grid size $G$ per inner/outer map. For KANs, the sup-norm error on smooth $f$ using B-spline parameterization of each univariate function is $O(G^{-(k+1)})$ for $k$ times continuously differentiable maps, with no $n$ in the exponent [2411.10622, 2502.16664].

Deep learning architectures can leverage this result. By replacing classical MLPs with networks whose hidden layers implement sums of trainable univariate functions (e.g., via splines or other bases), one obtains universal approximation with parameter counts that do not exhibit exponential growth in $n$ [2412.17853].

The theorem also provides concrete strategies for approximating both the inner and outer univariate functions—for example, via deep ReLU or sinusoidal networks—offering explicit error bounds and leading to the design of efficient architectures with provable scalability [2508.00247, 1906.11945].

### Table: Comparison of Dimensional Scaling

| Method                       | # Terms/Params required   | Dependence on $n$        |
|------------------------------|--------------------------|--------------------------|
| Grid-based (classical)       | $O(m^n)$                 | Exponential              |
| Kolmogorov-Arnold Theorem    | $O(nG)$                  | Linear                   |

Parameter scaling for KST-based decompositions is linear in input dimension, while classical grid methods are exponential.

## 5. Variants: Sinusoidal, Spline, and Geometric Approximations

Recent developments extend the theorem by restricting the functional forms of the inner and outer maps. "Sinusoidal Approximation Theorem for Kolmogorov-Arnold Networks" proves that both layers can be realized as finite sums of sinusoids with learnable frequencies and amplitudes, and fixed, linearly spaced phases [2508.00247]. The main theorems yield, for any $f:[0,1]^n\to\mathbb{R}$, an arbitrarily accurate representation using such sinusoidal expansions:
\[
f(x) \approx \sum_{q=1}^{2n+1} \sum_{j=0}^M B_{qj}\,\sin\Bigl(\omega_{2j} \Bigl( \sum_{p=1}^n \sum_{k=0}^N A_{pq,k}\,\sin(\omega_{1k} x_p + \varphi_{1k}) \Bigr) + \varphi_{2j}\Bigr)
\]
ensuring that universal approximation capability is preserved under these architectural constraints.

Similarly, KANs are typically instantiated with the inner and outer functions parameterized as B-splines, trained via backpropagation [2411.10622, 2412.17853]. The "Geometric Kolmogorov-Arnold Superposition Theorem" generalizes the construction to enforce $O(n)$, $S_n$, $O(1,n)$, and $GL$ invariance or equivariance, enabling the modeling of physical systems with rigid-motion or permutation symmetries:
\[
f(\{\mathbf{x}_i\}) = \sum_{q=1}^{2mn+1} \psi_q\left( \sum_{i=1}^m \sum_{j=1}^n \phi_{q, ij}(\langle\mathbf{x}_i, \mathbf{x}_j\rangle) \right )
\]
with explicit guarantees for symmetry preservation [2502.16664].

## 6. Applications in Machine Learning and Scientific Modeling

The Kolmogorov-Arnold decomposition has motivated the design of Kolmogorov-Arnold Networks (KANs), a class of neural architectures that implement the theorem’s superposition structure. Each KAN layer aggregates learnable univariate transformations of input coordinates, summed and passed through outer univariate functions, often parameterized via splines or other bases [2412.17853, 2411.10622].

KANs have been successfully applied to:

- Time-series forecasting, including zero-shot domain adaptation scenarios with doubly-residual N-BEATS backbones and adversarial training for invariant representations [2412.17853]
- Physical and chemical modeling with built-in geometric or permutation symmetry constraints [2502.16664]
- Scientific data modeling where interpretability of the functional decomposition is as crucial as accuracy

Empirically, KANs demonstrate parameter efficiency and robust scaling laws, outperforming classical MLPs in high-dimensional regimes and symmetry-constrained domains, while maintaining universal approximation guarantees [2502.16664, 2411.10622].

## 7. Limitations, Open Problems, and Future Directions

Key limitations of the Kolmogorov-Arnold Theorem and its neural instantiations include:

- The high non-uniqueness of the decomposition, as multiple equivalent sets of inner/outer univariate functions can realize the same $f$ [2502.16664]
- Training complexity—KANs require the optimization of numerous univariate functions, which can pose practical challenges at scale
- Expressivity-efficiency trade-offs: while theoretically universal, practical KAN implementations may truncate the number of terms or basis functions, impacting empirical accuracy depending on task and domain
- Extension to wider classes of group symmetries remains an active research direction [2502.16664]

Ongoing research also investigates the nontrivial relationship between the smoothness properties of $f$ and those of the induced inner and outer functions, as well as architectural variants that combine the KST-principled design with message-passing, attention, or non-traditional bases [2007.15884, 2508.00247].

---

In summary, the Kolmogorov-Arnold Theorem provides a mathematically rigorous decomposition of continuous multivariate functions into superpositions of continuous univariate maps. This result supplies the theoretical foundation for a new class of neural architectures (KANs) that promise scalability, interpretability, and direct address of the curse of dimensionality, with active research extending these ideas to sinusoidal bases and geometric symmetries [2508.00247, 2411.10622, 2412.17853, 2502.16664].

Source: https://www.emergentmind.com/topics/kolmogorov-arnold-theorem-eaf2a743-dbb4-404b-afba-4a3fc6236cdf