Kolmogorov-Arnold Theorem

• Kolmogorov-Arnold Theorem is a foundational result that decomposes continuous multivariate functions into finite sums of compositions of univariate functions.
• It provides a constructive blueprint to reduce the curse of dimensionality, influencing modern machine learning architectures like Kolmogorov-Arnold Networks.
• Constructive proofs using space-filling techniques and Cantor-set embeddings lead to explicit univariate approximations with practical error bounds.

The Kolmogorov-Arnold Theorem—often called the Kolmogorov-Arnold Superposition Theorem or Kolmogorov Superposition Theorem (KST)—is a foundational result in real analysis and approximation theory. It establishes that every multivariate continuous function can be exactly represented as a finite sum of compositions of continuous univariate functions. This theorem provides a constructive alternative to the Universal Approximation Theorem and underpins the development of Kolmogorov-Arnold Networks (KANs), with significant implications for both pure mathematics and modern machine learning, particularly regarding the curse of dimensionality (Gleyzer et al., 1 Aug 2025, Basina et al., 2024, Alesiani et al., 23 Feb 2025).

1. Historical Background and Motivation

The theorem originates from the resolution of Hilbert’s 13th problem, which questioned whether solutions to the general seventh-degree polynomial could be expressed as superpositions of functions of only two variables. The prevailing belief in the early 20th century was that continuous multivariate functions intrinsically required higher-arity functional representations. Kolmogorov’s 1957 result, and Arnold’s subsequent refinements, overturned this assumption by proving that for any continuous function $f:[0,1]^n\rightarrow\mathbb{R}$, there exists an explicit, finite, and exact decomposition into sums and compositions of continuous univariate maps (Basina et al., 2024).

This result demonstrates a deep structural property of function spaces and provides a constructive blueprint for reducing the effective complexity of high-dimensional function approximation to tractable univariate operations, a principle increasingly central to the design of scalable machine learning models (Alesiani et al., 23 Feb 2025).

2. Formal Statement and Canonical Representations

For $n\geq 1$ and any continuous function $f:[0,1]^n\to\mathbb{R}$, there exist continuous “inner” functions $\{\phi_{q, p}\}$ and continuous “outer” functions $\{\Phi_q\}$ such that

$$f(x_1, ..., x_n) = \sum_{q=1}^{2n+1} \Phi_q\left(\sum_{p=1}^n \phi_{q,p}(x_p)\right)$$

for all $(x_1, ..., x_n)\in[0,1]^n$ (Alesiani et al., 23 Feb 2025, Basina et al., 2024, Gleyzer et al., 1 Aug 2025). The inner functions act independently on each coordinate, and their sum serves as the argument for each outer function.

Further simplifications by Lorentz and Sprecher show that it is possible to reduce the set of outer and inner functions—up to specific shifts and scalings—to a single monotonic inner function and a single continuous outer function: $$f(x) = \sum_{q=0}^{2n} g\left(\sum_{p=1}^n \lambda_p\,\psi(x_p + q\,a) + c_q\right)$$ with constants $\lambda_p$, $a$, $n\geq 1$0 and continuous, monotonic $n\geq 1$1, and continuous $n\geq 1$2 (Gleyzer et al., 1 Aug 2025).

3. Constructive Proofs and Smoothness Properties

The original proofs by Kolmogorov and Arnold are constructive, relying on space-filling or Cantor set embeddings to reduce the multivariate domain to a single univariate argument and then reconstruct the original function through the compositional structure (Schmidt-Hieber, 2020, Alesiani et al., 23 Feb 2025). However, the constructed inner functions are generally only continuous and can exhibit highly irregular (non-smooth, even Cantor staircase-like) behavior.

Recent work addresses the inherent irregularity of the outer and inner functions in this decomposition. For instance, a smoothness-preserving variant based on Cantor-set embeddings ensures that if $n\geq 1$3 is Hölder-$n\geq 1$4 smooth, the induced outer function $n\geq 1$5 retains controlled Hölder smoothness, albeit with an exponent reduced as a function of the input dimension. Specifically, for $n\geq 1$6 Hölder-$n\geq 1$7, the induced $n\geq 1$8 is Hölder with exponent $n\geq 1$9 on the Cantor set (Schmidt-Hieber, 2020).

4. Implications for Approximation, Neural Networks, and the Curse of Dimensionality

The Kolmogorov-Arnold Theorem provides an exact representation for all continuous multivariate functions using a fixed number of univariate continuous functions—critically, this number is finite and scales only linearly in input dimension. In contrast to grid-based approximation, which is exponential in $f:[0,1]^n\to\mathbb{R}$0 (the curse of dimensionality), the KST-based constructions reduce the effective parameterization to $f:[0,1]^n\to\mathbb{R}$1 for grid size $f:[0,1]^n\to\mathbb{R}$2 per inner/outer map. For KANs, the sup-norm error on smooth $f:[0,1]^n\to\mathbb{R}$3 using B-spline parameterization of each univariate function is $f:[0,1]^n\to\mathbb{R}$4 for $f:[0,1]^n\to\mathbb{R}$5 times continuously differentiable maps, with no $f:[0,1]^n\to\mathbb{R}$6 in the exponent (Basina et al., 2024, Alesiani et al., 23 Feb 2025).

Deep learning architectures can leverage this result. By replacing classical MLPs with networks whose hidden layers implement sums of trainable univariate functions (e.g., via splines or other bases), one obtains universal approximation with parameter counts that do not exhibit exponential growth in $f:[0,1]^n\to\mathbb{R}$7 (Bhattacharya et al., 2024).

The theorem also provides concrete strategies for approximating both the inner and outer univariate functions—for example, via deep ReLU or sinusoidal networks—offering explicit error bounds and leading to the design of efficient architectures with provable scalability (Gleyzer et al., 1 Aug 2025, Montanelli et al., 2019).

Table: Comparison of Dimensional Scaling

Method	# Terms/Params required	Dependence on $f:[0,1]^n\to\mathbb{R}$8
Grid-based (classical)	$f:[0,1]^n\to\mathbb{R}$9	Exponential
Kolmogorov-Arnold Theorem	$\{\phi_{q, p}\}$0	Linear

Parameter scaling for KST-based decompositions is linear in input dimension, while classical grid methods are exponential.

5. Variants: Sinusoidal, Spline, and Geometric Approximations

Recent developments extend the theorem by restricting the functional forms of the inner and outer maps. "Sinusoidal Approximation Theorem for Kolmogorov-Arnold Networks" proves that both layers can be realized as finite sums of sinusoids with learnable frequencies and amplitudes, and fixed, linearly spaced phases (Gleyzer et al., 1 Aug 2025). The main theorems yield, for any $\{\phi_{q, p}\}$1, an arbitrarily accurate representation using such sinusoidal expansions: $\{\phi_{q, p}\}$2 ensuring that universal approximation capability is preserved under these architectural constraints.

Similarly, KANs are typically instantiated with the inner and outer functions parameterized as B-splines, trained via backpropagation (Basina et al., 2024, Bhattacharya et al., 2024). The "Geometric Kolmogorov-Arnold Superposition Theorem" generalizes the construction to enforce $\{\phi_{q, p}\}$3, $\{\phi_{q, p}\}$4, $\{\phi_{q, p}\}$5, and $\{\phi_{q, p}\}$6 invariance or equivariance, enabling the modeling of physical systems with rigid-motion or permutation symmetries: $\{\phi_{q, p}\}$7 with explicit guarantees for symmetry preservation (Alesiani et al., 23 Feb 2025).

6. Applications in Machine Learning and Scientific Modeling

The Kolmogorov-Arnold decomposition has motivated the design of Kolmogorov-Arnold Networks (KANs), a class of neural architectures that implement the theorem’s superposition structure. Each KAN layer aggregates learnable univariate transformations of input coordinates, summed and passed through outer univariate functions, often parameterized via splines or other bases (Bhattacharya et al., 2024, Basina et al., 2024).

KANs have been successfully applied to:

• Time-series forecasting, including zero-shot domain adaptation scenarios with doubly-residual N-BEATS backbones and adversarial training for invariant representations (Bhattacharya et al., 2024)
• Physical and chemical modeling with built-in geometric or permutation symmetry constraints (Alesiani et al., 23 Feb 2025)
• Scientific data modeling where interpretability of the functional decomposition is as crucial as accuracy

Empirically, KANs demonstrate parameter efficiency and robust scaling laws, outperforming classical MLPs in high-dimensional regimes and symmetry-constrained domains, while maintaining universal approximation guarantees (Alesiani et al., 23 Feb 2025, Basina et al., 2024).

7. Limitations, Open Problems, and Future Directions

Key limitations of the Kolmogorov-Arnold Theorem and its neural instantiations include:

• The high non-uniqueness of the decomposition, as multiple equivalent sets of inner/outer univariate functions can realize the same $\{\phi_{q, p}\}$8 (Alesiani et al., 23 Feb 2025)
• Training complexity—KANs require the optimization of numerous univariate functions, which can pose practical challenges at scale
• Expressivity-efficiency trade-offs: while theoretically universal, practical KAN implementations may truncate the number of terms or basis functions, impacting empirical accuracy depending on task and domain
• Extension to wider classes of group symmetries remains an active research direction (Alesiani et al., 23 Feb 2025)

Ongoing research also investigates the nontrivial relationship between the smoothness properties of $\{\phi_{q, p}\}$9 and those of the induced inner and outer functions, as well as architectural variants that combine the KST-principled design with message-passing, attention, or non-traditional bases (Schmidt-Hieber, 2020, Gleyzer et al., 1 Aug 2025).

---

In summary, the Kolmogorov-Arnold Theorem provides a mathematically rigorous decomposition of continuous multivariate functions into superpositions of continuous univariate maps. This result supplies the theoretical foundation for a new class of neural architectures (KANs) that promise scalability, interpretability, and direct address of the curse of dimensionality, with active research extending these ideas to sinusoidal bases and geometric symmetries (Gleyzer et al., 1 Aug 2025, Basina et al., 2024, Bhattacharya et al., 2024, Alesiani et al., 23 Feb 2025).