Kolmogorov-Arnold Autoencoder (KAE)

• KAE is an autoencoder that replaces fixed nonlinear activations with learnable univariate functions on connections, enhancing model expressivity.
• It spans variants like spline, polynomial, and SHM KAEs, each tailored for tasks such as image reconstruction, denoising, and guided-wave structural monitoring.
• Empirical results show competitive performance relative to CNN autoencoders, though with trade-offs in parameter efficiency and interpretability.

Kolmogorov-Arnold Autoencoder (KAE) denotes an autoencoder in which encoder and decoder mappings are implemented with Kolmogorov-Arnold Network (KAN) layers rather than ordinary MLP blocks, and in some variants rather than convolutional layers. The defining architectural move is to place learnable univariate functions on connections, not fixed nonlinearities on neurons. In the current literature, this idea appears in a spline-based image autoencoder benchmarked against CNN autoencoders, a polynomial KAN autoencoder for reconstruction, retrieval, classification, and denoising, and an unsupervised guided-wave structural health monitoring model that reconstructs virtual baseline signals directly from raw measurements (Moradi et al., 2024, Yu et al., 2024, Liao et al., 1 Aug 2025).

1. Definition and terminological scope

In the autoencoder setting, the basic problem is unsupervised representation learning. Given an input $x$, an encoder produces a latent vector $z$, a decoder reconstructs $\hat{x}$, and training minimizes reconstruction error: $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$ The KAE literature considered here uses mean squared error: $\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$ What changes is the parameterization of $f_\theta$ and $g_\phi$: instead of standard linear layers followed by fixed activations, KAE uses KAN-style mappings with learnable one-dimensional functions on edges (Moradi et al., 2024, Yu et al., 2024).

The operational distinction is explicit. In a standard MLP layer,

$y=\sigma(Wx+b),$

where $W$ and $b$ are learned and $z$0 is fixed. In a KAN, each connection from input coordinate $z$1 to output unit $z$2 carries a learnable univariate transform $z$3, and a layer can be written schematically as

$z$4

The KAE papers interpret this as a shift from “weights only” to “functions on connections,” with consequences for expressivity, parameter count, and potentially interpretability (Moradi et al., 2024, Yu et al., 2024).

A recurrent source of confusion is the acronym itself. In a separate kernel-methods literature, “KAE” denotes “Kernel Autoencoder,” not “Kolmogorov-Arnold Autoencoder.” That earlier use concerns vv-RKHSs, operator-valued kernels, and representer theorems, and is unrelated to KAN-based edge-function architectures (Laforgue et al., 2018).

2. Kolmogorov-Arnold motivation and what it does—and does not—establish

The KAE literature grounds KANs in the Kolmogorov-Arnold representation theorem. One form used in the image-autoencoder work is

$z$5

and the polynomial KAE paper uses the closely related statement

$z$6

These forms motivate KANs as networks composed of learnable univariate functions. In both autoencoder papers, the theorem is used as architectural justification for replacing node activations with trainable edge functions; neither paper proves a new theorem specific to autoencoders (Moradi et al., 2024, Yu et al., 2024).

This theorem-based motivation is materially qualified by the analysis in “The Kolmogorov-Arnold representation theorem revisited” (Schmidt-Hieber, 2020). That paper argues that the classical theorem is a poor direct justification for practical shallow “KA-style” architectures because the outer function can depend very badly on the target function and can become highly irregular even when the target is smooth. It further argues that the more natural neural interpretation is not a shallow two-hidden-layer template but a deep architecture in which most layers are required to approximate the interior function. For KAE, this sharply limits any claim that the theorem by itself validates a shallow encoder-decoder construction.

The same revisited analysis also provides a more constructive perspective. It derives modified KA representations based on Cantor-set and space-filling constructions that transfer Hölder smoothness of the represented function to the outer function for $z$7, while showing that the interior map is the difficult part and is more naturally implemented by a deep hierarchical network (Schmidt-Hieber, 2020). This suggests that KAE should be read as theorem-inspired rather than theorem-proved, and that any strong theoretical reading must distinguish between classical KA existence results and trainable autoencoder parameterizations.

3. Architectural realizations and function families

The current KAE literature contains at least three distinct realizations.

Variant	Core layer family	Representative use
Spline KAE	B-spline KAN, Efficient KAN	Image autoencoding against CNN baselines
Polynomial KAE	Learnable polynomial edge functions	Reconstruction, retrieval, classification, denoising
SHM KAE	KAN-based encoder-decoder with learnable functional mappings	Guided-wave damage detection and localization

The spline-based image KAE is intentionally simple and asymmetric. Its encoder consists of a KAN layer, a ReLU activation, and a dense layer; its decoder mirrors this with a dense layer, a ReLU, and a KAN layer. For flattened images, the KAN maps $z$8 and the dense layer maps $z$9. The paper gives the MNIST example

$\hat{x}$0

for the encoder and

$\hat{x}$1

for the decoder. The KAN implementation is spline-based, with each univariate edge function represented as a linear combination of B-spline basis functions: $\hat{x}$2 The paper uses Efficient KAN rather than the original KAN implementation. The stated reason is that the original implementation expands intermediate tensors to shape $\hat{x}$3, whereas Efficient KAN evaluates basis functions on inputs and linearly combines them afterward, making computation closer to matrix multiplication and reducing memory usage. The regularization also changes from an L1 penalty defined on input samples to standard L1 regularization on weights (Moradi et al., 2024).

The polynomial KAE replaces spline, Fourier, and wavelet function families with a polynomial layer: $\hat{x}$4 Here $\hat{x}$5 are learnable coefficient matrices, $\hat{x}$6 is a bias, and $\hat{x}$7 is the outer activation; in experiments $\hat{x}$8 is sigmoid. This architecture is fully KAN-based in the paper’s proposed design: both encoder and decoder are composed of KAN/KAE layers, with no dense MLP layers required in the main variant. The experimental model is shallow, with structure $\hat{x}$9, and latent dimensions $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$0 and $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$1. The paper explicitly contrasts this with standard KAN using B-splines, FourierKAN using Fourier functions, and WavKAN using wavelet functions, and argues that polynomial functions are more stable and effective for reconstruction and inverse-like encoder/decoder behavior (Yu et al., 2024).

The structural health monitoring KAE uses a standard encoder-bottleneck-decoder form for raw guided-wave signals. The network widths are given explicitly as encoder $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$2 and decoder $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$3, so the latent dimension is $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$4. The manuscript describes the KAN foundation in terms of learnable activation functions and spline functions instead of traditional weight parameters, but does not provide a full layerwise spline expansion formula or otherwise fully specify the exact KAN basis-function parameterization. Its practical role is different from the image works: the model reconstructs a virtual pristine baseline signal from a measured signal, and the reconstruction error becomes a damage index for subsequent localization (Liao et al., 1 Aug 2025).

4. Training objectives, optimization, and evaluation protocols

Across the reported KAE variants, the training objective is deterministic reconstruction rather than probabilistic latent modeling. The polynomial KAE paper is explicit that the model is not a VAE and does not include stochastic latent variables, KL regularization, adversarial objectives, contractive penalties, or a dedicated denoising-autoencoder training loss. Denoising is evaluated by applying noise to inputs and using the trained autoencoder to reconstruct clean structure, while the training loss remains MSE (Yu et al., 2024).

The spline-based image KAE is trained on MNIST, SVHN, and CIFAR-10. Both AE-KAN and AE-CNN were trained under identical conditions: 10 epochs, AdamW, learning rate $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$5, weight decay $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$6, and MSE reconstruction loss. Evaluation used test-set reconstruction loss and the quality of latent representations measured by a KNN classifier on latent codes, reporting Accuracy and F1-score. The paper also performed a bottleneck-size sweep for KAE, with best bottleneck sizes reported as 150 for MNIST, 500 for CIFAR-10, and 500 for SVHN; 50 was also reasonable for MNIST, 200 also very good for CIFAR-10, and 200 also good for SVHN. As bottleneck size increases, reconstructions become clearer and more defined, while smaller bottlenecks produce more blurred outputs (Moradi et al., 2024).

The polynomial KAE is evaluated on MNIST, FashionMNIST, CIFAR10, and CIFAR100. Training details are concrete: Adam optimizer, learning rates $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$7 and $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$8, weight decay $$z=f_\theta(x), \qquad \hat{x}=g_\phi(z), \qquad \min_{\theta,\phi}\mathcal{L}(x,\hat{x}).$$9 and $\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$0, batch size 256, 10 epochs, sigmoid activation, polynomial orders $\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$1, and 10 runs with random seeds 2024 to 2033; reported values are mean $\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$2 standard deviation of the best hyperparameter configuration. Tasks include reconstruction, latent-space similarity search using Recall 10@$\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$3, nearest-neighbor classification in latent space, and denoising under Gaussian noise

$\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$4

and salt-and-pepper noise with probability $\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$5 of pixels being set to 0 or 1 (Yu et al., 2024).

In the structural health monitoring system, training is healthy-only. Raw guided-wave signals are normalized to $\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$6, and KAE is trained directly on those normalized raw signals without prior feature extraction. The reconstruction objective is

$\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$7

The path-wise reconstruction error is then used as damage index: $\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$8 Regional damage detection uses a 95th-percentile health index,

$\mathcal{L}(x,\hat{x})=\|x-\hat{x}\|^2.$9

with threshold calibrated from pristine data,

$f_\theta$0

The localization map is produced by MRAPID through

$f_\theta$1

The KAE is implemented in PyTorch and, for the wind turbine blade case, uses Adam, learning rate $f_\theta$2, weight decay $f_\theta$3, gamma $f_\theta$4, and 200 epochs; the composite flat plate case also uses 200 epochs, with batch size $f_\theta$5 (Liao et al., 1 Aug 2025).

5. Reported empirical performance

On image reconstruction and latent-space evaluation, the spline-based KAE gives mixed but nontrivial results against CNN autoencoders. On MNIST at small bottlenecks, AE-KAN outperforms AE-CNN. For example, AE-CNN with bottleneck $f_\theta$6 reports 617 parameters, time 39.24, loss 0.2667, accuracy 0.74, and F1 0.74, whereas AE-KAN with bottleneck 4 reports 62,796 parameters, time 40.82, loss 0.2170, accuracy 0.76, and F1 0.75. At the largest tested MNIST bottleneck, however, the ranking reverses: AE-CNN with bottleneck $f_\theta$7 reports loss 0.1722, accuracy 0.90, and F1 0.90, whereas AE-KAN with bottleneck 18 reports loss 0.2238, accuracy 0.78, and F1 0.77. On SVHN, KAE consistently gives lower reconstruction loss than CNN AE, with losses 0.0129, 0.0089, and 0.0082 for AE-KAN against 0.0251, 0.0167, and 0.0114 for AE-CNN across the tested bottlenecks. On CIFAR-10, KAE also consistently beats the CNN baseline in reconstruction loss, with 0.0497, 0.0402, and 0.0341 for AE-KAN against 0.0659, 0.0523, and 0.0414 for AE-CNN, while using about $f_\theta$8 parameters rather than roughly $f_\theta$9. The paper’s own interpretation is that KAE is viable and competitive but far less parameter-efficient than the CNN baselines (Moradi et al., 2024).

The polynomial KAE reports broader and stronger gains over standard AE and other KAN variants. In reconstruction, $g_\phi$0 is usually best. On MNIST, AE reports 0.056 at latent 16 and 0.043 at latent 32, while KAE $g_\phi$1 reports 0.024 and 0.015; on FashionMNIST, AE reports 0.045 and 0.034, while KAE $g_\phi$2 reports 0.018 and 0.015; on CIFAR10, AE reports 0.034 and 0.029, while KAE $g_\phi$3 reports 0.016 and 0.012; on CIFAR100, AE reports 0.037 and 0.030, while KAE $g_\phi$4 reports 0.016 and 0.012. Retrieval results favor $g_\phi$5: for MNIST, Recall values increase from 0.354 to 0.596 at latent 16 and from 0.483 to 0.689 at latent 32; analogous gains are reported on FashionMNIST, CIFAR10, and CIFAR100. For nearest-neighbor classification, $g_\phi$6 is best overall: MNIST rises from 0.853 to 0.940 at latent 16 and from 0.916 to 0.964 at latent 32; CIFAR10 rises from 0.243 to 0.315 at latent 16 and from 0.283 to 0.354 at latent 32. Denoising improves under both Gaussian and salt-and-pepper noise, with the paper summarizing Gaussian absolute MSE reductions of about $g_\phi$7 to $g_\phi$8 and salt-and-pepper reductions of about $g_\phi$9 to $y=\sigma(Wx+b),$0. Capacity analysis on MNIST with $y=\sigma(Wx+b),$1 reports that KAE $y=\sigma(Wx+b),$2 uses around 75K–101K parameters, KAN/FourierKAN use around 250K–251K, training time is around 30–36 seconds, and KAE reaches higher classification accuracy with fewer parameters than those KAN-family baselines (Yu et al., 2024).

The structural health monitoring KAE is evaluated differently because its task is not image representation but baseline-free damage detection and localization. For the wind turbine blade, average healthy-state reconstruction error is reported as below $y=\sigma(Wx+b),$3 across all regions. The method is reported to detect and localize single damage, detect and localize two simultaneous damages, and retain localization accuracy as damage size decreases from 35 mm to 25 mm to 15 mm, although DI values decrease and the density shifts left. In comparisons against RAPID, Modified RAPID, and CAE-RAPID, the proposed KAE-MRAPID reports RMSE 14.2 mm, MRE 5.51%, and time 66.19 s, compared with RAPID at RMSE 42.43 mm and MRE 29.70%, Modified RAPID at RMSE 17.46 mm, MRE 9.57%, and time 109.22 s, and CAE-RAPID at RMSE 27.87 mm, MRE 12.03%, and time 53.72 s. An ablation-like comparison further reports CAE-RAPID at RMSE 27.87 mm, MAPE 11.99%, MRE 12.03%; KAE-RAPID at RMSE 26.41 mm, MAPE 11.26%, MRE 11.33%; KAE-MRAPID at RMSE 14.2 mm, MAPE 5.96%, MRE 5.51%; and CAE-MRAPID at RMSE 25.43 mm, MAPE 13.71%, MRE 11.92%. The paper interprets this as evidence that KAE reconstructs virtual baselines better and produces more robust damage indices (Liao et al., 1 Aug 2025).

6. Limitations, misconceptions, and open research questions

Several limitations recur across the literature. The image KAE benchmark is explicit that AE-KAN includes two extra dense layers compared to AE-CNN, so the comparison is not a pure “swap CNN for KAN” under matched parameterization. It is also explicit that parameter count is vastly larger: about 63k parameters on MNIST versus fewer than 1k for CNN AE, about 1M on SVHN versus thousands to a few tens of thousands, and about 2M on CIFAR versus about 7k. The same paper also notes that interpretability is asserted, not measured, and that experiments are restricted to image datasets, with applicability to text, time series, and other modalities unexplored (Moradi et al., 2024).

The polynomial KAE addresses some of the parameter-efficiency problem relative to spline, Fourier, and wavelet KAN baselines, but its own scope is also limited. The experiments are on a shallow three-layer autoencoder; scaling to deeper architectures is future work. The benchmarks are MNIST, FashionMNIST, CIFAR10, and CIFAR100 rather than large-scale modern representation-learning benchmarks. The paper does not provide a formal theorem guaranteeing invertibility or reconstruction stability, does not compare against convolutional autoencoders, VAEs, or more modern self-supervised methods, and does not deeply inspect the learned polynomial edge functions. It also stresses that KAN performance is highly dependent on the chosen function class, which means that “replace MLP with KAN” is not a sufficient design rule (Yu et al., 2024).

The structural health monitoring KAE introduces a different misconception. “Baseline-free” in that paper does not mean absence of healthy reference information; it means no need for a stored baseline signal for every sensing path during operation, no damaged data for training, and no manual DI engineering. Healthy-state data are still required to train KAE and calibrate $y=\sigma(Wx+b),$4. The paper also states that the method is not suitable for damage quantification, does not address temperature effects, describes offline training rather than an explicit online update mechanism, and uses simulated rather than real damage in the wind-turbine-blade study (Liao et al., 1 Aug 2025).

A broader theoretical misconception is that the Kolmogorov-Arnold theorem directly validates practical shallow KAE architectures. The revisited theorem literature argues against that reading: classical outer functions may be highly irregular, exact interior maps are discontinuous or fractal, the most natural neural interpretation is deep rather than shallow, and the latent geometry is not ordinary Euclidean structure but can be Cantor-like (Schmidt-Hieber, 2020). This suggests that theorem-based claims for KAE should be made cautiously.

Taken together, the reported studies establish that KAE is a real and technically heterogeneous design family rather than a single canonical model. The common thread is the replacement of ordinary affine-plus-fixed-activation blocks by learnable functional mappings on edges. The main unresolved questions stated or implied in the literature are whether KANs still win under matched parameter budgets, whether interpretability can be quantified rather than asserted, how KAE compares to stronger MLP and CNN autoencoder baselines, how deeper architectures behave, how robustly the approach transfers beyond moderate-scale image benchmarks, and, in structural health monitoring, how the framework behaves under temperature variation and other operational covariates (Moradi et al., 2024, Yu et al., 2024, Liao et al., 1 Aug 2025).