Tiny Kolmogorov-Arnold Network (TiKAN) Overview

• TiKAN is a compact Kolmogorov-Arnold network that uses low-rank factorization and spline-based nonlinearities to efficiently transform features.
• It is designed for real-time, resource-constrained applications such as structural defect segmentation, achieving up to 97% parameter reduction compared to traditional models.
• TiKAN integrates adaptively within architectures like KARMA and FORTRESS, selectively deploying its transformations to balance expressivity with minimal computational overhead.

Tiny Kolmogorov-Arnold Network (TiKAN) denotes a parameter-efficient instantiation of a Kolmogorov-Arnold Network (KAN) designed for settings where representational expressivity must be retained under severe parameter and compute constraints. The most explicit architectural definition appears in KARMA, where TiKAN is introduced as a module that leverages low-rank factorization for KAN-based feature transformation inside a semantic segmentation system for structural defect inspection. In that formulation, TiKAN serves as the compact nonlinear representation-learning core of a lightweight segmentation backbone, combining low-rank linear structure, spline-based univariate nonlinearities, shared splines, pruning, and selective deployment across feature scales (Ferdaus et al., 11 Aug 2025).

1. Origin and problem setting

TiKAN emerged from a specific deployment problem: structural defect segmentation in civil infrastructure requires both high segmentation accuracy and extreme parameter/computational efficiency for real-time, resource-constrained deployment. In KARMA, this requirement is framed against the limitation that conventional deep learning models, including CNNs and MLPs, require millions of parameters, which restricts practical use in inspection systems. The architectural opportunity is taken from the Kolmogorov-Arnold theorem, which states that any continuous multivariate function can be represented as compositions of one-dimensional functions; this motivates compact function-composition models rather than conventional dense or convolutional parameterization (Ferdaus et al., 11 Aug 2025).

Within this context, TiKAN is not presented as a generic synonym for “small KAN,” but as a concrete design goal: a parameter-efficient instantiation of KAN by low-rank constraints, tailored for segmentation within the KARMA framework. A related but distinct use appears in FORTRESS, where adaptive TiKAN integration selectively applies function composition transformations only when computationally beneficial. This suggests that, across the literature, TiKAN is best understood as a family of compact KAN design strategies rather than a single universally fixed blueprint (Thrainer et al., 16 Jul 2025).

2. Mathematical basis

TiKAN inherits its theoretical motivation from the Kolmogorov-Arnold representation theorem. In the KARMA formulation, the theorem is written as

$$f(x_1,\dots,x_n) = \sum_{q=0}^{2n} \Phi_q\left( \sum_{p=1}^n \psi_p(x_p) + c_q \right),$$

where $f$ is any continuous function and the representation is decomposed into sums and compositions of continuous univariate functions $\psi_p$ and $\Phi_q$ (Ferdaus et al., 11 Aug 2025).

The original KAN program operationalizes this idea by replacing scalar edge weights with learnable univariate functions, typically spline-parameterized, while nodes perform summation. In that setting, a KAN layer is written as

$x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$

and an individual activation can be parameterized as

$\phi(x)=w_b b(x)+w_s\sum_i c_i B_i(x),$

with a base activation and a spline term (Liu et al., 2024).

TiKAN modifies this general KAN pattern for compactness. In KARMA, its principle is to approximate feature transforms as compositions of base (linear, low-rank) and non-linear (spline-shaped, learnable) 1D functions, balancing expressiveness and efficiency. For a feature map $F_l$, the module performs

$\hat{F}_l = \psi_l(F_l; \Theta_l),$

with core transformation

$$\psi_l(F_l) = \phi_{base}(F_l) \cdot s_{base} + \phi_{spline}(F_l) \cdot s_{spline},$$

where $s_{base}$ and $f$0 are learnable scaling factors (Ferdaus et al., 11 Aug 2025).

3. Module architecture in KARMA

In KARMA, TiKAN modules are strategically integrated at multiple FPN feature levels inside an adaptive feature pyramid backbone. The low-rank base transformation is

$f$1

where $f$2 and $f$3 are low-rank factorized weights of rank $f$4, $f$5 is a bias term, and $f$6 is SiLU activation. The stated purpose is to provide the main linear pathway for feature transformation with parameter complexity reduced from $f$7 to $f$8 (Ferdaus et al., 11 Aug 2025).

The spline-based nonlinear branch is

$f$9

where $\psi_p$0 and $\psi_p$1 are low-rank factorized spline parameters, $\psi_p$2 is spline factor rank, $\psi_p$3 is grid size, and $\psi_p$4 is spline order. The implementation initializes grid points uniformly in $\psi_p$5, sets spline order to $\psi_p$6 for cubic interpolation, and uses shared splines across channels for efficiency when appropriate (Ferdaus et al., 11 Aug 2025).

The module’s compactness depends on several explicit mechanisms. Both the base and spline pathways are low-rank factorized; shared spline functions are reused across channels; pruning is applied during training according to

$\psi_p$7

and gradient clipping is used to ensure stable optimization (Ferdaus et al., 11 Aug 2025).

KARMA also places a specialized TiKAN Enhancement Module at the deepest level $\psi_p$8: $\psi_p$9 Here, $\Phi_q$0 is PatchEmbed, $\Phi_q$1 is a KANBlock, and $\Phi_q$2 is Unpatchify. The KANBlock wraps a KANLayer built from two KANLinear operations plus depthwise separable convolution, batch normalization, and ReLU, together with normalization and residual structure. The KANLinear operation is

$\Phi_q$3

A $\Phi_q$4 convolution then performs channel reduction after TiKAN enhancement (Ferdaus et al., 11 Aug 2025).

4. Functional role and empirical evidence

Within KARMA, TiKAN is described as the central representation learning engine in the AFPN backbone. Its function is to inject strong nonlinear expressivity where needed, especially at bottlenecks and high-level features, without growing parameter count, and to isolate efficient Kolmogorov-Arnold representation learning as a module that can be flexibly applied at multiple scales in the FPN. The reported outcome is that KARMA can outperform prior KAN-based methods such as U-KAN and rival or beat state-of-the-art CNNs and Transformers with 1–3% of their parameters (Ferdaus et al., 11 Aug 2025).

The principal efficiency figures reported for KARMA are 0.959M parameters and 0.264 GFLOPS. The baseline comparisons listed in the same study are U-Net with 31.04M parameters, FPN with 21M, and U-KAN with 25.36M. The reduction is described as approximately 97% fewer parameters and 95–99% less computational cost than leading methods. On the CSDD dataset, KARMA with TiKAN reports $\Phi_q$5 (without background) of 0.731 and $\Phi_q$6 (without background) of 0.838, while U-Net and U-KAN each report $\Phi_q$7 0.729 and $\Phi_q$8 0.836, with substantially larger parameter counts. On the S2DS dataset, the paper reports similar trends, with competitive to better accuracy at a minute fraction of the cost (Ferdaus et al., 11 Aug 2025).

The ablation evidence is central to the interpretation of TiKAN. A KARMA variant using FPN without TiKAN has 13.58M parameters and $\Phi_q$9 0.734 on CSDD, whereas KARMA with TiKAN has 0.959M parameters and $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$0 0.731. A further ablation, KARMA without low-rank adaptation, has 5.57M parameters with unchanged $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$1. These results are used to support two claims: low-rank adaptation gives a major efficiency gain with negligible performance change, and removing TiKAN causes parameter count to increase by roughly $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$2 for essentially the same results (Ferdaus et al., 11 Aug 2025).

The practical deployment numbers are equally explicit. KARMA inference is reported at 12.8 ms per image for $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$3 inputs, 78 FPS, and 432MB memory, versus U-Net at 45 ms, 22 FPS, and 1.8GB. KARMA-Flash is reported at 8.9 ms, 112 FPS, and 298MB; KARMA-High at 24 ms, 41 FPS, and 789MB. All TiKAN KARMA variants are stated to exceed the 30 FPS real-time threshold and to run on small or edge GPUs. The source code is given at https://github.com/faeyelab/karma (Ferdaus et al., 11 Aug 2025).

5. Adaptive integration and related compact KAN directions

FORTRESS provides a second explicit TiKAN formulation. In that architecture, TiKAN is wrapped within KANDoubleConv blocks in an encoder-decoder model and is invoked conditionally: $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$4 This criterion activates TiKAN only when the feature map has sufficient channels and limited spatial resolution. The TiKAN transformation itself is written as

$x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$5

with learnable spline functions and optional depthwise convolution enhancement (Thrainer et al., 16 Jul 2025).

FORTRESS reports 2.89M parameters, 1.17 GFLOPs, $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$6-score 0.771, and $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$7 0.677, compared with U-Net at 31.04M parameters and 13.69 GFLOPs, SA-UNet at 7.86M and 3.62 GFLOPs, and U-KAN at 25.36M and 1.73 GFLOPs. The stated reductions are 91% in parameters and 91% in computational complexity relative to the 31M-parameter baseline, together with a $x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$8 inference speed improvement. FORTRESS therefore supports the view that TiKAN-style modules can be integrated selectively rather than uniformly across all stages (Thrainer et al., 16 Jul 2025).

Related research pursues similar compact-KAN objectives through other mechanisms. LArctan-SKAN introduces single-parameterized KAN variants based on learnable trigonometric functions, with LArctan-SKAN defined by

$x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),$9

On MNIST, it is reported to outperform all pure KAN variants compared, including FourierKAN, LSS-SKAN, and Spl-KAN, while also improving training speed by 535.01% and 49.55% compared to MLP+rKAN and MLP+fKAN, respectively (Chen et al., 2024).

At the hardware level, KANELÉ treats compact KAN deployment as a lookup-table design problem. It maps each learned spline activation directly to an FPGA LUT, co-optimizes quantization and pruning during training, and reports sub-1k-LUT models as competitive across scientific, tabular, and real-world benchmarks. For control applications, it reports quantized KAN policies with approximately $\phi(x)=w_b b(x)+w_s\sum_i c_i B_i(x),$0 fewer parameters than MLPs and 4.5ns latency, while more generally claiming up to a 2700x speedup and orders-of-magnitude resource savings relative to prior KAN-on-FPGA approaches (Hoang et al., 14 Dec 2025).

6. Training behavior, interpretation, and limitations

TiKAN inherits the strengths and liabilities of the broader KAN family. A comparative study of KAN training finds that KANs are an effective alternative to MLP architectures on high-dimensional datasets and have somewhat better parameter efficiency, but suffer from more unstable training dynamics. The same study states that KANs are significantly more sensitive than MLPs to initialization scheme, optimizer, and learning rate, and that overfitting is much more pronounced in KANs. In small or shallow regimes explicitly associated with “Tiny” KANs or TiKANs, the paper reports comparable or slightly better efficiency scores versus MLPs, but still observes high sensitivity to training scheme and overfitting (Sohail, 2024).

The recommendations given there are specific: Kaiming Normal initialization, Adam as optimizer, low learning rate $\phi(x)=w_b b(x)+w_s\sum_i c_i B_i(x),$1, GeLU activation for KA units, and early stopping. The same source argues that increasing B-spline degree for a fixed parameter count does not substitute for increasing width or layers, because extra degree leads to earlier overfitting. For TiKAN-like models, this makes compactness a multidimensional design problem involving rank, spline complexity, placement, and training stability rather than parameter count alone (Sohail, 2024).

A recurrent misconception is that TiKAN is simply a smaller copy of a vanilla KAN. The available evidence does not support that simplification. In KARMA, TiKAN is tiny because KANLinear is made tiny by carefully controlled hidden dimensions and ranks, shared splines, pruning, and low-rank factorization; in FORTRESS, it is tiny because it is adaptively activated only on selected feature maps; in KANELÉ, compactness is achieved through quantization, pruning, and direct LUT mapping. This suggests that TiKAN is best regarded as a design regime for compact Kolmogorov-Arnold representation learning rather than a single canonical layer recipe (Ferdaus et al., 11 Aug 2025).

The broader theoretical backdrop also warrants caution. Reviews of KANs emphasize their grounding in the Kolmogorov-Arnold representation theorem and their use of trainable univariate functions on edges instead of fixed node activations (Basina et al., 2024). At the same time, a re-examination of the Kolmogorov-Arnold representation theorem argues that the theorem is more naturally interpreted as a deep neural network where most of the layers are required to approximate the interior function, rather than as a direct prescription for shallow architectures. A plausible implication is that practical TiKAN systems succeed not by literal transcription of the theorem, but by combining the theorem’s function-composition intuition with aggressive architectural constraints tailored to deployment requirements (Schmidt-Hieber, 2020).