Self-Optimizing Kolmogorov-Arnold Network

• SO-KAN is a network framework that self-optimizes its architecture and synaptic nonlinearities, enabling automated design and improved interpretability.
• It employs genetic algorithms, differentiable gating, and hardware-in-the-loop tuning to automatically set parameters like depth, connectivity, and spline resolution.
• Empirical results on benchmarks and physical systems show that SO-KAN reduces parameter count while maintaining or enhancing accuracy and efficiency.

Searching arXiv for the cited SO-KAN/KAN optimization papers to ground the article in current preprints. Self-Optimizing Kolmogorov-Arnold Network (SO-KAN) denotes a class of Kolmogorov-Arnold Network formulations in which architectural degrees of freedom or synaptic nonlinearities are optimized automatically rather than fixed by manual design. In the recent literature, the label covers at least three distinct realizations: GA-KAN, which automates KAN design with a genetic algorithm for classification; overprovisioned KANs with differentiable sparsification, which treat architecture search as an end-to-end optimization problem under a minimum-description-length objective; and physical KANs built from Synaptic Nonlinear Elements (SYNEs), which adapt trainable nonlinear device responses in closed loop with a differentiable digital twin (Long et al., 29 Jan 2025, Bagrow et al., 13 Dec 2025, Taglietti et al., 20 Jan 2026).

1. Mathematical and architectural basis

KANs were introduced to address the issue of interpretability in multilayer perceptrons. Their construction is motivated by the Kolmogorov-Arnold representation theorem, stated in one of the cited formulations as

$$f(x_1,\dots,x_n)=\sum_{q=1}^{2n+1}\Phi_q\!\Bigl(\sum_{p=1}^n \phi_{q,p}(x_p)\Bigr),$$

for continuous $f:[0,1]^n\to\mathbb R$, where the $\phi_{q,p}$ and $\Phi_q$ are univariate functions. A relaxed KAN replaces the fixed $2n+1$ width by arbitrary layers and widths,

$$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$

with each $\Phi_l$ an $n_{l+1}\times n_l$ matrix of univariate activation functions $\phi_{l,j,i}$ (Long et al., 29 Jan 2025).

The architectural distinction from a conventional MLP is that KANs place the nonlinear transformation on edges rather than at neurons. In the spline-based formulation, each edge activation is

$$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$

where $f:[0,1]^n\to\mathbb R$0 is the number of spline grid intervals, $f:[0,1]^n\to\mathbb R$1 is fixed, $f:[0,1]^n\to\mathbb R$2 is a fixed basis, and $f:[0,1]^n\to\mathbb R$3 are trainable. In the physical realization, the same functional role is assigned to device-native synaptic nonlinearities $f:[0,1]^n\to\mathbb R$4 and outer functions $f:[0,1]^n\to\mathbb R$5, with linear summation at neurons and trainable nonlinear response curves at synapses (Long et al., 29 Jan 2025, Taglietti et al., 20 Jan 2026).

This shared mathematical template makes “self-optimization” meaningful at multiple levels. It can refer to automatic selection of depth, width, connectivity, and spline resolution; to learned binary gates on edges and summation nodes; or to continual retuning of physical nonlinear transfer curves. A plausible implication is that SO-KAN is better understood as an optimization paradigm for KANs than as a single canonical architecture.

2. Principal realizations of self-optimization

The current literature presents several non-equivalent mechanisms for making KANs self-optimizing.

Realization	Optimized objects	Optimization mechanism
GA-KAN	Connectivity bits, effective depth, grid parameter $f:[0,1]^n\to\mathbb R$6	Genetic algorithm, sparse decoding, validation-loss fitness
Differentiable SO-KAN	KAN parameters $f:[0,1]^n\to\mathbb R$7, edge gates, node gates	Hard-concrete relaxation, MDL-style complexity penalty, gradient training
Physical SO-KAN	$f:[0,1]^n\to\mathbb R$8, $f:[0,1]^n\to\mathbb R$9, input range, gain $\phi_{q,p}$0, device count	Differentiable digital twin, hardware-in-the-loop reprogramming, adaptive pruning

GA-KAN is framed as an automated optimization approach that requires no human intervention in the design process. Its core contributions are automated optimization, a new encoding strategy, and a new decoding process, with the aim of improving accuracy and interpretability while reducing the number of parameters (Long et al., 29 Jan 2025).

Bagrow and Bongard’s formulation studies overprovisioned architectures combined with sparsification to learn compact, interpretable KANs without sacrificing accuracy. The defining step is differentiable sparsification, which turns architecture search into an end-to-end optimization problem. Overprovisioning and sparsification are reported as synergistic, with the combination outperforming either alone (Bagrow et al., 13 Dec 2025).

The physical realization extends the self-optimizing idea into hardware. There, the network does not merely prune software parameters; it trains the synaptic nonlinearity itself, using SYNE devices whose voltage-current response curves are tuned during learning. The paper characterizes this as learning nonlinear heterogeneity in physical KANs, with closed-loop adaptation to the underlying device physics (Taglietti et al., 20 Jan 2026).

A common misconception is that SO-KAN names a single training rule. The literature instead uses the term for several families of automatic adaptation mechanisms layered onto the KAN formalism.

3. Genetic architecture search and sparse decoding

In GA-KAN, the search space is encoded directly into a chromosome. A maximal depth $\phi_{q,p}$1 and maximal neurons per layer $\phi_{q,p}$2 are fixed, with layer sizes labeled $\phi_{q,p}$3 and $\phi_{q,p}$4. For each layer $\phi_{q,p}$5, the presence or absence of each of the $\phi_{q,p}$6 connections is encoded by a bit $\phi_{q,p}$7. The spline grid parameter $\phi_{q,p}$8 is encoded in $\phi_{q,p}$9 bits, and the actual depth is encoded in $\Phi_q$0 bits representing depth minus one. The total chromosome length is

$\Phi_q$1

This directly couples neural architecture search to KAN-specific spline resolution (Long et al., 29 Jan 2025).

The decoding process enforces sparsity and variable depth. The depth bits are applied as zero masks that zero out all $\Phi_q$2 bits in layers beyond the chosen depth. If, after masking, a layer’s bits are all zero, the layer is dropped altogether by a degradation mechanism. The resulting network is therefore a sparse KAN of depth at most $\Phi_q$3, with only those edges retained for which $\Phi_q$4. The final step reads the 6 grid bits, maps them to $\Phi_q$5, and builds each edge activation’s spline with $\Phi_q$6 intervals (Long et al., 29 Jan 2025).

Fitness evaluation is not purely structural. Each decoded individual is trained on $\Phi_q$7 for $\Phi_q$8 epochs with LBFGS, while the minimal validation loss is tracked:

$\Phi_q$9

Invalid networks with no path from input to output are assigned $2n+1$0, and the GA minimizes this loss. The reported GA parameters are population size $2n+1$1, generations $2n+1$2, crossover rate $2n+1$3, and mutation rate $2n+1$4 (Long et al., 29 Jan 2025).

This scheme realizes self-optimization by discrete search rather than differentiable relaxation. It automates connectivity, effective depth, and spline granularity, while retaining the original KAN emphasis on symbolic interpretability.

4. Differentiable sparsification and MDL-style optimization

A second SO-KAN formulation replaces evolutionary search with differentiable gating. The objective is to learn both the KAN parameters $2n+1$5 and binary gates on every edge and summation node so as to minimize prediction error plus a complexity penalty. In summary form, training minimizes

$2n+1$6

where the expected gate count functions as a model-complexity term, and with $2n+1$7 this quantity is the expected number of open gates (Bagrow et al., 13 Dec 2025).

Hard binary gates are replaced by a hard-concrete relaxation. For each gate,

$2n+1$8

$2n+1$9

with fixed $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$0, $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$1, and $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$2. The expected open-gate probability is

$$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$3

At test time the method thresholds deterministically,

$$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$4

so every gate is exactly $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$5 or $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$6, preserving interpretability (Bagrow et al., 13 Dec 2025).

The gating mechanism is coupled to heavy overprovisioning. A trunk KAN with layer sizes such as $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$7 is augmented with DenseNet-style forward connections, so that every layer $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$8 sees the concatenation of $$\mathrm{KAN}(\mathbf x)=\bigl(\Phi_{L-1}\circ\cdots\circ\Phi_1\circ\Phi_0\bigr)(\mathbf x),$$9. Every such edge and node carries its own gate. Training initializes gate logits at $\Phi_l$0, uses Adam with learning rate $\Phi_l$1, and may include a warm-up phase with $\Phi_l$2 before sparsity is imposed. Early stopping is triggered if the fraction of gates decisively in $\Phi_l$3 exceeds $\Phi_l$4 (Bagrow et al., 13 Dec 2025).

This realization makes architecture search continuous and end to end. Depth, width, and connectivity emerge from thresholded gate configurations rather than from an explicit chromosome. The single hyperparameter $\Phi_l$5 then governs the error-size trade-off within a principled MDL framework.

5. Hardware-native SO-KAN and SYNE devices

The physical realization of SO-KAN moves self-optimization into the substrate. SYNE-based KANs operate at room temperature, $\Phi_l$6–$\Phi_l$7 microampere currents, and $\Phi_l$8 MHz speeds, with no observed degradation over $\Phi_l$9 measurements and months-long timescales. Instead of training linear synaptic weights while treating device nonlinearities as fixed, the method trains the synaptic nonlinearity itself (Taglietti et al., 20 Jan 2026).

In the SYNE implementation, each synapse is realized as a combination of device responses. For a given synapse $n_{l+1}\times n_l$0, the input voltage $n_{l+1}\times n_l$1 is applied to constituent SYNE devices. Each device has independent tuning voltages $n_{l+1}\times n_l$2 and $n_{l+1}\times n_l$3 that shape its $n_{l+1}\times n_l$4–$n_{l+1}\times n_l$5 curve, and each measured output current $n_{l+1}\times n_l$6 is scaled by a trainable gain $n_{l+1}\times n_l$7. The synapse output is approximated as

$n_{l+1}\times n_l$8

with outer functions $n_{l+1}\times n_l$9 realized similarly or by a small MLP in the digital twin (Taglietti et al., 20 Jan 2026).

Training proceeds through a differentiable digital twin: a data-driven MLP with three hidden layers and ReLU maps $\phi_{l,j,i}$0 to $\phi_{l,j,i}$1, with training MSE $\phi_{l,j,i}$2 on held-out SYNE data. Each SYNE device has five trainable parameters: $\phi_{l,j,i}$3 and $\phi_{l,j,i}$4 in $\phi_{l,j,i}$5, the learned input range $\phi_{l,j,i}$6, and output gain $\phi_{l,j,i}$7, with one bias per neuron. Regression uses MSE, classification uses binary cross-entropy, and optimization uses Adam under hardware-motivated constraints such as gain clipping and penalties for tuning voltages outside the high-SNR range (Taglietti et al., 20 Jan 2026).

The self-optimizing loop is explicitly hardware-in-the-loop. After each training epoch, updated tuning parameters are reprogrammed into the SYNE array or time-multiplexed via FPGA, and fresh $\phi_{l,j,i}$8–$\phi_{l,j,i}$9 measurements are taken. The method also incorporates expressivity-guided architecture selection via the $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$0-packing expressivity metric, adaptive pruning of low-impact synaptic nonlinearities, and on-device re-tuning as aging or drift alters the device curves. This makes self-optimization a joint adaptation of network structure and physical nonlinear response rather than a purely software pruning procedure (Taglietti et al., 20 Jan 2026).

6. Empirical behavior, interpretability, and limitations

GA-KAN is validated on two toy datasets and five UCI classification benchmarks. On Iris and Wine, GA-KAN and the standard KAN both reach $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$1 accuracy, but GA-KAN uses far fewer parameters; on Rice, WDBC, and Raisin, GA-KAN improves or matches the standard KAN while also reducing parameter count (Long et al., 29 Jan 2025).

Dataset	GA-KAN	Reference KAN
Iris	100.00%; 156 params	KAN[4,9,3]: 100.00%; 882 params
Wine	100.00%; 390 params	KAN[13,27,3]: 100.00%; 6,912 params
Rice	95.14%; AUC 0.985; 585 params	KAN[7,15,2]: 94.23%; AUC 0.984; 1,620 params
WDBC	100.00%; AUC 1.000; 1,560 params	KAN[30,61,2]: 100.00%; AUC 1.000; 27,328 params
Raisin	90.00%; AUC 0.938; 1,064 params	KAN[7,15,2]: 86.67%; AUC 0.935; 3,240 params

For interpretability, GA-KAN extracts symbolic formulae in two stages: auto_symbolic first chooses among a library of primitives by $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$2 fit, after which a manual optional stage can inspect shapes, fix repeated patterns, and retrain coefficients. The Iris example is reported as

$$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$3

with $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$4 giving class probabilities (Long et al., 29 Jan 2025).

The differentiable SO-KAN literature reports a similar accuracy-complexity pattern under a different optimization regime. On Nguyen symbolic-regression benchmarks, Baseline and FC-Only fit with $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$5 but retain $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$6 trunk activations; Gates-Only with $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$7 prunes to $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$8 activations with $$\phi(x)=w_b\,b(x)+w_s\sum_{i=0}^{G+k-1}c_i\,B_{i,k}(x),$$9; and the Full model with $f:[0,1]^n\to\mathbb R$00 often prunes the trunk entirely, using $f:[0,1]^n\to\mathbb R$01 trunk $f:[0,1]^n\to\mathbb R$02 forward connection with $f:[0,1]^n\to\mathbb R$03. On the Ikeda map, the Full model with $f:[0,1]^n\to\mathbb R$04 prunes to $f:[0,1]^n\to\mathbb R$05 of activations while multi-step RMSE rises only $f:[0,1]^n\to\mathbb R$06. On real-world regression tasks, the Full setting with $f:[0,1]^n\to\mathbb R$07–$f:[0,1]^n\to\mathbb R$08 cuts to $f:[0,1]^n\to\mathbb R$09–$f:[0,1]^n\to\mathbb R$10 of activations and reduces RMSE by $f:[0,1]^n\to\mathbb R$11–$f:[0,1]^n\to\mathbb R$12 (Bagrow et al., 13 Dec 2025).

The physical SO-KAN results extend the same theme to device efficiency. In $f:[0,1]^n\to\mathbb R$13D composite regression with architecture $f:[0,1]^n\to\mathbb R$14 and $f:[0,1]^n\to\mathbb R$15 SYNEs per synapse, the model uses $f:[0,1]^n\to\mathbb R$16 trainable parameters and $f:[0,1]^n\to\mathbb R$17 SYNEs with test MSE $f:[0,1]^n\to\mathbb R$18, while a software MLP to match the MSE requires $f:[0,1]^n\to\mathbb R$19 parameters. In classification with architecture $f:[0,1]^n\to\mathbb R$20 and $f:[0,1]^n\to\mathbb R$21 SYNEs per synapse, the model uses $f:[0,1]^n\to\mathbb R$22 parameters and $f:[0,1]^n\to\mathbb R$23 devices, reaches $f:[0,1]^n\to\mathbb R$24 accuracy on yin-yang and $f:[0,1]^n\to\mathbb R$25 on “5×checker,” and is compared with an MLP baseline needing $f:[0,1]^n\to\mathbb R$26 parameters. For Li-ion battery end-of-life prediction, architecture $f:[0,1]^n\to\mathbb R$27 uses $f:[0,1]^n\to\mathbb R$28 devices and $f:[0,1]^n\to\mathbb R$29 parameters to achieve normalized MSE $f:[0,1]^n\to\mathbb R$30 on a $f:[0,1]^n\to\mathbb R$31-battery test set, while a matching MLP is reported as $f:[0,1]^n\to\mathbb R$32k parameters (Taglietti et al., 20 Jan 2026).

The main limitations are method-specific. Current GA-KAN is costly because fitness evaluation requires training KANs and has been tested on low-dimensional data; proposed extensions include surrogate fitness, weight-sharing, GPU-gradient acceleration, multi-objective GA, and on-device deployment via further quantization or pruning (Long et al., 29 Jan 2025). In the differentiable setting, the trade-off between expressivity and compactness is controlled by $f:[0,1]^n\to\mathbb R$33, and for the ecosystem task heavy pruning requires longer training to recover the attractor (Bagrow et al., 13 Dec 2025). In the physical setting, continual re-optimization is integral rather than incidental, because hardware drift and aging are part of the operating regime (Taglietti et al., 20 Jan 2026).

Taken together, these results show that “self-optimizing” in SO-KAN can mean automated neural architecture search, differentiable architecture pruning under a complexity penalty, or hardware-native adaptation of synaptic nonlinearities. The unifying theme is that KAN expressivity is not treated as a fixed handcrafted design, but as an object of optimization in its own right.