---
title: Hybrid LSTM-KAN Model Overview
url: https://www.emergentmind.com/topics/hybrid-lstm-kan-model
type: topic
---

# Hybrid LSTM-KAN Model Overview

Searching arXiv for hybrid LSTM–KAN and closely related TKAN/KAN time-series papers.
Hybrid LSTM-KAN models are neural architectures that combine Long Short-Term Memory (LSTM) sequence modeling with Kolmogorov-Arnold Networks (KANs), a class of networks in which learnable univariate functions are placed on edges rather than fixed activations on nodes. In the recent literature, the term denotes not a single canonical network but a family of designs that couple LSTM-style memory and gating with KAN-style functional approximation. The hybridization appears in at least three forms: a KAN used as a post-recurrent prediction head, a recurrent cell in which KAN computations are integrated into the gating or state-update mechanism, and larger composite systems in which LSTM-derived sequence encoders and KAN projection layers are embedded inside encoder-decoder pipelines [2405.07344]. The hybrid idea is motivated by standalone evidence that KANs can outperform similarly shaped MLPs in time-series forecasting with fewer learnable parameters [2405.08790].

## 1. Theoretical basis and defining components

The KAN component is grounded in the Kolmogorov-Arnold representation theorem, written in the time-series literature as
\[
f(\mathbf{x})=\sum_{i=1}^{2n+1}\Phi_i\left(\sum_{j=1}^n \phi_{i,j}(x_j)\right),
\]
or equivalently,
\[
f(x_1,\dots,x_n)=\sum_{q=1}^{2n+1}\Phi_q\left(\sum_{p=1}^n \phi_{q,p}(x_p)\right).
\]
In KAN-based models, this theorem is interpreted as a network whose nonlinearities are carried by trainable univariate edge functions rather than by fixed node activations. A KAN layer is therefore a matrix of functions \(\mathbf{\Phi}\), and the forward rule can be written as
\[
x_{l+1,j}=\sum_{i=1}^{n_l}\phi_{l,j,i}(x_{l,i}),
\qquad
\mathbf{x}_{l+1}=\mathbf{\Phi}_l\mathbf{x}_l.
\]
In the time-series forecasting work on satellite traffic, these edge functions are approximated with B-splines, with spline order \(k\) and grid size \(G\), and the network is trained by backpropagation because the construction is fully differentiable [2405.08790].

The LSTM component contributes stateful temporal memory through gated recurrence. In the loan-default formulation, the standard update is
\[
f_t = \sigma(W_f[h_{t-1}]+b_f),\quad
i_t = \sigma(W_i[h_{t-1},x_t]+b_i),
\]
\[
\tilde{c}_t = \tanh(W_c[h_{t-1},x_t]+b_c),\quad
c_t = f_t \odot c_{t-1} + i_t \odot \tilde{c}_t,
\]
\[
o_t = \sigma(W_o[h_{t-1},x_t]+b_o),\quad
h_t = o_t \odot \tanh(c_t).
\]
This mechanism preserves useful historical information while filtering irrelevant past states [2507.13685].

The hybrid rationale is explicit across the literature: LSTM is used for temporal dependency capture and memory management, while KAN is used for more expressive nonlinear transformation after, within, or around the recurrent state evolution. This suggests a division of labor in which recurrence handles sequence structure and KAN handles adaptive feature-to-output mapping.

## 2. Architectural patterns

The literature suggests three recurring integration patterns.

| Pattern | Representative structure | Example papers |
|---|---|---|
| Post-recurrent KAN head | recurrent encoder \(\rightarrow\) KAN \(\rightarrow\) classifier/regressor | [2507.13685], [2601.03610] |
| Intrinsic recurrent hybrid | KAN or RKAN inside LSTM-style gating/state update | [2405.07344], [2512.05049] |
| Composite encoder-decoder hybrid | LSTM-derived backbone with KAN projection blocks | [2501.07017], [2603.28985] |

In the post-recurrent design, the LSTM acts as a feature extractor and the KAN replaces the usual dense decision head. The loan-default model uses preprocessing, masking, two stacked LSTM layers, Batch Normalization, a KAN layer, a dense layer, Dropout, and a sigmoid output layer; the first LSTM has 128 units with `return_sequences=True`, the second has 64 units with `return_sequences=False`, and the KAN is specified with `output_dim=1` and `num_functions=10` [2507.13685]. A related respiratory-sound classifier uses a BiLSTM with hidden size \(64\), Dropout \(0.3\), and a KAN back-end with one hidden KAN layer of \(32\) neurons and an output layer of \(6\) neurons, using cubic splines of order \(3\) and grid size \(3\) [2601.03610].

In the intrinsic recurrent hybrid, the KAN machinery is folded into the recurrent dynamics themselves. TKAN introduces Recurrent Kolmogorov-Arnold Network (RKAN) layers together with LSTM-like gates and cell state, so that KAN computations become temporally aware through per-layer memory variables and the output gate is driven by an RKAN-based composite representation rather than by a standard affine map [2405.07344]. QKAN-LSTM pushes this further by replacing the affine gate pre-activations with sums of KAN-like quantum-inspired variational activation functions, while preserving the external LSTM gate structure [2512.05049].

In the composite encoder-decoder pattern, hybridization is distributed across a larger architecture. UNETVL replaces ViT encoder blocks in a UNETR-like 3D segmentation system with Vision-LSTM blocks and then replaces MLP-style up- and down-projection layers inside the modified ViL block with Chebyshev KAN layers [2501.07017]. In IoT threat detection, the reported KAN-LSTM architecture is explicitly `Preprocessing → 2 CNN layers → 1 LSTM layer → 2 KAN layers → classifier/output`, assigning local feature extraction to CNN, temporal encoding to LSTM, and adaptive nonlinear decision mapping to KAN [2603.28985].

## 3. Mathematical forms of hybridization

A direct post-recurrent hybrid is summarized in the loan-default work as
\[
x_{1:T} \xrightarrow{\text{LSTM/GRU}} h \xrightarrow{\text{KAN}} z \xrightarrow{\text{Dense + Dropout}} \hat{y},
\]
where \(\hat{y}\) is the predicted default probability [2507.13685]. In this formulation, the recurrent block compresses the sequence into a hidden representation and the KAN transforms that representation using learnable univariate nonlinear functions before final classification.

TKAN uses a deeper integration. Its RKAN update is written as
\[
x_{l+1,j}(t)=\sum_{i=1}^{n_l}\phi_{l,j,i,t}(x_{l,i}(t),h_{l,i}(t)),
\]
with recurrent memory
\[
h_{l,i}(t)=W_{hh}h_{l,i}(t-1)+W_{hz}x_{l,i}(t).
\]
The LSTM-style gates remain largely standard,
\[
f_t = \sigma(W_f x_t + U_f h_{t-1} + b_f),\quad
i_t = \sigma(W_i x_t + U_i h_{t-1} + b_i),
\]
\[
\tilde{c}_t = \sigma(W_c x_t + U_c h_{t-1} + b_c),\quad
c_t = f_t \odot c_{t-1} + i_t \odot \tilde{c}_t,
\]
but the output gate becomes
\[
o_t = \sigma(\xi(\mathbf{x}, t)),
\]
where \(\xi(\mathbf{x}, t)\) is the RKAN-based composite representation, and the hidden state is
\[
h_t = o_t \odot \tanh(c_t).
\]
The defining feature is therefore a dual memory structure: RKAN memory inside KAN layers for short-term temporal context and LSTM cell state for long-term dependency management [2405.07344].

QKAN-LSTM retains the LSTM shell while replacing each gate pre-activation with a KAN-like sum of learned functional units. For gate \(g \in \{f,i,C,o\}\), with \(v_t=[h_{t-1};x_t]\), the paper defines
\[
\Phi_g(v_t;\Theta_g)=\sum_{p=1}^{\alpha}\phi_{g,p}(v_t;\boldsymbol{\theta}_{g,p}),
\]
and then
\[
f_t=\sigma(\Phi_f(v_t;\Theta_f)),\quad
i_t=\sigma(\Phi_i(v_t;\Theta_i)),
\]
\[
\tilde{C}_t=\tanh(\Phi_C(v_t;\Theta_C)),\quad
o_t=\sigma(\Phi_o(v_t;\Theta_o)).
\]
This formulation makes the hybridization occur directly at the gate level rather than in a separate readout head [2512.05049].

In Chebyshev-KAN-based Vision-LSTM, the KAN module is defined through polynomial basis expansion rather than B-splines. The basis recursion is
\[
T_0(x)=1,\quad T_1(x)=x,\quad T_m(x)=2xT_{m-1}(x)-T_{m-2}(x),\quad m\ge 2,
\]
and the output is formed by Einstein summation over basis tensor \(\mathbf{T}\) and coefficient tensor \(\mathbf{C}\). This is a distinct KAN instantiation but preserves the central idea of edge-wise functional approximation [2501.07017].

## 4. Empirical record across domains

In time-series forecasting, the empirical basis for hybrid LSTM-KAN models begins with standalone KAN results. On satellite traffic forecasting, a 4-depth KAN with architecture \([168,40,40,40,24]\), using B-spline activations with \(k=3\) and \(G=5\), reports MSE \(5.08\times 10^{-3}\), RMSE \(7.12\times 10^{-2}\), MAE \(5.06\times 10^{-2}\), MAPE \(0.52\), and \(109\)k parameters; the compared 3-depth and 4-depth MLPs use \(238\)k and \(329\)k parameters, respectively, with worse error metrics [2405.08790]. This does not yet constitute a hybrid model, but it provides the empirical premise for replacing MLP heads in recurrent systems.

TKAN supplies the first explicit recurrent KAN-LSTM synthesis for multi-step forecasting. On Bitcoin market traded notional values, its \(R^2\) is \(0.350845\) at 1-step ahead, slightly below GRU at \(0.365136\), but at 6 steps ahead TKAN reaches \(0.140543\) while GRU drops to \(0.082504\) and LSTM to \(-0.225838\); at 12 and 15 steps ahead, GRU and LSTM are negative while TKAN remains positive at \(0.105111\) and \(0.086077\) [2405.07344]. The reported training and validation curves are also described as much closer together for TKAN than for GRU or LSTM.

In financial risk modeling, LSTM-KAN and GRU-KAN are evaluated for loan-default early prediction on the Freddie Mac Single-Family Loan-Level dataset. The abstract reports over \(92\%\) accuracy three months in advance and over \(88\%\) accuracy eight months in advance. Within the early-prediction analysis, the best AUC at the 3-month blank interval is reported for LSTM-KAN with average AUC \(=0.9278\), and the paper states that the proposed models can achieve at a 5-month blank interval roughly the same accuracy, recall, and F1 that baseline models achieve at 3 months, implying about 2 additional months of warning time [2507.13685].

In imbalanced respiratory sound classification on the ICBHI 2017 Respiratory Sound Database, the Hybrid LSTM-KAN model reports \(94.55\%\) accuracy, macro precision \(0.7292\), macro recall \(0.6978\), macro-F1 \(0.7033\), and weighted-F1 \(0.9436\), with the dominant COPD class comprising about \(86.5\%\) of the filtered dataset. The ablation study reports a baseline cross-entropy system at \(91.23\%\) accuracy and macro-F1 \(0.5821\), rising to \(94.55\%\) and \(0.7033\) when focal loss, class-specific augmentation, and SMOTE are combined. Per-class F1 for the proposed model is \(0.8387\) for Bronchiectasis, \(0.7730\) for Pneumonia, \(0.4390\) for URTI, and \(0.4538\) for Bronchiolitis [2601.03610].

In cyber-security threat detection, KAN-LSTM is benchmarked on UNSW-NB15, NSL-KDD, CICIDS2017, and a combined Tri-IDS dataset. Reported results are \(95.89\%\) accuracy, \(94.50\%\) precision, \(93.20\%\) recall, and \(94.01\%\) F1 on UNSW-NB15; \(94.05\%\), \(98.95\%\), \(92.15\%\), and \(94.35\%\) on NSL-KDD; \(99.28\%\), \(99.06\%\), \(99.30\%\), and \(99.66\%\) on CICIDS2017; and \(93.76\%\), \(95.85\%\), \(91.06\%\), and \(93.39\%\) on Tri-IDS [2603.28985].

Beyond 1D sequence tasks, hybridization also appears in 3D medical image segmentation. UNETVL reports mean Dice \(91.59\%\) on ACDC versus \(85.34\%\) for UNETR, and \(88.57\%\) on AMOS2022 versus \(76.59\%\) for UNETR. In the ablation table, the ViL-only configuration at \(K=384\) gives \(90.14\%\) Dice on ACDC and \(86.52\%\) on AMOS, while the KAN-enabled version at the same width improves to \(91.59\%\) and \(88.57\%\) [2501.07017].

## 5. Interpretability, efficiency, and limitations

Interpretability is a recurring argument for KAN-based hybrids. Because KAN replaces fixed activations and scalar weights with learnable univariate functions, the learned edge functions can in principle be inspected. The satellite-traffic paper explicitly contrasts this with opaque dense weight matrices, and the respiratory-sound study similarly presents KAN’s spline functions as offering insight into feature-to-class mappings [2405.08790]. This interpretability claim is architectural rather than fully validated across all applications.

Efficiency claims are more conditional. In standalone forecasting, KAN can be markedly more parameter-efficient than matched MLP baselines, as in the \(109\)k-parameter KAN versus \(238\)k and \(329\)k MLPs on satellite traffic [2405.08790]. In respiratory-sound classification, the reported Hybrid LSTM-KAN has \(287\)k parameters, training time \(3.8\) min/epoch, inference time \(2.1\) ms/sample, about \(640\) MB GPU memory during training, and about \(180\) MB during inference [2601.03610]. QKAN-LSTM reports a stronger parameter-efficiency result on Urban Telecommunication, with \(58\) total trainable parameters for QKAN-LSTM versus \(277\) for LSTM, corresponding to the paper’s headline \(79\%\) reduction [2512.05049].

Yet KAN does not uniformly reduce cost. In UNETVL, enabling KAN at \(K=384\) increases parameters from \(94.96\)M to \(158.61\)M while improving Dice. The paper explicitly frames this as a tradeoff between expressiveness and model complexity [2501.07017]. The lesson is that parameter efficiency depends on where KAN is inserted and which basis functions and widths are used.

The limitations are likewise consistent across papers. TKAN is not especially superior at very short horizons, where GRU can be slightly better, and its ablation study does not fully isolate the incremental contribution of RKAN versus the gate modifications [2405.07344]. The standalone KAN forecasting study notes that robustness across diverse datasets is not yet fully established and that the architecture has not yet been shown to compete directly with more advanced temporal models such as LSTMs, GRUs, CNNs, or attention-based methods [2405.08790]. The cyber-security study acknowledges slow processing, limited scalability, expensive training, and some stability issues for KANs [2603.28985]. The respiratory-sound work notes that rare classes remain difficult, feature aggregation may lose fine-grained temporal information, only one public dataset is used, and no real-world clinical validation has yet been conducted [2601.03610].

## 6. Research trajectory and conceptual boundaries

A common misconception is that “Hybrid LSTM-KAN” names a settled, standardized architecture. The literature indicates otherwise. TKAN is a recurrent KAN with explicit LSTM-like memory control; the loan-default and respiratory-sound systems are LSTM encoders with KAN heads; UNETVL is a Vision-LSTM plus Chebyshev-KAN encoder module inside a 3D segmentation pipeline; and QKAN-LSTM replaces affine gate transforms with KAN-like quantum-inspired activations. These are related by design philosophy rather than by a unique blueprint.

The broader research trajectory reinforces this point. In deep state-space recurrent modeling for Bitcoin regime prediction, the switching TKAN variant, m-TKAN, reports accuracy \(0.72\) versus \(0.67\) for m-LSTM and \(0.56\) for m-GRU, together with the strongest reported in-sample and out-of-sample risk-adjusted backtesting metrics among the three switching models [2407.15236]. In option pricing, by contrast, the literature provides a complementary result: KAN outperforms MLP and TDNN as a nonlinear approximator, while the best overall model is an LSTM-GRU hybrid RNN with attention, and the authors explicitly identify integrating LSTM or GRU layers and attention mechanisms into MLP and KAN models as future work [2409.06724]. This suggests that the current field treats KAN and recurrent modeling as complementary rather than mutually exclusive.

A plausible implication is that the appropriate hybrid design depends on the bottleneck of the target problem. If the main difficulty lies in mapping a recurrent latent state to a complex output manifold, a KAN head may be sufficient. If the difficulty lies inside the recurrent transition itself, an intrinsic recurrent-KAN design such as TKAN or QKAN-LSTM is more directly aligned with the problem. If long-range context, local structure, and dense reconstruction must all be handled simultaneously, as in 3D segmentation or network-traffic analysis, composite hybrids become more natural.

Hybrid LSTM-KAN modeling therefore designates a developing research area rather than a finalized architecture class. Its unifying theme is the attempt to combine recurrent memory control with adaptive edge-wise function approximation, while negotiating a recurring tradeoff among expressiveness, parameter count, training stability, and implementation complexity.

Source: https://www.emergentmind.com/topics/hybrid-lstm-kan-model