LTBs-KAN: Linear-Time B-splines Kolmogorov-Arnold Networks

Abstract: Kolmogorov-Arnold Networks (KANs) are a recent neural network architecture offering an alternative to Multilayer Perceptrons (MLPs) with improved explainability and expressibility. However, KANs are significantly slower than MLPs due to the recursive nature of B-spline function computations, limiting their application. This work addresses these issues by proposing a novel base-spline Linear-Time B-splines Kolmogorov-Arnold Network (LTBs-KAN) with linear complexity. Unlike previous methods that rely on the Boor-Mansfield-Cox spline algorithm or other computationally intensive mathematical functions, our approach significantly reduces the computational burden. Additionally, we further reduce model's parameter through product-of-sums matrix factorization in the forward pass without sacrificing performance. Experiments on MNIST, Fashion-MNIST and CIFAR-10 demonstrate that LTBs-KAN achieves good time complexity and parameter reduction, when used as building architectural blocks, compared to other KAN implementations.

• The paper introduces LTBs-KAN, a novel Kolmogorov-Arnold network that leverages a linear-time B-spline algorithm to significantly reduce computational complexity.
• It employs structured parameter reduction via product-of-sums matrix factorization, achieving high accuracy with fewer parameters across standard benchmarks.
• The architecture extends to convolutional networks, demonstrating competitive accuracy and efficiency on MNIST, Fashion-MNIST, and CIFAR-10 datasets.

Linear-Time B-splines Kolmogorov-Arnold Networks: Algorithmic and Empirical Advances

Background and Motivations

Kolmogorov-Arnold Networks (KANs) were introduced as a function-approximating architecture motivated by the Kolmogorov-Arnold representation theorem. By employing learnable univariate functions—typically realized as B-splines—on the edges between nodes, KANs vastly increase architectural expressivity over classic MLPs relying on fixed activations. This enables more interpretable internal representations and advanced function approximation properties. However, the computational burden of evaluating and differentiating B-spline basis functions—especially through the widely used Boor-Mansfield-Cox recursive formula—renders KANs orders of magnitude slower than parameter-matched MLPs, curtailing their practical utility [[liu2024kan]].

Prior attempts to reduce KAN complexity—such as approximating splines with switch activations (EfficientKAN), or with radial basis functions (FastKAN, BSRBF-KAN), and substituting polynomial bases (Gottlieb-KAN)—have improved speed but at significant cost either in predictive performance or flexibility. There remains a gap for a KAN architecture attaining linear-time spline evaluation, with parameter efficiency, while safeguarding empirical accuracy.

Linear-Time B-spline Coefficient Computation

The central contribution of this work is the development of the LTBs Algorithm: a linear-time, parallelizable method to compute B-spline basis coefficients in their Bernstein-Bézier representation. The algorithm is rooted in recent theoretical advances from Chudy et al. [[Chudy2023LinearTimeBSpline]], who established a divided-difference-based recurrence reducing operations—most notably replacing $O(m^2)$ division operations required by Cox-De Boor recursion with just $O(m)$ for splines of degree $m$.

The algorithm divides computations into two block steps: (1) Explicit initialization of endpoint coefficients (forward initialization), and (2) Backward recursion for filling the remaining coefficient tensor using the optimized recurrence. The knots are adaptively recalibrated to the range of each input batch to maximize representational stability. The outcome is a tensorized polynomial coefficient representation for all local spline intervals, supporting efficient parallel evaluation on GPUs/TPUs.

Significantly, the entire process achieves $O(nm)$ sequential complexity (with $n$ grid size and $m$ spline order), and exposes parallelism for a theoretical $T_1/T_\infty \sim O(n/m)$ speedup, leveraging reduction-tree primitives for further runtime reduction.

Model Parameterization and Complexity

Beyond algorithmic acceleration, the paper introduces structured parameter reductions using a product-of-sums matrix factorization for the standard linear matrix mapping within each KAN layer. Instead of learning dense weight matrices, $W$ is written as a linear combination of a small set of fixed basis matrices: $W = \sum_{j=1}^p \sum_{k=1}^s a_{jk} M_{jk}$, with only $ps$ scalar coefficients $O(m)$0 as trainable parameters. This approach scales more favorably and enhances regularization, with only modest impact on representational power for practical $O(m)$1 values.

The layer output is a composition of a SiLU-linear base output and a spline interaction term (with coefficients from the LTBs algorithm), optionally followed by dropout and layer normalization stages. The computational complexity per layer is now $O(m)$2 for batch size $O(m)$3, input/output dims $O(m)$4 and $O(m)$5, with grid/knot adaptation amortized.

In the parallel computation model, key routines—tensor contractions, matrix multiplications, normalization—admit $O(m)$6 span, providing a principled path to exploiting high-throughput parallel architectures.

Extension to Convolutional KANs (KAN-ConvNet)

The framework is further extended to convolutional architectures. Instead of replacing kernel-dot-product operations with simple nonlinear activation, KAN-Conv2D layers apply the spline-based KAN transformation after patch extraction (im2col), yielding expressively adaptive filter responses while maintaining structural equivariance. Unlike previous KAN-ConvNet variants (KAConvNet, SpectralKAN), this approach separates spatial aggregation and functional approximation, facilitating general architectural compatibility and code modularity.

Experimental Evaluation

Empirical assessment is performed on MNIST, Fashion-MNIST, and CIFAR-10, comparing LTBs-KAN against standard KANs (EfficientKAN, FastKAN, BSRBF-KAN, Gottlieb-KAN, FasterKAN), as well as benchmark MLP and CNN baselines. Particular attention is given to parameter count, accuracy, precision, recall, F1-score, and wall-clock training time.

On MNIST, LTBs-KAN attains accuracy of 0.9632 with 203,378 parameters, outperforming all KANs except MLP, which achieves 0.9695 at triple the parameter count. BSRBF-KAN and FastKAN approach similar performance only at cost of thousands of additional parameters and slower computation.

On Fashion-MNIST, LTBs-KAN trails FastKAN and FasterKAN in accuracy by 0.01, though maintaining strong parameter efficiency and loss convergence close to MLP—suggesting minor loss of bias-variance tradeoff due to spline-degree regularization.

In CIFAR-10 experiments with convolutional architectures, LTBs-KAN ConvNet (with two levels of parameterization, $O(m)$7 and $O(m)$8) surpasses all KAN and MLP/CNN baselines in accuracy and F1, e.g., 0.8254 versus EfficientKAN-ConvNet’s 0.8222 and AlexNet’s 0.8208, at a moderate parameter count and substantially reduced training time (956.8–1734.6 s versus EfficientKAN-ConvNet’s 3298.8 s). MLP is dramatically outperformed (0.5144), indicating the importance of both convolutional structure and architectural expressivity.

Numerically, strong evidence is presented that LTBs-KAN optimally balances accuracy, model size, and computational overhead among all tested KAN variants, especially under real-world data and hardware constraints.

Practical and Theoretical Implications

The proposed LTBs-KAN model resolves a longstanding bottleneck in KAN adoption: spline evaluation cost. It shows that with theoretically grounded recurrence relations and appropriate parallel implementation, spline-based adaptive architectures are competitive with, and in some scenarios superior to, fixed-activation networks both in expressiveness and efficiency. The parameter reduction approach further enhances deployability in resource-constrained environments.

The LTBs-KAN ConvNet reveals that spline-based adaptability can be successfully married to convolutional inductive biases, opening further avenues in vision and structured data modeling where deeper functional approximation is needed. Results indicate that with appropriate knot/basis adaptation and modest regularization, KANs can generalize as well as classical deep networks but with better interpretability.

Limitations arise in balancing spline degree and data complexity (the bias-variance dilemma). Empirical findings highlight occasional marginal underperformance on harder domains—suggesting that further regularization, spline basis adaptation, or hybrid architectures may be needed for optimal transfer beyond canonical datasets.

Future Directions

Possible extensions include: (1) regularization strategies on spline degree and knot placement adaptive to data complexity; (2) fusion with other parameter-efficient approaches (e.g., sparsity, low-rank); (3) end-to-end comparison with efficient Transformer and SSM architectures in vision and sequence modeling; (4) application to modal data and non-Euclidean domains, exploiting the adaptability of B-spline kernels.

Conclusion

The paper establishes LTBs-KAN as a computationally efficient, parameter-light, and expressively competitive instantiation of Kolmogorov-Arnold Networks. By providing a linear-time algorithm for B-spline coefficient computation, structured parameterization via matrix factorization, and seamless extension to convolutional architectures, it addresses major challenges in KAN scalability and applicability. Results across vision benchmarks validate its effectiveness both as a standalone and as a building block for more advanced deep learning systems. Theoretical analysis confirms that LTBs-KAN is well-suited to current and future high-throughput parallel computing platforms, suggesting broad relevance for scalable, interpretable, and expressive function-approximating neural models.

Reference: "LTBs-KAN: Linear-Time B-splines Kolmogorov-Arnold Networks" (2604.22034)