Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash 103 tok/s
Gemini 2.5 Pro 54 tok/s Pro
GPT-5 Medium 27 tok/s
GPT-5 High 37 tok/s Pro
GPT-4o 92 tok/s
GPT OSS 120B 467 tok/s Pro
Kimi K2 241 tok/s Pro
2000 character limit reached

FC-KAN: Function Combinations in Kolmogorov-Arnold Networks (2409.01763v3)

Published 3 Sep 2024 in cs.LG and cs.CL

Abstract: In this paper, we introduce FC-KAN, a Kolmogorov-Arnold Network (KAN) that leverages combinations of popular mathematical functions such as B-splines, wavelets, and radial basis functions on low-dimensional data through element-wise operations. We explore several methods for combining the outputs of these functions, including sum, element-wise product, the addition of sum and element-wise product, representations of quadratic and cubic functions, concatenation, linear transformation of the concatenated output, and others. In our experiments, we compare FC-KAN with a multi-layer perceptron network (MLP) and other existing KANs, such as BSRBF-KAN, EfficientKAN, FastKAN, and FasterKAN, on the MNIST and Fashion-MNIST datasets. Two variants of FC-KAN, which use a combination of outputs from B-splines and Difference of Gaussians (DoG) and from B-splines and linear transformations in the form of a quadratic function, outperformed overall other models on the average of 5 independent training runs. We expect that FC-KAN can leverage function combinations to design future KANs. Our repository is publicly available at: https://github.com/hoangthangta/FC_KAN.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. Kan: Kolmogorov-arnold networks. arXiv preprint arXiv:2404.19756, 2024a.
  2. Kan 2.0: Kolmogorov-arnold networks meet science. arXiv preprint arXiv:2408.10205, 2024b.
  3. Andrei Nikolaevich Kolmogorov. On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition. In Doklady Akademii Nauk, volume 114, pages 953–956. Russian Academy of Sciences, 1957.
  4. Ziyao Li. Kolmogorov-arnold networks are radial basis function networks. arXiv preprint arXiv:2405.06721, 2024.
  5. Athanasios Delis. Fasterkan. https://github.com/AthanasiosDelis/faster-kan/, 2024.
  6. Wav-kan: Wavelet kolmogorov-arnold networks. arXiv preprint arXiv:2405.12832, 2024.
  7. Sidharth SS. Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation. arXiv preprint arXiv:2405.07200, 2024.
  8. Hoang-Thang Ta. Bsrbf-kan: A combination of b-splines and radial basis functions in kolmogorov-arnold networks. arXiv preprint arXiv:2406.11173, 2024.
  9. Activation space selectable kolmogorov-arnold networks. arXiv preprint arXiv:2408.08338, 2024.
  10. On a constructive proof of kolmogorov’s superposition theorem. Constructive approximation, 30:653–675, 2009.
  11. Treedrnet: a robust deep model for long term time series forecasting. arXiv preprint arXiv:2206.12106, 2022.
  12. Space-filling curves and kolmogorov superposition-based neural networks. Neural Networks, 15(1):57–67, 2002.
  13. Mario Köppen. On the training of a kolmogorov network. In Artificial Neural Networks—ICANN 2002: International Conference Madrid, Spain, August 28–30, 2002 Proceedings 12, pages 474–479. Springer, 2002.
  14. On the realization of a kolmogorov network. Neural Computation, 5(1):18–20, 1993.
  15. The kolmogorov spline network for image processing. In Image Processing: Concepts, Methodologies, Tools, and Applications, pages 54–78. IGI Global, 2013.
  16. The kolmogorov superposition theorem can break the curse of dimensionality when approximating high dimensional functions. arXiv preprint arXiv:2112.09963, 2021.
  17. Representation properties of networks: Kolmogorov’s theorem is irrelevant. Neural Computation, 1(4):465–469, 1989.
  18. AG Vitushkin. On hilbert’s thirteenth problem. In Dokl. Akad. Nauk SSSR, volume 95, pages 701–704, 1954.
  19. Věra Kůrková. Kolmogorov’s theorem is relevant. Neural computation, 3(4):617–622, 1991.
  20. Vikas Dhiman. Kan: Kolmogorov–arnold networks: A review. https://vikasdhiman.info/reviews/KAN_a_review.pdf, 2024.
  21. Carl De Boor. On calculating with b-splines. Journal of Approximation theory, 6(1):50–62, 1972.
  22. Deepokan: Deep operator network based on kolmogorov arnold networks for mechanics problems. arXiv preprint arXiv:2405.19143, 2024.
  23. Subhransu S. Bhattacharjee. Torchkan: Simplified kan model with variations. https://github.com/1ssb/torchkan/, 2024.
  24. Fourierkan-gcf: Fourier kolmogorov-arnold network–an effective and efficient feature transformation for graph collaborative filtering. arXiv preprint arXiv:2406.01034, 2024.
  25. Seyd Teymoor Seydi. Unveiling the power of wavelets: A wavelet-based kolmogorov-arnold network for hyperspectral image classification. arXiv preprint arXiv:2406.07869, 2024.
  26. Seyd Teymoor Seydi. Exploring the potential of polynomial basis functions in kolmogorov-arnold networks: A comparative study of different groups of polynomials. arXiv e-prints, pages arXiv–2406, 2024.
  27. Andrei Vladimirovich Chernov. Gaussian functions combined with kolmogorov’s theorem as applied to approximation of functions of several variables. Computational Mathematics and Mathematical Physics, 60:766–782, 2020.
  28. Johannes Schmidt-Hieber. The kolmogorov–arnold representation theorem revisited. Neural networks, 137:119–126, 2021.
  29. Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6):141–142, 2012.
  30. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
  31. Kan or mlp: A fairer comparison. arXiv preprint arXiv:2407.16674, 2024.
Citations (4)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

  • The paper introduces FC-KAN, an innovative approach that combines B-splines, wavelets, and radial basis functions to process low-dimensional data effectively.
  • It demonstrates that a quadratic combination of B-splines and Difference of Gaussians functions outperforms standard KANs and MLPs on benchmarks like MNIST.
  • The findings suggest that integrating diverse function combinations in KANs enhances model accuracy, interpretability, and efficiency for future neural network designs.

FC-KAN, or Function Combinations in Kolmogorov-Arnold Networks, represents an advanced and innovative approach to enhancing Kolmogorov-Arnold Networks (KANs) by leveraging combinations of various mathematical functions. In the paper titled "FC-KAN: Function Combinations in Kolmogorov-Arnold Networks" (2409.01763), the authors propose using combinations of B-splines, wavelets, and radial basis functions to process low-dimensional data. The main idea is to combine the outputs of these functions through several methods, such as sums, element-wise products, quadratic function representations, and concatenations.

KANs stand out from conventional Multi-Layer Perceptrons (MLPs) due to their unique use of learnable activation functions on edges instead of fixed activation functions on nodes. This structure theoretically allows KANs to achieve better performance and interpretability while requiring fewer parameters compared to MLPs (Liu et al., 30 Apr 2024). Building on this foundational idea, FC-KAN explores how combining different mathematical functions can further enhance the versatility and performance of these networks.

The FC-KAN architecture demonstrates superior performance in experiments. Specifically, a variant using a quadratic combination of B-splines and Difference of Gaussians (DoG) functions outperformed other models, including standard KANs and MLPs, on benchmark datasets like MNIST and Fashion-MNIST (2409.01763). These results suggest that function combinations allow FC-KAN to better capture the intricacies of the data, making it a promising advancement in the design of future KANs.

Additionally, other papers suggest interesting applications and variations of KANs that complement FC-KAN’s methodology. For instance, Convolutional KANs integrate the non-linear activation functions from KANs into convolutional layers, maintaining high accuracy with significantly fewer parameters (Bodner et al., 19 Jun 2024). Another variation, Wav-KAN, employs wavelet functions to balance between representing data accurately and minimizing overfitting, enhancing training speed and robustness (Bozorgasl et al., 21 May 2024).

In summary, FC-KAN extends the capabilities of Kolmogorov-Arnold Networks by employing combinations of diverse mathematical functions, improving both accuracy and efficiency. This development, along with related innovations in the KAN framework, highlights the potential of KANs as powerful, adaptable tools in neural network architectures.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Ai Generate Text Spark Streamline Icon: https://streamlinehq.com

Paper Prompts

Sign up for free to create and run paper prompts using GPT-5.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - hoangthangta/FC_KAN: Function Combinations in Kolmogorov-Arnold Networks (17 stars)

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube