Support Vector Machine Implementation on MPI-CUDA and Tensorflow Framework (2311.14908v1)
Abstract: Support Vector Machine (SVM) algorithm requires a high computational cost (both in memory and time) to solve a complex quadratic programming (QP) optimization problem during the training process. Consequently, SVM necessitates high computing hardware capabilities. The central processing unit (CPU) clock frequency cannot be increased due to physical limitations in the miniaturization process. However, the potential of parallel multi-architecture, available in both multi-core CPUs and highly scalable GPUs, emerges as a promising solution to enhance algorithm performance. Therefore, there is an opportunity to reduce the high computational time required by SVM for solving the QP optimization problem. This paper presents a comparative study that implements the SVM algorithm on different parallel architecture frameworks. The experimental results show that SVM MPI-CUDA implementation achieves a speedup over SVM TensorFlow implementation on different datasets. Moreover, SVM TensorFlow implementation provides a cross-platform solution that can be migrated to alternative hardware components, which will reduces the development time.
- Michael J Flynn. Some computer organizations and their effectiveness. IEEE transactions on computers, 100(9):948–960, 1972.
- Network and Parallel Computing: IFIP International Conference, NPC 2008, Shanghai, China, October 18-20, 2008, Proceedings, volume 5245. Springer, 2008.
- Parallel computing for real-time signal processing and control. Springer Science & Business Media, 2003.
- Scaling up machine learning: Parallel and distributed approaches. Cambridge University Press, 2011.
- Rs/6000 sp: Practical MPI programming. IBM Poughkeepsie, New York, 1999.
- {{\{{TensorFlow}}\}}: a system for {{\{{Large-Scale}}\}} machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16), pages 265–283, 2016.
- Shane Cook. CUDA programming: a developer’s guide to parallel computing with GPUs. Newnes, 2012.
- CUDA Nvidia. Nvidia cuda c programming guide, version 4.2. NVIDIA: Santa Clara, CA, 66, 2010.
- CUDA by example: an introduction to general-purpose GPU programming. Addison-Wesley Professional, 2010.
- Design and implementation of a hybrid mpi-cuda model for the smith-waterman algorithm. International journal of data mining and bioinformatics, 12(3):313–327, 2015.
- A stream processor cluster architecture model with the hybrid technology of mpi and cuda. In 2009 First International Conference on Information Science and Engineering, pages 86–89. IEEE, 2009.
- Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST), 2(3):1–27, 2011.
- Fast support vector machine training and classification on graphics processors. In Proceedings of the 25th international conference on Machine learning, pages 104–111, 2008.
- A comparison of methods for multiclass support vector machines. IEEE transactions on Neural Networks, 13(2):415–425, 2002.
- Advances in kernel methods: support vector learning. MIT press, 1999.
- Working set selection using second order information for training support vector machines. Journal of machine learning research, 6(12), 2005.
- Improvements to platt’s smo algorithm for svm classifier design. Neural computation, 13(3):637–649, 2001.
- Gpu machine learning library (gpumlib). Machine Learning for Adaptive Many-Core Machines-A Practical Approach, pages 15–36, 2015.
- Gpu parallel implementation of support vector machines for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(10):4647–4656, 2015.
- Multi-class support vector machine training and classification based on mpi-gpu hybrid parallel architecture. In Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2018 4, pages 179–188. Springer, 2019.
- Peter Goldsborough. A tour of tensorflow. arXiv preprint arXiv:1610.01178, 2016.
- Comparison of gpu and fpga based hardware platforms for ultrasonic flaw detection using support vector machines. In 2017 IEEE International Ultrasonics Symposium (IUS), pages 1–4. IEEE, 2017.