Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multi-objective Binary Coordinate Search for Feature Selection (2402.12616v1)

Published 20 Feb 2024 in cs.LG and cs.NE

Abstract: A supervised feature selection method selects an appropriate but concise set of features to differentiate classes, which is highly expensive for large-scale datasets. Therefore, feature selection should aim at both minimizing the number of selected features and maximizing the accuracy of classification, or any other task. However, this crucial task is computationally highly demanding on many real-world datasets and requires a very efficient algorithm to reach a set of optimal features with a limited number of fitness evaluations. For this purpose, we have proposed the binary multi-objective coordinate search (MOCS) algorithm to solve large-scale feature selection problems. To the best of our knowledge, the proposed algorithm in this paper is the first multi-objective coordinate search algorithm. In this method, we generate new individuals by flipping a variable of the candidate solutions on the Pareto front. This enables us to investigate the effectiveness of each feature in the corresponding subset. In fact, this strategy can play the role of crossover and mutation operators to generate distinct subsets of features. The reported results indicate the significant superiority of our method over NSGA-II, on five real-world large-scale datasets, particularly when the computing budget is limited. Moreover, this simple hyper-parameter-free algorithm can solve feature selection much faster and more efficiently than NSGA-II.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (27)
  1. G. Chandrashekar and F. Sahin, “A survey on feature selection methods,” Computers & Electrical Engineering, vol. 40, no. 1, pp. 16–28, 2014.
  2. M. F. Ghalwash, X. H. Cao, I. Stojkovic, and Z. Obradovic, “Structured feature selection using coordinate descent optimization,” BMC bioinformatics, vol. 17, no. 1, pp. 1–14, 2016.
  3. A. A. Bidgoli, H. Ebrahimpour-Komleh, and S. Rahnamayan, “An evolutionary decomposition-based multi-objective feature selection for multi-label classification,” PeerJ Computer Science, vol. 6, p. e261, 2020.
  4. Y. Saeys, I. Inza, and P. Larranaga, “A review of feature selection techniques in bioinformatics,” bioinformatics, vol. 23, no. 19, pp. 2507–2517, 2007.
  5. M. H. Aghdam, N. Ghasem-Aghaee, and M. E. Basiri, “Text feature selection using ant colony optimization,” Expert systems with applications, vol. 36, no. 3, pp. 6843–6853, 2009.
  6. A. A. Bidgoli, S. Rahnamayan, T. Dehkharghanian, A. Riasatian, and H. Tizhoosh, “Evolutionary computation in action: Hyperdimensional deep embedding spaces of gigapixel pathology images,” IEEE Transactions on Evolutionary Computation, 2022.
  7. A. Asilian Bidgoli, S. Rahnamayan, T. Dehkharghanian, A. Grami, and H. R. Tizhoosh, “Bias reduction in representation of histopathology images using deep feature selection,” Scientific reports, vol. 12, no. 1, p. 19994, 2022.
  8. P. Agrawal, H. F. Abutarboush, T. Ganesh, and A. W. Mohamed, “Metaheuristic algorithms on feature selection: A survey of one decade of research (2009-2019),” IEEE Access, vol. 9, pp. 26 766–26 791, 2021.
  9. I. Guyon and A. Elisseeff, “An introduction to variable and feature selection,” Journal of machine learning research, vol. 3, no. Mar, pp. 1157–1182, 2003.
  10. H. Liu, H. Motoda, R. Setiono, and Z. Zhao, “Feature selection: An ever evolving frontier in data mining,” in Feature selection in data mining.   PMLR, 2010, pp. 4–13.
  11. J. H. Park, M. S. Khan, and M. Lee, “Modified coordinate descent methodology for solving process design optimization problems: application to natural gas plant,” Journal of Natural Gas Science and Engineering, vol. 27, pp. 32–41, 2015.
  12. D. Z. Farsa, A. A. Bidgoli, E. Rokhsat-Yazdi, and S. Rahnamayan, “Population-based coordinate descent algorithm with majority voting,” in Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2021, pp. 1283–1289.
  13. S. J. Wright, “Coordinate descent algorithms,” Mathematical Programming, vol. 151, no. 1, pp. 3–34, 2015.
  14. S. Rahnamayan and S. J. Mousavirad, “Towards solving large-scale expensive optimization problems efficiently using coordinate descent algorithm,” in 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC).   IEEE, 2020, pp. 2506–2513.
  15. M. M. Feraidooni, D. Gharavian, M. Alaee-Kerahroodi, and S. Imani, “A coordinate descent framework for probing signal design in cognitive mimo radars,” IEEE Communications Letters, vol. 24, no. 5, pp. 1115–1118, 2020.
  16. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga-ii,” IEEE transactions on evolutionary computation, vol. 6, no. 2, pp. 182–197, 2002.
  17. R. Jiao, B. Xue, and M. Zhang, “Solving multi-objective feature selection problems in classification via problem reformulation and duplication handling,” IEEE Transactions on Evolutionary Computation, 2022.
  18. F. Cheng, J. J. Cui, Q. J. Wang, and L. Zhang, “A variable granularity search based multi-objective feature selection algorithm for high-dimensional data classification,” IEEE Transactions on Evolutionary Computation, 2022.
  19. Z. Wang, S. Gao, M. Zhou, S. Sato, J. Cheng, and J. Wang, “Information-theory-based nondominated sorting ant colony optimization for multiobjective feature selection in classification,” IEEE Transactions on Cybernetics, 2022.
  20. L. He, R. Chiong, W. Li, S. Dhakal, Y. Cao, and Y. Zhang, “Multiobjective optimization of energy-efficient job-shop scheduling with dynamic reference point-based fuzzy relative entropy,” IEEE Transactions on Industrial Informatics, vol. 18, no. 1, pp. 600–610, 2021.
  21. A. A. Bidgoli and S. Rahnamayan, “Memetic differential evolution using coordinate descent,” in 2021 IEEE Congress on Evolutionary Computation (CEC).   IEEE, 2021, pp. 359–366.
  22. E. Frandi and A. Papini, “Coordinate search algorithms in multilevel optimization,” Optimization Methods and Software, vol. 29, no. 5, pp. 1020–1041, 2014.
  23. P. Tseng, “Convergence of a block coordinate descent method for nondifferentiable minimization,” Journal of optimization theory and applications, vol. 109, no. 3, pp. 475–494, 2001.
  24. A. Asilian Bidgoli, S. Rahnamayan, B. Erdem, Z. Erdem, A. Ibrahim, K. Deb, and A. Grami, “Machine learning-based framework to cover optimal pareto-front in many-objective optimization,” Complex & Intelligent Systems, vol. 8, no. 6, pp. 5287–5308, 2022.
  25. A. A. Bidgoli, H. Ebrahimpour-Komleh, and S. Rahnamayan, “Reference-point-based multi-objective optimization algorithm with opposition-based voting scheme for multi-label feature selection,” Information Sciences, vol. 547, pp. 1–17, 2021.
  26. Z. Zhao, F. Morstatter, S. Sharma, S. Alelyani, A. Anand, and H. Liu, “Advancing feature selection research,” ASU feature selection repository, pp. 1–28, 2010.
  27. J. Adamczyk. (2020) Make knn 300 times faster than scikit-learn’s in 20 lines! [Online]. Available: https://towardsdatascience.com/make-knn-300-times-faster-than-scikit-learns-in-20-lines-5e29d74e76bb
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets