Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

An Adaptive Cost-Sensitive Learning and Recursive Denoising Framework for Imbalanced SVM Classification (2403.08378v4)

Published 13 Mar 2024 in cs.CV

Abstract: Category imbalance is one of the most popular and important issues in the domain of classification. Emotion classification model trained on imbalanced datasets easily leads to unreliable prediction. The traditional machine learning method tends to favor the majority class, which leads to the lack of minority class information in the model. Moreover, most existing models will produce abnormal sensitivity issues or performance degradation. We propose a robust learning algorithm based on adaptive cost-sensitivity and recursive denoising, which is a generalized framework and can be incorporated into most stochastic optimization algorithms. The proposed method uses the dynamic kernel distance optimization model between the sample and the decision boundary, which makes full use of the sample's prior information. In addition, we also put forward an effective method to filter noise, the main idea of which is to judge the noise by finding the nearest neighbors of the minority class. In order to evaluate the strength of the proposed method, we not only carry out experiments on standard datasets but also apply it to emotional classification problems with different imbalance rates (IR). Experimental results show that the proposed general framework is superior to traditional methods in Accuracy, G-mean, Recall and F1-score.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
  2. Improved prediction of palmitoylation sites using pwms and svm. Protein and peptide letters, 18(2):186–193, 2011.
  3. Text document preprocessing with the bayes formula for classification using the support vector machine. IEEE Transactions on Knowledge and Data engineering, 20(9):1264–1272, 2008.
  4. Recent advances on support vector machines research. Technological and economic development of Economy, 18(1):5–33, 2012.
  5. Higher rank support tensor machines for visual recognition. Pattern Recognition, 45(12):4192–4203, 2012.
  6. Primal explicit max margin feature selection for nonlinear support vector machines. Pattern recognition, 47(6):2153–2164, 2014.
  7. An iterative svm approach to feature selection and classification in high-dimensional datasets. Pattern Recognition, 46(9):2531–2537, 2013.
  8. Mirpara: a svm-based software tool for prediction of most probable microrna coding regions in genome scale sequences. BMC bioinformatics, 12(1):1–14, 2011.
  9. A locality correlation preserving support vector machine. Pattern Recognition, 47(9):3168–3178, 2014.
  10. A combined svm and lda approach for classification. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 3, pages 1455–1459. IEEE, 2005.
  11. Efficient classification for additive kernel svms. IEEE transactions on pattern analysis and machine intelligence, 35(1):66–77, 2012.
  12. Cutting-plane training of structural svms. Machine learning, 77:27–59, 2009.
  13. A dual coordinate descent method for large-scale linear svm. In Proceedings of the 25th international conference on Machine learning, pages 408–415, 2008.
  14. Neighbors’ distribution property and sample reduction for support vector machines. Applied Soft Computing, 16:201–209, 2014.
  15. A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory, pages 144–152, 1992.
  16. Xuegong Zhang. Using class-center vectors to build support vector machines. In Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No. 98TH8468), pages 3–11. IEEE, 1999.
  17. Large scale online learning. Advances in neural information processing systems, 16, 2003.
  18. Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers, pages 177–186. Springer, 2010.
  19. Applying support vector machines to imbalanced datasets. In Machine Learning: ECML 2004: 15th European Conference on Machine Learning, Pisa, Italy, September 20-24, 2004. Proceedings 15, pages 39–50. Springer, 2004.
  20. Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE transactions on pattern analysis and machine intelligence, 28(7):1088–1099, 2006.
  21. Fuzzy support vector machines. IEEE transactions on neural networks, 13(2):464–471, 2002.
  22. An efficient weighted lagrangian twin support vector machine for imbalanced data classification. Pattern Recognition, 47(9):3158–3167, 2014.
  23. Entropy-based fuzzy support vector machine for imbalanced datasets. Knowledge-Based Systems, 115:87–99, 2017.
  24. A distance-based weighted undersampling scheme for support vector machines and its application to imbalanced classification. IEEE transactions on neural networks and learning systems, 29(9):4152–4165, 2017.
  25. Weighted support vector machine for data classification. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 2, pages 859–864. IEEE, 2005.
  26. Incorporating prior knowledge with weighted margin support vector machines. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 326–333, 2004.
  27. Li Zhang and Wei-Da Zhou. Density-induced margin support vector machines. Pattern Recognition, 44(7):1448–1460, 2011.
  28. A novel weighted support vector machine based on particle swarm optimization for gene selection and tumor classification. Computational and mathematical methods in medicine, 2012, 2012.
  29. A feature selection method based on multiple kernel learning with expression profiles of different types. BioData mining, 10(1):1–16, 2017.
  30. Extended nearest neighbor chain induced instance-weights for svms. Pattern Recognition, 60:863–874, 2016.
  31. Weighted support vector machine using fuzzy rough set theory. Soft Computing, 25(13):8461–8481, 2021.
  32. A stochastic approximation method. The annals of mathematical statistics, pages 400–407, 1951.
  33. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(7), 2011.
  34. Coursera: Neural networks for machine learning. Lecture 9c: Using noise as a regularizer, 2012.
  35. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  36. A stochastic quasi-newton method for online convex optimization. In Artificial intelligence and statistics, pages 436–443. PMLR, 2007.
  37. Global convergence of online limited memory bfgs. The Journal of Machine Learning Research, 16(1):3151–3181, 2015.
  38. A stochastic quasi-newton method for large-scale optimization. SIAM Journal on Optimization, 26(2):1008–1031, 2016.
  39. A stochastic quasi-newton method with nesterov’s accelerated gradient. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I, pages 743–760. Springer, 2020.
  40. A new fast admm for kernelless svm classifier with truncated fraction loss. Knowledge-Based Systems, 283:111214, 2024.
  41. Janez Demšar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 7:1–30, 2006.
  42. A comparison framework of classification models for software defect prediction. Advanced Science Letters, 20(10-11):1945–1950, 2014.
  43. Three-way imbalanced learning based on fuzzy twin svm. Applied Soft Computing, 150:111066, 2024.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com