Papers
Topics
Authors
Recent
Search
2000 character limit reached

EXACT: How to Train Your Accuracy

Published 19 May 2022 in cs.LG and cs.CV | (2205.09615v5)

Abstract: Classification tasks are usually evaluated in terms of accuracy. However, accuracy is discontinuous and cannot be directly optimized using gradient ascent. Popular methods minimize cross-entropy, hinge loss, or other surrogate losses, which can lead to suboptimal results. In this paper, we propose a new optimization framework by introducing stochasticity to a model's output and optimizing expected accuracy, i.e. accuracy of the stochastic model. Extensive experiments on linear models and deep image classification show that the proposed optimization method is a powerful alternative to widely used classification losses.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. An ensemble of simple convolutional neural network models for mnist digit recognition. arXiv preprint arXiv:2008.10400, 2020.
  2. Tabnet: Attentive interpretable tabular learning. In AAAI, volume 35, pp.  6679–6687, 2021.
  3. sigmoidf1: A smooth f1 score surrogate loss for multilabel classification. arXiv preprint arXiv:2108.10566, 2021.
  4. Bishop, C. M. Mixture density networks. 1994.
  5. A limited memory algorithm for bound constrained optimization. SIAM Journal on scientific computing, 16(5):1190–1208, 1995.
  6. On the (non-) existence of convex, calibrated surrogate losses for ranking. Advances in Neural Information Processing Systems, 25, 2012.
  7. Efficient auc optimization for classification. In European conference on principles of data mining and knowledge discovery, pp.  42–53. Springer, 2007.
  8. Data uncertainty learning in face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  5710–5719, 2020.
  9. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  113–123, 2019.
  10. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml.
  11. Multi-atlas classification of autism spectrum disorder with hinge loss trained deep architectures: Abide i results. Applied Soft Computing, 107:107375, 2021. ISSN 1568-4946. doi: https://doi.org/10.1016/j.asoc.2021.107375. URL https://www.sciencedirect.com/science/article/pii/S1568494621002982.
  12. Linear hinge loss and average margin. Advances in neural information processing systems, 11, 1998.
  13. Genz, A. Numerical computation of multivariate normal probabilities. Journal of computational and graphical statistics, 1(2):141–149, 1992.
  14. Bayesian neural networks: An introduction and survey. In Case Studies in Applied Bayesian Data Science, pp. 45–87. Springer, 2020.
  15. Learning surrogate losses. arXiv preprint arXiv:1905.10108, 2019.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  17. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448–456. PMLR, 2015.
  18. Traffic sign recognition with hinge loss trained convolutional neural networks. IEEE transactions on intelligent transportation systems, 15(5):1991–2000, 2014.
  19. Deep metric learning: A survey. Symmetry, 11(9):1066, 2019.
  20. Adam: A method for stochastic optimization. In ICLR (Poster), 2015.
  21. Auto-encoding variational bayes. In Bengio, Y. and LeCun, Y. (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
  22. Learning multiple layers of features from tiny images. 2009.
  23. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
  24. Asam: Adaptive sharpness-aware minimization for scale-invariant learning of deep neural networks. In International Conference on Machine Learning, pp. 5905–5914. PMLR, 2021.
  25. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  26. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pp.  142–150, 2011.
  27. Machine learning, volume 1, pp.  170. McGraw-hill New York, 1997a.
  28. Machine learning, volume 1, pp.  175. McGraw-hill New York, 1997b.
  29. Reading digits in natural images with unsupervised feature learning. 2011.
  30. Levenberg–marquardt multi-classification using hinge loss function. Neural Networks, 143:564–571, 2021. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2021.07.010. URL https://www.sciencedirect.com/science/article/pii/S0893608021002732.
  31. Pytorch: An imperative style, high-performance deep learning library. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 32, pp.  8024–8035. Curran Associates, Inc., 2019a.
  32. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019b.
  33. Recall@ k surrogate loss with large batches and similarity mixup. arXiv preprint arXiv:2108.11179, 2021.
  34. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  35. Discriminative feature selection via employing smooth and robust hinge loss. IEEE Transactions on Neural Networks and Learning Systems, 30(3):788–802, 2018.
  36. Probabilistic face embeddings. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  6902–6911, 2019.
  37. A comprehensive survey of loss functions in machine learning. Annals of Data Science, 9(2):187–212, 2022.
  38. Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research, 10(2), 2009.
  39. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256, 1992.
  40. Large-scale robust deep auc maximization: A new surrogate loss and empirical studies on medical image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  3040–3049, 2021.
  41. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.
  42. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up