The Adversarial Consistency of Surrogate Risks for Binary Classification (2305.09956v3)
Abstract: We study the consistency of surrogate risks for robust binary classification. It is common to learn robust classifiers by adversarial training, which seeks to minimize the expected $0$-$1$ loss when each example can be maliciously corrupted within a small ball. We give a simple and complete characterization of the set of surrogate loss functions that are \emph{consistent}, i.e., that can replace the $0$-$1$ loss without affecting the minimizing sequences of the original adversarial risk, for any data distribution. We also prove a quantitative version of adversarial consistency for the $\rho$-margin loss. Our results reveal that the class of adversarially consistent surrogates is substantially smaller than in the standard setting, where many common surrogates are known to be consistent.
- Calibration and consistency of adversarial surrogate losses. NeurIps, 2021a.
- On the existence of the adversarial bayes classifier (extended version). arxiv, 2021b.
- A finer calibration analysis for adversarial robustness. arxiv, 2021c.
- H-consistency bounds for surrogate loss minimizers. In Proceedings of the 39th International Conference on Machine Learning. PMLR, 2022.
- Calibrated surrogate losses for adversarially robust classification. arxiv, 2021.
- Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473), 2006.
- On the difficulty of approximately maximizing agreements. Journal of Computer System Sciences, 2003.
- R. Bhattacharjee and K. Chaudhuri. When are non-parametric methods robust? PMLR, 2020.
- R. Bhattacharjee and K. Chaudhuri. Consistent non-parametric methods for maximizing robustness. NeurIps, 2021.
- Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pages 387–402. Springer, 2013.
- The geometry of adversarial training in binary classification. arxiv, 2021.
- G. B. Folland. Real analysis: modern techniques and their applications, volume 40. John Wiley & Sons, 1999.
- N. S. Frank and J. Niles-Weed. Existence and minimax theorems for adversarial surrogate risks in binary classification. arXiv, 2023.
- Adaptive square attack: Fooling autonomous cars with adversarial traffic signs. IEEE Internet of Things Journal, 8(8), 2021.
- Y. Lin. A note on margin-based loss functions in classification. Statistics & Probability Letters, 68(1):73–82, 2004.
- Cross-entropy loss functions: Theoretical analysis and applications, 2023.
- Towards consistency in adversarial classification. arXiv, 2022.
- S. A. Mingyuan Zhang. Consistency vs. h-consistency: The interplay between surrogate loss functions and the scoring function class. NeurIps, 2020.
- Generalizability vs. robustness: Adversarial examples for medical imaging. Springer, 2018.
- R. A. S. Philip M. Long. Consistency versus realizable h-consistency for multiclass classification. ICML, 2013.
- M. S. Pydi and V. Jog. The many faces of adversarial risk. Neural Information Processing Systems, 2021.
- Surrogate regret bounds for proper losses. In Proceedings of the 26th Annual International Conference on Machine Learning, New York, NY, USA, 2009. Association for Computing Machinery.
- I. Steinwart. How to compare different loss functions and their risks. Constructive Approximation, 2007.
- Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
- N. G. Trillos and R. Murray. Adversarial classification: Necessary conditions and geometric flows. arxiv, 2020.
- The multimarginal optimal transport formulation of adversarial multiclass classification. arXiv, 2022.
- On the existence of solutions to adversarial training in multiclass classification, 2023.
- T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. The Annals of Statistics, 2004.