Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification (2206.09098v4)

Published 18 Jun 2022 in cs.LG, math.ST, and stat.TH

Abstract: Adversarial training is one of the most popular methods for training methods robust to adversarial attacks, however, it is not well-understood from a theoretical perspective. We prove and existence, regularity, and minimax theorems for adversarial surrogate risks. Our results explain some empirical observations on adversarial robustness from prior work and suggest new directions in algorithm development. Furthermore, our results extend previously known existence and minimax theorems for the adversarial classification risk to surrogate risks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Calibration and consistency of adversarial surrogate losses. NeurIps, 2021a.
  2. On the existence of the adversarial bayes classifier (extended version). arxiv, 2021b.
  3. Calibrated surrogate losses for adversarially robust classification. arxiv, 2021.
  4. V. Barbu and T. Precupanu. Convexity and Optimization in Banach Spaces. Springer Monographs in Mathematics, 4th edition, 2012.
  5. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473), 2006.
  6. On the difficulty of approximately maximizing agreements. Journal of Computer System Sciences, 2003.
  7. Stochastic Optimal Control: The Discrete-Time Case. Athena Scientific, 1996.
  8. Lower bounds on adversarial robustness from optimal transport. In Advances in Neural Information Processing Systems, pages 7498–7510, 2019.
  9. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pages 387–402. Springer, 2013.
  10. V. I. Bogachev. Measure Theory, volume II. Springer, 2007.
  11. The geometry of adversarial training in binary classification. arxiv, 2021.
  12. Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks. CoRR, 2018.
  13. A mean-field analysis of two-player zero-sum games, 2021.
  14. G. B. Folland. Real analysis: modern techniques and their applications, volume 40. John Wiley & Sons, 1999.
  15. N. S. Frank and J. Niles-Weed. The adversarial consistency of surrogate risks for binary classification. arxiv, 2023.
  16. Wasserstein distributionally robust optimization and variation regularization, 2022.
  17. Explaining and harnessing adversarial examples. ICLR, 2014.
  18. Badnets: Identifying vulnerabilities in the machine learning model supply chain. CoRR, 2017.
  19. H. Jylhä. The l∞superscript𝑙l^{\infty}italic_l start_POSTSUPERSCRIPT ∞ end_POSTSUPERSCRIPT optimal transport: Infinite cyclical monotonicity and the existence of optimal transport maps. Calculus of Variations and Partial Differential Equations, 2014.
  20. Adversarial logit pairing. CoRR, 2018.
  21. Adversarial machine learning at scale. ICLR, 2017.
  22. J. D. Li and M. Telgarsky. On achieving optimal adversarial test error, 2023.
  23. Y. Lin. A note on margin-based loss functions in classification. Statistics & Probability Letters, 68(1):73–82, 2004.
  24. Towards deep learning models resistant to adversarial attacks. ICLR, 2019.
  25. Towards consistency in adversarial classification. arXiv, 2022.
  26. S. A. Mingyuan Zhang. Consistency vs. h-consistency: The interplay between surrogate loss functions and the scoring function class. NeurIps, 2020.
  27. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach, 2019.
  28. Practical black-box attacks against deep learning systems using adversarial examples. CoRR, abs/1602.02697, 2016. URL http://arxiv.org/abs/1602.02697.
  29. R. A. S. Philip M. Long. Consistency versus realizable h-consistency for multiclass classification. ICML, 2013.
  30. M. S. Pydi and V. Jog. Adversarial risk via optimal transport and optimal couplings. arXiv preprint arXiv:1912.02794, 2019.
  31. M. S. Pydi and V. Jog. The many faces of adversarial risk. Neural Information Processing Systems, 2021.
  32. Are accuracy and robustness correlated? CoRR, 2016.
  33. F. Santambrogio. Optimal Transport for Applied Mathematicians. Birkhäuser, 1st edition, 2015.
  34. Adversarial training for free! CoRR, 2019.
  35. I. Steinwart. How to compare different loss functions and their risks. Constructive Approximation, 2007.
  36. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
  37. A. Tewari and P. L. Bartlett. On the consistency of multiclass classification methods. Journal of Machine Learning Research, 8(36), 2007.
  38. The space of transferable adversarial examples. arXiv, 2017.
  39. On adversarial robustness and the use of wasserstein ascent-descent dynamics to enforce it, 2023.
  40. N. G. Trillos and R. Murray. Adversarial classification: Necessary conditions and geometric flows. arxiv, 2020.
  41. The multimarginal optimal transport formulation of adversarial multiclass classification. arXiv, 2022.
  42. C. Villani. Topics in Optimal Transportation. American Mathematical Society, 2nd edition, 2003.
  43. G. Wang and L. Chizat. An exponentially converging particle method for the mixed nash equilibrium of continuous games, 2023.
  44. On the convergence and robustness of adversarial training. ICML, 2021.
  45. Wasserstein adversarial examples via projected Sinkhorn iterations. In Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR, 2019.
  46. Fast is better than free: Revisiting adversarial training. CoRR, abs/2001.03994, 2020.
  47. Stronger and faster wasserstein adversarial attacks, 2020.
  48. Feature denoising for improving adversarial robustness. CoRR, 2018.
  49. Robustness for non-parametric classification: A generic attack and defense. Proceedings of Machine Learning Research, 2020.
  50. T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. The Annals of Statistics, 2004.
Citations (13)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now