Papers
Topics
Authors
Recent
Search
2000 character limit reached

Cooperation or Competition: Avoiding Player Domination for Multi-Target Robustness via Adaptive Budgets

Published 27 Jun 2023 in cs.AI, cs.CV, and cs.LG | (2306.15482v1)

Abstract: Despite incredible advances, deep learning has been shown to be susceptible to adversarial attacks. Numerous approaches have been proposed to train robust networks both empirically and certifiably. However, most of them defend against only a single type of attack, while recent work takes steps forward in defending against multiple attacks. In this paper, to understand multi-target robustness, we view this problem as a bargaining game in which different players (adversaries) negotiate to reach an agreement on a joint direction of parameter updating. We identify a phenomenon named player domination in the bargaining game, namely that the existing max-based approaches, such as MAX and MSD, do not converge. Based on our theoretical analysis, we design a novel framework that adjusts the budgets of different adversaries to avoid any player dominance. Experiments on standard benchmarks show that employing the proposed framework to the existing approaches significantly advances multi-target robustness.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pages 274–283, 2018.
  2. Adversarial training and provable defenses: Bridging the gap. In International Conference on Learning Representations, 2020.
  3. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference on Learning Representations, 2018.
  4. Accurate, reliable and fast robustness evaluation. In Advances in Neural Information Processing Systems, pages 12841–12851, 2019.
  5. Towards evaluating the robustness of neural networks. In IEEE symposium on security and privacy, pages 39–57, 2017.
  6. Class-aware robust adversarial training for object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10420–10429, 2021.
  7. Provable robustness against all adversarial lpsubscript𝑙𝑝l_{p}italic_l start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT-perturbations for p≥1𝑝1p\geq 1italic_p ≥ 1. In International Conference on Learning Representations, 2020.
  8. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, pages 2206–2216, 2020.
  9. Imagenet: A large-scale hierarchical image database. In IEEE conference on computer vision and pattern recognition, pages 248–255, 2009.
  10. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12(61):2121–2159, 2011.
  11. A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations. 2018.
  12. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment. Nature communications, 12(1):1–14, 2021.
  13. Fast and reliable evaluation of adversarial robustness with minimum-margin attack. In International Conference on Machine Learning, pages 7144–7163, 2022.
  14. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations, 2015.
  15. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  16. Deep Residual Learning for Image Recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  17. Fooling detection alone is not enough: Adversarial attack against multiple object tracking. In International Conference on Learning Representations, 2020.
  18. Transfer of Adversarial Robustness Between Perturbation Types. In arxiv:1905.01034, 2019.
  19. Adversarial Logit Pairing. In arXiv:1803.06373, 2018.
  20. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
  21. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
  22. Adversarial examples in the physical world. In International Conference on Learning Representations Workshop, 2017.
  23. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  24. Nattack: Learning the distributions of adversarial examples for an improved black-box attack on deep neural networks. In International Conference on Machine Learning, pages 3866–3876, 2019.
  25. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
  26. Adversarial Robustness Against the Union of Multiple Perturbation Models. In International Conference on Machine Learning, pages 6640–6650, 2020.
  27. Logit pairing methods can fool gradient-based attacks. In NeurIPS 2018 Workshop on Security in Machine Learning, 2018.
  28. Mixup inference: Better exploiting mixup to defend adversarial attacks. In International Conference on Learning Representations, 2020.
  29. Bag of tricks for adversarial training. In International Conference on Learning Representations, 2021.
  30. Boosting adversarial training with hypersphere embedding. In Advances in Neural Information Processing Systems, pages 7779–7792, 2020.
  31. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy, pages 582–597, 2016.
  32. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, pages 8024–8035, 2019.
  33. Certified Defenses against Adversarial Examples. In International Conference on Learning Representations, 2022.
  34. A stochastic approximation method. The annals of mathematical statistics, pages 400–407, 1951.
  35. Towards the first adversarially robust neural network model on MNIST. In International conference on Learning Representations, 2019.
  36. Are adversarial examples inevitable? In International Conference on Learning Representations, 2019.
  37. Informative dropout for robust representation learning: A shape-bias perspective. pages 8828–8839, 2020.
  38. Leslie N. Smith. A disciplined approach to neural network hyper-parameters: Part 1 - learning rate, batch size, momentum, and weight decay. In arXiv:1803.09820, 2018.
  39. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
  40. William Thomson. Chapter 35 Cooperative models of bargaining. In Handbook of Game Theory with Economic Applications, volume 2, pages 1237–1284. Elsevier, 1994.
  41. Adversarial Training and Robustness for Multiple Perturbations. In Advances in Neural Information Processing Systems, pages 5858–5868, 2019.
  42. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Representations, 2018.
  43. Robustness May Be at Odds with Accuracy. In International Conference on Learning Representations, 2019.
  44. Once-for-all adversarial training: In-situ tradeoff between robustness and accuracy for free. In Advances in Neural Information Processing Systems, 2020.
  45. Self-ensemble adversarial training for improved robustness. In International Conference on Learning Representations, 2022.
  46. Probabilistic margins for instance reweighting in adversarial training. In Advances in Neural Information Processing Systems, pages 23258–23269, 2021.
  47. Searching privately by imperceptible lying: A novel private hashing method with differential privacy. In ACM International Conference on Multimedia, page 2700–2709, 2020.
  48. An adversarial domain adaptation network for cross-domain fine-grained recognition. In IEEE Winter Conference on Applications of Computer Vision, pages 1217–1225, 2020.
  49. Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope. In International Conference on Machine Learning, pages 5286–5295, 2018.
  50. Adversarial weight perturbation helps robust generalization. In Advances in Neural Information Processing Systems, pages 2958–2969, 2020.
  51. Adversarial weight perturbation improves generalization in graph neural network. arXiv preprint arXiv:2212.04983, 2022.
  52. Retrievalguard: Provably robust 1-nearest neighbor image retrieval. In International Conference on Machine Learning, pages 24266–24279. PMLR, 2022.
  53. Building robust ensembles via margin boosting. In International Conference on Machine Learning, pages 26669–26692, 2022.
  54. You only propagate once: Accelerating adversarial training via maximal principle. In Advances in Neural Information Processing Systems, pages 227–238, 2019.
  55. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pages 7472–7482, 2019.
  56. Attacks which do not kill training make adversarial learning stronger. In International Conference on Machine Learning, 2020.
  57. Adversarial robustness through the lens of causality. In International Conference on Learning Representations, 2022.
  58. Efficient adversarial training with transferable adversarial examples. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1178–1187, 2020.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now