Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Optimal Randomized Strategies in Adversarial Example Game (2306.16738v1)

Published 29 Jun 2023 in cs.LG, cs.CR, and cs.GT

Abstract: The vulnerability of deep neural network models to adversarial example attacks is a practical challenge in many artificial intelligence applications. A recent line of work shows that the use of randomization in adversarial training is the key to find optimal strategies against adversarial example attacks. However, in a fully randomized setting where both the defender and the attacker can use randomized strategies, there are no efficient algorithm for finding such an optimal strategy. To fill the gap, we propose the first algorithm of its kind, called FRAT, which models the problem with a new infinite-dimensional continuous-time flow on probability distribution spaces. FRAT maintains a lightweight mixture of models for the defender, with flexibility to efficiently update mixing weights and model parameters at each iteration. Furthermore, FRAT utilizes lightweight sampling subroutines to construct a random strategy for the attacker. We prove that the continuous-time limit of FRAT converges to a mixed Nash equilibria in a zero-sum game formed by a defender and an attacker. Experimental results also demonstrate the efficiency of FRAT on CIFAR-10 and CIFAR-100 datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Recent Advances in Adversarial Training for Adversarial Robustness. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, 4312–4321.
  2. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, 387–402. Springer.
  3. A generalized conditional gradient method for nonlinear operator equations with sparsity constraints. Inverse Problems, 23(5): 2041.
  4. Adversarial example games. Advances in neural information processing systems, 33: 8921–8934.
  5. A generalized conditional gradient method and its connection to an iterative shrinkage method. Computational Optimization and applications, 42(2): 173–193.
  6. Sampling from a log-concave distribution with projected langevin monte carlo. Discrete & Computational Geometry, 59(4): 757–783.
  7. Randomized prediction games for adversarial machine learning. IEEE transactions on neural networks and learning systems, 28(11): 2466–2478.
  8. Chizat, L. 2022. Sparse optimization on measures with over-parameterized gradient descent. Mathematical Programming, 194(1): 487–532.
  9. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, 1310–1320. PMLR.
  10. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, 2206–2216. PMLR.
  11. Stochastic Activation Pruning for Robust Adversarial Defense. In International Conference on Learning Representations.
  12. A mean-field analysis of two-player zero-sum games. Advances in neural information processing systems, 33: 20215–20226.
  13. Explaining and harnessing adversarial examples. In International Conference on Learning Representations.
  14. Improving robustness using generated data. Advances in Neural Information Processing Systems, 34: 4218–4233.
  15. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
  16. On the global convergence of stochastic fictitious play. Econometrica, 70(6): 2265–2294.
  17. Finding mixed nash equilibria of generative adversarial networks. In International Conference on Machine Learning, 2810–2819. PMLR.
  18. Learning multiple layers of features from tiny images. Citeseer.
  19. The continuous logit dynamic and price dispersion. Institute of Mathematical Economics Working Paper, (521).
  20. The logit dynamic for games with continuous strategy sets. Games and Economic Behavior, 91: 268–282.
  21. Optimal entropy-transport problems and a new Hellinger–Kantorovich distance between positive measures. Inventiones mathematicae, 211(3): 969–1117.
  22. Infinite-Dimensional Optimization for Zero-Sum Games via Variational Transport. In International Conference on Machine Learning, 7033–7044. PMLR.
  23. Liu, Q. 2017. Stein variational gradient descent as gradient flow. Advances in neural information processing systems, 30.
  24. Sliced-Wasserstein flows: Nonparametric generative modeling via optimal transport and diffusions. In International Conference on Machine Learning, 4104–4113. PMLR.
  25. Provably convergent quasistatic dynamics for mean-field two-player zero-sum games. In International Conference on Learning Representations.
  26. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations.
  27. Adversarial robustness against the union of multiple perturbation models. In International Conference on Machine Learning, 6640–6650. PMLR.
  28. Mixed nash equilibria in the adversarial examples game. In International Conference on Machine Learning, 7677–7687. PMLR.
  29. Robustness via curvature regularization, and vice versa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9078–9086.
  30. Bag of Tricks for Adversarial Training. In International Conference on Learning Representations.
  31. Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (SP), 582–597. IEEE.
  32. Robust attacks against multiple classifiers. arXiv preprint arXiv:1906.02816.
  33. Stochastic fictitious play with continuous action sets. Journal of Economic Theory, 152: 179–213.
  34. Randomization matters how to defend against strong adversarial attacks. In International Conference on Machine Learning, 7717–7727. PMLR.
  35. Theoretical evidence for adversarial robustness through randomization. Advances in Neural Information Processing Systems, 32.
  36. Global convergence of neuron birth-death dynamics. In International Conference on Machine Learning.
  37. Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models. In International Conference on Learning Representations.
  38. Towards the first adversarially robust neural network model on MNIST. In International Conference on Learning Representations.
  39. Understanding adversarial training: Increasing local stability of supervised models through robust optimization. Neurocomputing, 307: 195–204.
  40. A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights. Advances in neural information processing systems, 27.
  41. Follow the perturbed leader: Optimism and fast parallel algorithms for smooth minimax games. Advances in Neural Information Processing Systems, 33: 22316–22326.
  42. Intriguing properties of neural networks. In International Conference on Learning Representations.
  43. Resnets ensemble via the feynman-kac formalism to improve natural and robust accuracies. Advances in Neural Information Processing Systems, 32.
  44. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11), 681–688. Citeseer.
  45. Mitigating Adversarial Effects Through Randomization. In International Conference on Learning Representations.
  46. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, 7472–7482. PMLR.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now