Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Bridging Optimal Transport and Jacobian Regularization by Optimal Trajectory for Enhanced Adversarial Defense (2303.11793v3)

Published 21 Mar 2023 in cs.CV

Abstract: Deep neural networks, particularly in vision tasks, are notably susceptible to adversarial perturbations. To overcome this challenge, developing a robust classifier is crucial. In light of the recent advancements in the robustness of classifiers, we delve deep into the intricacies of adversarial training and Jacobian regularization, two pivotal defenses. Our work is the first carefully analyzes and characterizes these two schools of approaches, both theoretically and empirically, to demonstrate how each approach impacts the robust learning of a classifier. Next, we propose our novel Optimal Transport with Jacobian regularization method, dubbed OTJR, bridging the input Jacobian regularization with the a output representation alignment by leveraging the optimal transport theory. In particular, we employ the Sliced Wasserstein distance that can efficiently push the adversarial samples' representations closer to those of clean samples, regardless of the number of classes within the dataset. The SW distance provides the adversarial samples' movement directions, which are much more informative and powerful for the Jacobian regularization. Our empirical evaluations set a new standard in the domain, with our method achieving commendable accuracies of 52.57% on CIFAR-10 and 28.3% on CIFAR-100 datasets under the AutoAttack. Further validating our model's practicality, we conducted real-world tests by subjecting internet-sourced images to online adversarial attacks. These demonstrations highlight our model's capability to counteract sophisticated adversarial perturbations, affirming its significance and applicability in real-world scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Square attack: a query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision (ECCV). Springer, 484–501.
  2. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning. PMLR, 274–283.
  3. Improving adversarial robustness via channel-wise activation suppressing. International Conference on Learning Representations (ICLR) (2021).
  4. Adversarial attack vulnerability of medical image analysis systems: Unexplored factors. Medical Image Analysis (2021), 102141.
  5. Optimal transport as a defense against adversarial attacks. In 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 5044–5051.
  6. A unified wasserstein distributional robustness framework for adversarial training. arXiv preprint arXiv:2202.13437 (2022).
  7. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705 (2019).
  8. Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). IEEE, 39–57.
  9. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).
  10. Jacobian Regularization for Mitigating Universal Adversarial Perturbations. 30th International Conference on Artificial Neural Networks (ICANN) (2021).
  11. Francesco Croce and Matthias Hein. 2020a. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning (ICML). PMLR, 2196–2205.
  12. Francesco Croce and Matthias Hein. 2020b. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning. PMLR, 2206–2216.
  13. Learnable boundary guided adversarial training. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 15721–15730.
  14. Marco Cuturi. 2013. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems (NeurIPS) 26 (2013), 2292–2300.
  15. An analysis of adversarial attacks and defenses on autonomous driving models. In 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1–10.
  16. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 9185–9193.
  17. Statistics of robust optimization: A generalized empirical likelihood approach. Mathematics of Operations Research 46, 3 (2021), 946–969.
  18. Wasserstein distributionally robust optimization and variation regularization. Operations Research (2022).
  19. Simple black-box adversarial attacks. In International Conference on Machine Learning (ICML). PMLR, 2484–2493.
  20. Sigurdur Helgason. 2010. Integral geometry and Radon transforms. Springer Science & Business Media.
  21. Robust learning with Jacobian regularization. arXiv preprint arXiv:1908.02729 (2019).
  22. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).
  23. Daniel Jakubovitz and Raja Giryes. 2018. Improving DNN robustness to adversarial attacks using Jacobian regularization. In Proceedings of the European Conference on Computer Vision (ECCV). 514–529.
  24. Adversarial logit pairing. arXiv preprint arXiv:1803.06373 (2018).
  25. Learning multiple layers of features from tiny images. Technical Report. University of Toronto, Toronto.
  26. Wasserstein distributionally robust optimization: Theory and applications in machine learning. In Operations research & management science in the age of analytics. Informs, 130–166.
  27. Adversarial examples in the physical world.
  28. Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition 110 (2021), 107332.
  29. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations (ICLR) (2018).
  30. Large-scale optimal transport map estimation using projection pursuit. Advances in Neural Information Processing Systems (NeurIPS) (2019).
  31. Online Adversarial Attacks. In International Conference on Learning Representations.
  32. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2574–2582.
  33. Bag of tricks for adversarial training. arXiv preprint arXiv:2010.00467 (2020).
  34. Feature robust optimal transport for high-dimensional data. arXiv preprint arXiv:2005.12123 (2020).
  35. Ning Qian. 1999. On the momentum term in gradient descent learning algorithms. Neural networks 12, 1 (1999), 145–151.
  36. Hamed Rahimian and Sanjay Mehrotra. 2019. Distributionally robust optimization: A review. arXiv preprint arXiv:1908.05659 (2019).
  37. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning. PMLR, 8093–8104.
  38. Decoupling direction and norm for efficient gradient-based l2 adversarial attacks and defenses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4322–4330.
  39. Adversarial training for free! arXiv preprint arXiv:1904.12843 (2019).
  40. Distributionally robust logistic regression. Advances in Neural Information Processing Systems 28 (2015).
  41. François-Xavier Vialard. 2019. An elementary introduction to entropic regularization and proximal methods for numerical optimal transport. (2019).
  42. Cédric Villani. 2008. Optimal transport: old and new. Vol. 338. Springer Science & Business Media.
  43. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems 33 (2020), 2958–2969.
  44. Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 (2016).
  45. Haichao Zhang and Jianyu Wang. 2019. Defense against adversarial attacks using feature scattering-based adversarial training. Advances in Neural Information Processing Systems (NeurIPS) 32 (2019), 1831–1841.
  46. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning (ICML). PMLR, 7472–7482.
  47. Geometry-aware instance-reweighted adversarial training. International Conference on Learning Representations (ICLR) (2020).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now