Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adaptive Batch Normalization Networks for Adversarial Robustness (2405.11708v2)

Published 20 May 2024 in cs.LG and cs.CV

Abstract: Deep networks are vulnerable to adversarial examples. Adversarial Training (AT) has been a standard foundation of modern adversarial defense approaches due to its remarkable effectiveness. However, AT is extremely time-consuming, refraining it from wide deployment in practical applications. In this paper, we aim at a non-AT defense: How to design a defense method that gets rid of AT but is still robust against strong adversarial attacks? To answer this question, we resort to adaptive Batch Normalization (BN), inspired by the recent advances in test-time domain adaptation. We propose a novel defense accordingly, referred to as the Adaptive Batch Normalization Network (ABNN). ABNN employs a pre-trained substitute model to generate clean BN statistics and sends them to the target model. The target model is exclusively trained on clean data and learns to align the substitute model's BN statistics. Experimental results show that ABNN consistently improves adversarial robustness against both digital and physically realizable attacks on both image and video datasets. Furthermore, ABNN can achieve higher clean data performance and significantly lower training time complexity compared to AT-based approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In International Conference on Machine learning (ICML), 2018.
  2. Domain-specific batch normalization for unsupervised domain adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  3. All about structure: Adapting structural information across domains for boosting semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  4. Evaluating the adversarial robustness of adaptive test-time defenses. In International Conference on Machine Learning (ICML), 2022.
  5. Imagenet: A large-scale hierarchical image database. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
  6. Y. Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. In International Conference on Machine learning (ICML), 2015.
  7. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
  8. Countering adversarial images using input transformations. In International Conference on Learning Representations (ICLR), 2018.
  9. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  10. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  11. X. Huang and S. Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In IEEE/CVF International Conference on Computer Vision (ICCV), 2017.
  12. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine learning (ICML), 2015.
  13. The kinetics human action video dataset. In arXiv preprint arXiv:1705.06950, 2017.
  14. K. A. Kinfu and R. Vidal. Analysis and extensions of adversarial training for video classification. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop (CVPRW), 2022.
  15. A. Krizhevsky. Learning multiple layers of features from tiny images. 2009.
  16. Revisiting batch normalization for practical domain adaptation. In International Conference on Learning Representations Workshop (ICLRW), 2017.
  17. Defense against adversarial attacks using high-level representation guided denoiser. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  18. Spatio-temporal pixel-level contrastive learning-based source-free domain adaptation for video semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  19. Defending against multiple and unforeseen adversarial videos. In IEEE Transactions on Image Processing (T-IP), 2021.
  20. Exploring adversarially robust training for unsupervised domain adaptation. In Asian Conference on Computer Vision (ACCV), 2022.
  21. Overcomplete representations against adversarial videos. In IEEE International Conference on Image Processing (ICIP), 2021.
  22. Learning feature decomposition for domain adaptive monocular depth estimation. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022.
  23. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
  24. Pytorch: An imperative style, high-performance deep learning library. In Conference on Neural Information Processing Systems (NeurIPS), 2019.
  25. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR), 2015.
  26. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.
  27. Robustness may be at odds with accuracy. In International Conference on Learning Representations (ICLR), 2019.
  28. On-the-fly test-time adaptation for medical image segmentation. In Medical Imaging with Deep Learning (MIDL), 2023.
  29. Interactive portrait harmonization. In International Conference on Learning Representations (ICLR), 2023.
  30. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations (ICLR), 2021.
  31. Sparse adversarial perturbations for videos. In AAAI Conference on Artificial Intelligence (AAAI), 2019.
  32. Defending against physically realizable attacks on image classification. In International Conference on Learning Representations (ICLR), 2020.
  33. Adversarial examples improve image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  34. C. Xie and A. Yuille. Intriguing properties of sdversarial training at scale. In International Conference on Learning Representations (ICLR), 2020.
  35. Aggregated residual transformations for deep neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  36. Feature squeezing: Detecting adversarial examples in deep neural networks. In Network and Distributed System Security Symposium (NDSS), 2018.
  37. Adversarial robustness through disentangled representations. In AAAI Conference on Artificial Intelligence (AAAI), 2021.
  38. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine learning (ICML), 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Shao-Yuan Lo (23 papers)
  2. Vishal M. Patel (230 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now