Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

AdCorDA: Classifier Refinement via Adversarial Correction and Domain Adaptation (2401.13212v1)

Published 24 Jan 2024 in cs.CV, cs.AI, and cs.LG

Abstract: This paper describes a simple yet effective technique for refining a pretrained classifier network. The proposed AdCorDA method is based on modification of the training set and making use of the duality between network weights and layer inputs. We call this input space training. The method consists of two stages - adversarial correction followed by domain adaptation. Adversarial correction uses adversarial attacks to correct incorrect training-set classifications. The incorrectly classified samples of the training set are removed and replaced with the adversarially corrected samples to form a new training set, and then, in the second stage, domain adaptation is performed back to the original training set. Extensive experimental validations show significant accuracy boosts of over 5% on the CIFAR-100 dataset. The technique can be straightforwardly applied to refinement of weight-quantized neural networks, where experiments show substantial enhancement in performance over the baseline. The adversarial correction technique also results in enhanced robustness to adversarial attacks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Efficient and effective augmentation strategy for adversarial training. Advances in Neural Information Processing Systems, 35:1488–1501, 2022.
  2. Square attack: A query-efficient black-box adversarial attack via random search. In Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, pages 484–501. Springer, 2020.
  3. Curriculum learning. In International Conference on Machine Learning, pages 41–48, 2009.
  4. Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings in Computational Statistics, pages 177–186. Physica-Verlag HD, 2010.
  5. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, pages 2206–2216. PMLR, 2020a.
  6. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning, pages 2196–2205. PMLR, 2020b.
  7. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  8. The activity-weight duality in feed forward neural networks: The geometric determinants of generalization. arXiv preprint arXiv:2203.10736, 2022.
  9. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
  10. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  11. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2704–2713, 2018.
  12. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
  13. Self-paced learning for latent variable models. Advances in neural information processing systems, 23, 2010.
  14. Adversarial examples in the physical world. In International Conference on Learning Representations. OpenReview.net, 2017.
  15. A review of adversarial attack and defense for classification methods. The American Statistician, 76(4):329–345, 2022.
  16. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.
  17. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
  18. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019.
  19. Foolbox: A python toolbox to benchmark the robustness of machine learning models. In International Conference on Machine Learning Workshop, 2017.
  20. Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX. Journal of Open Source Software, 5(53):2607, 2020.
  21. Decoupling direction and norm for efficient gradient-based l2 adversarial attacks and defenses. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4322–4330, 2019.
  22. Robust learning meets generative models: Can proxy distributions improve adversarial robustness? arXiv preprint arXiv:2104.09425, 2021.
  23. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision, pages 618–626, 2017.
  24. Adversarial training for free! In Advances in Neural Information Processing Systems, pages 3353–3364, 2019.
  25. Fast fine-tuning using curriculum domain adaptation. In Conference on Robots and Vision, 2023.
  26. Deep CORAL: Correlation alignment for deep domain adaptation. In European Conference on Computer Vision Workshop, 2016.
  27. EfficientNetV2: Smaller models and faster training. In International Conference on Machine Learning, pages 10096–10106. PMLR, 2021.
  28. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):4555–4576, 2021.
  29. How transferable are features in deep neural networks? Advances in Neural Information Processing Systems, 27, 2014.
  30. Youshan Zhang. A survey of unsupervised domain adaptation for visual recognition. arXiv preprint arXiv:2112.06745, 2021.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets