Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective (2302.09457v2)

Published 19 Feb 2023 in cs.LG and cs.CR

Abstract: Adversarial machine learning (AML) studies the adversarial phenomenon of machine learning, which may make inconsistent or unexpected predictions with humans. Some paradigms have been recently developed to explore this adversarial phenomenon occurring at different stages of a machine learning system, such as backdoor attack occurring at the pre-training, in-training and inference stage; weight attack occurring at the post-training, deployment and inference stage; adversarial attack occurring at the inference stage. However, although these adversarial paradigms share a common goal, their developments are almost independent, and there is still no big picture of AML. In this work, we aim to provide a unified perspective to the AML community to systematically review the overall progress of this field. We firstly provide a general definition about AML, and then propose a unified mathematical framework to covering existing attack paradigms. According to the proposed unified framework, we build a full taxonomy to systematically categorize and review existing representative methods for each paradigm. Besides, using this unified framework, it is easy to figure out the connections and differences among different attack paradigms, which may inspire future researchers to develop more advanced attack paradigms. Finally, to facilitate the viewing of the built taxonomy and the related literature in adversarial machine learning, we further provide a website, \ie, \url{http://adversarial-ml.com}, where the taxonomies and literature will be continuously updated.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (143)
  1. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In USENIX, pages 1615–1631, 2018.
  2. Are image-agnostic universal adversarial perturbations for face recognition difficult to detect? In BTAS. IEEE, 2018.
  3. How to flip a bit? In IOLTS, 2010.
  4. Discrete cosine transform. IEEE Transactions on Computers, 100(1):90–93, 1974.
  5. Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access, 6:14410–14430, 2018.
  6. Sign bits are all you need for black-box attacks. In ICLR, 2019.
  7. Square attack: a query-efficient black-box adversarial attack via random search. In ECCV, 2020.
  8. Donovan Artz. Digital steganography: hiding data within data. IEEE Internet Computing, 5(3):75–80, 2001.
  9. Synthesizing robust adversarial examples. In ICML, 2018.
  10. How to backdoor federated learning. In AISTATS, 2020.
  11. Versatile weight attack via flipping limited bits. arXiv preprint arXiv:2207.12405, 2022.
  12. Targeted attack against deep neural networks via flipping limited weight bits. In ICLR, 2021.
  13. Shumeet Baluja. Hiding images in plain sight: Deep steganography. In NeurIPS, 2017.
  14. A new backdoor attack in cnns by training set corruption without label poisoning. In ICIP, 2019.
  15. Analyzing federated learning through an adversarial lens. In ICML, 2019.
  16. Pattern Recognition and Machine Learning. Springer, 2006.
  17. Architectural backdoors in neural networks. In CVPR, 2023.
  18. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning, 3(1):1–122, 2011.
  19. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In ICLR, 2018.
  20. Improving the transferability of targeted adversarial examples through object-based diverse input. In CVPR, 2022.
  21. Badprompt: backdoor attacks on continuous prompts. In NeurIPS, 2022.
  22. Black-box attacks via surrogate ensemble search. In NeurIPS, 2022.
  23. Towards evaluating the robustness of neural networks. In IEEE S&P, 2017.
  24. Audio adversarial examples: Targeted attacks on speech-to-text. In IEEE S&P Workshops, 2018.
  25. Uday K Chakraborty. Advances in differential evolution, volume 143. Springer, 2008.
  26. Poison attacks against text datasets with conditional adversarially regularized autoencoder. In Findings of EMNLP, 2020.
  27. Finding optimal least-significant-bit substitution in image hiding by dynamic programming strategy. Pattern Recognition, 36(7):1583–1595, 2003.
  28. Rowback: Robust watermarking for neural networks using backdoors. In ICMLA, 2021.
  29. Backdoor attacks on federated meta-learning. In NeurIPS, 2020.
  30. Proflip: Targeted trojan attack with progressive bit flips. In ICCV, 2021.
  31. Attacking visual language grounding with adversarial examples: A case study on neural image captioning. In ACL, 2018.
  32. Rays: A ray searching method for hard-label adversarial attack. In ACM SIGKDD, 2020.
  33. Hopskipjumpattack: A query-efficient decision-based attack. In IEEE S&P, pages 1277–1294. IEEE, 2020.
  34. Mag-gan: Massive attack generator via gan. Information Sciences, 536:67–90, 2020.
  35. The dark side of dynamic routing neural networks: Towards efficiency backdoor injection. In CVPR, 2023.
  36. Adversarial attack on attackers: Post-process to mitigate black-box score-based query attacks. In NeurIPS, 2022.
  37. Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector. In ECML PKDD, 2018.
  38. Trojdiff: Trojan attacks on diffusion models with diverse targets. In CVPR, 2023.
  39. Effective backdoor defense by exploiting sensitivity of poisoned samples. In NeurIPS, 2022.
  40. Boosting decision-based black-box adversarial attacks with random sign flip. In ECCV, 2020.
  41. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017.
  42. Badnl: Backdoor attacks against nlp models with semantic-preserving improvements. In ACSAC, 2021.
  43. Query-efficient hard-label black-box attack: An optimization-based approach. In ICLR, 2019.
  44. Improving black-box adversarial attacks with a transfer-based prior. In NeurIPS, 2019.
  45. Deep feature space trojan attack of neural networks by controlled detoxification. In AAAI, 2021.
  46. How to backdoor diffusion models? In CVPR, 2023.
  47. Villandiffusion: A unified backdoor attack framework for diffusion models. In NeurIPS, 2023.
  48. Robustbench: a standardized adversarial robustness benchmark. In NeurIPS Datasets and Benchmarks Track, 2021.
  49. Sparse and imperceivable adversarial attacks. In ICCV, 2019.
  50. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020.
  51. A unified evaluation of textual backdoor learning: Frameworks and benchmarks. In NeurIPS Datasets and Benchmarks Track, 2022.
  52. Advfaces: Adversarial face synthesis. In IJCB. IEEE, 2019.
  53. AdverTorch v0.1: An adversarial robustness toolbox based on pytorch. arXiv preprint arXiv:1902.07623, 2019.
  54. Privacy-preserving feature extraction via adversarial training. IEEE Transactions on Knowledge and Data Engineering, 34(4):1967–1979, 2020.
  55. Backdoor attack with imperceptible input and latent modification. In NeurIPS, 2021.
  56. Lira: Learnable, imperceptible and robust backdoor attacks. In ICCV, 2021.
  57. Marksman backdoor: Backdoor attacks with arbitrary target class. In NeurIPS, 2022.
  58. Query-efficient black-box adversarial attacks guided by a transfer-based prior. TPAMI, 44(12):9536–9548, 2022.
  59. Boosting adversarial attacks with momentum. In CVPR, 2018.
  60. Evading defenses to transferable adversarial examples by translation-invariant attacks. In CVPR, 2019.
  61. Efficient decision-based black-box adversarial attacks on face recognition. In CVPR, 2019.
  62. John R Douceur. The sybil attack. In International workshop on peer-to-peer systems. Springer, 2002.
  63. Query-efficient meta attack to deep neural networks. In ICLR, 2020.
  64. Ppt: Backdoor attacks on pre-trained models via poisoned prompt tuning. In IJCAI, 2022.
  65. Adversarial camouflage: Hiding physical-world attacks with natural styles. In CVPR, 2020.
  66. Exploring the landscape of spatial robustness. In ICML, 2019.
  67. Physical adversarial examples for object detectors. In USENIX Conference on Offensive Technologies, 2018.
  68. Robust physical-world attacks on deep learning visual classification. In CVPR, 2018.
  69. Sparse adversarial attack via perturbation factorization. In ECCV, 2020.
  70. Manitest: Are classifiers really invariant? In BMVC, 2015.
  71. Maximizing non-monotone submodular functions. SIAM Journal on Computing, 40(4):1133–1153, 2011.
  72. Stealthy backdoor attack with adversarial training. In ICASSP, 2022.
  73. Meta-attack: Class-agnostic and model-agnostic physical adversarial attack. In ICCV, 2021.
  74. Fiba: Frequency-injection based backdoor attack in medical image analysis. In CVPR, 2022.
  75. Boosting black-box attack with partially transferred conditional adversarial distribution. In CVPR, 2022.
  76. The limitations of federated learning in sybil settings. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses, 2020.
  77. Imperceptible and robust backdoor attack in 3d point cloud. arXiv preprint arXiv:2208.08052, 2022.
  78. Backdoor attacks and countermeasures on deep learning: A comprehensive review. arXiv preprint arXiv:2007.10760, 2020.
  79. Can adversarial weight perturbations inject neural backdoors. In ACM CIKM, 2020.
  80. Image style transfer using convolutional neural networks. In CVPR, pages 2414–2423, 2016.
  81. Stealthy attack on algorithmic-protected dnns via smart bit flipping. In ISQED, 2022.
  82. Defense-resistant backdoor attacks against deep neural networks in outsourced cloud environment. IEEE Journal on Selected Areas in Communications, 2021.
  83. Generative adversarial nets. In NIPS, 2014.
  84. Explaining and harnessing adversarial examples. In ICLR, 2015.
  85. Advbox: a toolbox to generate adversarial examples that fool neural networks, 2020.
  86. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 2019.
  87. Simple black-box adversarial attacks. In ICML, 2019.
  88. Backpropagating linearly improves transferability of adversarial examples. In NeurIPS, 2020.
  89. Subspace attack: Exploiting promising subspaces for query-efficient black-box attacks. In NeurIPS, 2019.
  90. Physical backdoor attacks to lane detection systems in autonomous driving. In ACM Multimedia, 2022.
  91. Few-shot backdoor attacks via neural tangent kernels. In ICLR, 2022.
  92. Handcrafted backdoors in deep neural networks. In NeurIPS, 2022.
  93. Membership inference via backdooring. arXiv preprint arXiv:2206.04823, 2022.
  94. Backdoor defense via decoupling the training process. In ICLR, 2022.
  95. Universal physical camouflage attacks on object detectors. In CVPR, 2020.
  96. Enhancing adversarial example transferability with an intermediate level attack. In ICCV, 2019.
  97. Black-box adversarial attack with transferable model-based embedding. In ICLR, 2020.
  98. Black-box adversarial attacks with limited queries and information. In ICML, 2018.
  99. Prior convictions: Black-box adversarial attacks with bandits and priors. In ICLR, 2019.
  100. Transferable perturbations of deep feature distributions. In ICLR, 2020.
  101. Perturbing across the feature hierarchy to improve standard and strict blackbox attack transferability. In NeurIPS, 2020.
  102. Connecting the digital and physical world: Improving the robustness of adversarial attacks. In AAAI, volume 33, 2019.
  103. Label poisoning is all you need. In NeurIPS, 2023.
  104. Las-at: Adversarial training with learnable attack strategy. In CVPR, 2022.
  105. Color backdoor: A robust poisoning attack in color space. In CVPR, 2023.
  106. Adversarial image perturbation for privacy protection–a game theory perspective. In Proceedings of the IEEE International Conference on Computer Vision, pages 1482–1491, 2017.
  107. Geometric robustness of deep networks: analysis and improvement. In CVPR, 2018.
  108. Lavan: Localized and visible adversarial noise. In ICML, 2018.
  109. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, 2019.
  110. Flipping bits in memory without accessing them: An experimental study of dram disturbance errors. ACM SIGARCH Computer Architecture News, 42(3):361–372, 2014.
  111. Advhat: Real-world adversarial attack on arcface face id system. In ICPR, 2021.
  112. Federated learning: Strategies for improving communication efficiency. In NIPS Workshop on Private Multi-Party Machine Learning, 2016.
  113. Physgan: Generating physical-world-resilient adversarial examples for autonomous driving. In CVPR, 2020.
  114. Adversarial examples in the physical world. In Artificial intelligence safety and security, pages 99–112. Chapman and Hall/CRC, 2018.
  115. Functional adversarial attacks. In NeurIPS, 2019.
  116. Universal adversarial perturbations against object detection. Pattern Recognition, 110:107584, 2021.
  117. Light can hack your face! black-box backdoor attack on face recognition systems. arXiv preprint arXiv:2009.06996, 2020.
  118. Qeba: Query-efficient boundary-based blackbox attack. In CVPR, 2020.
  119. Universal perturbation attack against image retrieval. In ICCV, 2019.
  120. Projection & probability-driven black-box attack. In CVPR, 2020.
  121. Backdoor attacks on pre-trained models by layerwise weight poisoning. In EMNLP, 2021.
  122. Towards transferable targeted attack. In CVPR, 2020.
  123. Hidden backdoors in human-centric language models. In ACM CCS, 2021.
  124. Invisible backdoor attacks on deep neural networks via steganography and regularization. TDSC, 2021.
  125. Compressing convolutional neural networks via factorized convolutional filters. In CVPR, 2019.
  126. Turning attacks into protection: Social media privacy protection using adversarial attacks. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pages 208–216. SIAM, 2021.
  127. Pointba: Towards backdoor attacks in 3d point cloud. In ICCV, 2021.
  128. Learning transferable adversarial examples via ghost networks. In AAAI, 2020.
  129. Untargeted backdoor watermark: Towards harmless and stealthy dataset copyright protection. NeurIPS, 2022.
  130. Deeprobust: A pytorch library for adversarial attacks and defenses. arXiv preprint arXiv:2005.06149, 2020.
  131. Nattack: Learning the distributions of adversarial examples for an improved black-box attack on deep neural networks. In ICML, 2019.
  132. Invisible backdoor attack with sample-specific triggers. In ICCV, 2021.
  133. Open-sourced dataset protection via backdoor watermarking. In NeurIPS Workshop on Dataset Curation and Security, 2020.
  134. Black-box dataset ownership verification via backdoor watermarking. IEEE TIFS, 2023.
  135. A proxy-free strategy for practically improving the poisoning efficiency in backdoor attacks. arXiv preprint arXiv:2306.08313, 2023.
  136. Explore the effect of data selection on poison efficiency in backdoor attacks. arXiv preprint arXiv:2310.09744, 2023.
  137. Parallel rectangle flip attack: A query-based black-box attack against object detection. In ICCV, 2021.
  138. Nesterov accelerated gradient and scale invariance for adversarial attacks. In ICLR, 2020.
  139. Composite backdoor attack for deep neural network by mixing existing benign features. In ACM CCS, pages 113–131, 2020.
  140. Deepsec: A uniform platform for security analysis of deep learning model. In 2019 IEEE Symposium on Security and Privacy (SP), pages 673–690. IEEE, 2019.
  141. Perceptual-sensitive gan for generating adversarial patches. In AAAI, 2019.
  142. Universal adversarial perturbation via prior driven uncertainty approximation. In ICCV, 2019.
  143. Greedyfool: Distortion-aware sparse adversarial attack. In NeurIPS, 2020.
Citations (17)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets