Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

BadActs: A Universal Backdoor Defense in the Activation Space (2405.11227v1)

Published 18 May 2024 in cs.CR and cs.CL

Abstract: Backdoor attacks pose an increasingly severe security threat to Deep Neural Networks (DNNs) during their development stage. In response, backdoor sample purification has emerged as a promising defense mechanism, aiming to eliminate backdoor triggers while preserving the integrity of the clean content in the samples. However, existing approaches have been predominantly focused on the word space, which are ineffective against feature-space triggers and significantly impair performance on clean data. To address this, we introduce a universal backdoor defense that purifies backdoor samples in the activation space by drawing abnormal activations towards optimized minimum clean activation distribution intervals. The advantages of our approach are twofold: (1) By operating in the activation space, our method captures from surface-level information like words to higher-level semantic concepts such as syntax, thus counteracting diverse triggers; (2) the fine-grained continuous nature of the activation space allows for more precise preservation of clean content while removing triggers. Furthermore, we propose a detection module based on statistical information of abnormal activations, to achieve a better trade-off between clean accuracy and defending performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Omer Antverg and Yonatan Belinkov. 2022. On the pitfalls of analyzing individual neurons in language models. In International Conference on Learning Representations.
  2. T-miner: A generative approach to defend against trojan attacks on dnn-based text classification. In USENIX Security Symposium.
  3. Eugene Bagdasaryan and Vitaly Shmatikov. 2022. Spinning language models: Risks of propaganda-as-a-service and countermeasures. In S&P.
  4. Language models are few-shot learners. In NIPS.
  5. Badprompt: Backdoor attacks on continuous prompts. In NeurIPS.
  6. Chuanshuai Chen and Jiazhu Dai. 2021. Mitigating backdoor attacks in lstm-based text classification systems by backdoor keyword identification. Neurocomputing.
  7. Holistic sentence embeddings for better out-of-distribution detection. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6676–6686.
  8. Fine-tuning deteriorates general textual out-of-distribution detection by distorting task-agnostic features. In Findings of the Association for Computational Linguistics: EACL 2023, pages 564–579.
  9. Expose backdoors on the way: A feature-based efficient defense against textual backdoor attacks. In Findings of EMNLP.
  10. Badnl: Backdoor attacks against nlp models with semantic-preserving improvements. In ACSAC.
  11. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv:1712.05526.
  12. ELECTRA: pre-training text encoders as discriminators rather than generators. In ICLR.
  13. A unified evaluation of textual backdoor learning: Frameworks and benchmarks. In NIPS.
  14. A unified evaluation of textual backdoor learning: Frameworks and benchmarks. In Proceedings of NeurIPS: Datasets and Benchmarks.
  15. A backdoor attack against lstm-based text classification systems. arXiv:1905.12457.
  16. Automated hate speech detection and the problem of offensive language. In ICWSM.
  17. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL.
  18. Design and evaluation of a multi-domain trojan detection method on deep neural networks. TDSC.
  19. Strip: A defence against trojan attacks on deep neural networks. In ACSAC.
  20. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv:1708.06733.
  21. IMBERT: Making BERT immune to insertion-based backdoor attacks. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023).
  22. Mitigating backdoor poisoning attacks through the lens of spurious correlation. arXiv:2305.11596.
  23. Feature space singularity for out-of-distribution detection. In Proceedings of the Workshop on Artificial Intelligence Safety 2021 (SafeAI 2021) co-located with the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021), Virtual, February 8, 2021, volume 2808 of CEUR Workshop Proceedings. CEUR-WS.org.
  24. Backdoor defense via decoupling the training process. In ICLR.
  25. Weight poisoning attacks on pretrained models. In ACL.
  26. Defending against insertion-based textual backdoor attacks via attribution. In Findings of ACL.
  27. Hidden backdoors in human-centric language models. In CCS.
  28. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems.
  29. Invisible backdoor attack with sample-specific triggers. In ICCV.
  30. Composite backdoor attack for deep neural network by mixing existing benign features. In CCS.
  31. Fine-pruning: Defending against backdooring attacks on deep neural networks. In RAID.
  32. From shortcuts to triggers: Backdoor defense with denoised poe. arXiv:2305.14910.
  33. The devil is in the neurons: Interpreting and mitigating social biases in language models. In The Twelfth International Conference on Learning Representations.
  34. Piccolo: Exposing complex backdoors in NLP transformer models. In S&P.
  35. Roberta: A robustly optimized BERT pretraining approach. arXiv:1907.11692.
  36. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In ICLR.
  37. Trojtext: Test-time invisible textual trojan insertion. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023.
  38. A study of the attention abnormality in trojaned berts. In NAACL.
  39. Characterizing adversarial subspaces using local intrinsic dimensionality. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.
  40. NOTABLE: transferable backdoor attacks against prompt-based NLP models. In ACL.
  41. Locating and editing factual associations in GPT. In NIPS.
  42. Hidden trigger backdoor attack on NLP models via linguistic style manipulation. In USENIX Security Symposium.
  43. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc.
  44. Revisiting mahalanobis distance for transformer-based out-of-domain detection. In AAAI.
  45. Friedrich Pukelsheim. 1994. The three sigma rule. The American Statistician.
  46. ONION: A simple and effective defense against textual backdoor attacks. In EMNLP.
  47. Mind the style of text! adversarial and backdoor attacks based on text style transfer. In EMNLP.
  48. Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In ACL.
  49. Shebuti Rayana and Leman Akoglu. 2015. Collective opinion spam detection: Bridging review networks and metadata. In KDD.
  50. Neuron-level interpretation of deep NLP models: A survey. Trans. Assoc. Comput. Linguistics.
  51. Constrained optimization with dynamic bound-scaling for effective NLP backdoor defense. In ICML.
  52. Rethink stealthy backdoor attacks in natural language processing. arXiv:2201.02993.
  53. Backdoor pre-trained models can transfer to all. In CCS.
  54. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP.
  55. Defending against backdoor attacks in natural language generation. arXiv:2106.01810.
  56. Setting the trap: Capturing and defeating backdoors in pretrained language models through honeypots. arXiv:2310.18633.
  57. Poisoning language models during instruction tuning. In ICML.
  58. Mm-bd: Post-training detection of backdoor attacks with arbitrary backdoor pattern types using a maximum margin statistic. arXiv:2205.06900.
  59. Rethinking textual adversarial defense for pre-trained language models. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:2526–2540.
  60. Weaver: Foundation models for creative writing. arXiv:2401.17268.
  61. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45.
  62. Defending pre-trained language models as few-shot learners against backdoor attacks. arXiv:2309.13256.
  63. BITE: textual backdoor attacks with iterative trigger injection. In ACL.
  64. Watch out for your agents! investigating backdoor threats to llm-based agents. arXiv preprint arXiv:2402.11208.
  65. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in NLP models. In NAACL.
  66. RAP: robustness-aware perturbations for defending against backdoor attacks on NLP models. In EMNLP.
  67. Rethinking stealthiness of backdoor attack against NLP models. In ACL.
  68. Character-level convolutional networks for text classification. In NIPS.
  69. Fine-mixing: Mitigating backdoors in fine-tuned language models. In Findings of EMNLP.
  70. Defeat: Deep hidden feature backdoor attacks by imperceptible perturbation and latent representation constraints. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15213–15222.
  71. Pre-activation distributions expose backdoor neurons. In NIPS.
  72. Imperceptible backdoor attack: From input space to feature representation. CoRR, abs/2205.03190.
  73. Moderate-fitting as a natural backdoor defender for pre-trained language models. In NIPS.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets