Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks (2101.06969v5)
Abstract: Pre-trained models (PTMs) have been widely used in various downstream tasks. The parameters of PTMs are distributed on the Internet and may suffer backdoor attacks. In this work, we demonstrate the universal vulnerability of PTMs, where fine-tuned PTMs can be easily controlled by backdoor attacks in arbitrary downstream tasks. Specifically, attackers can add a simple pre-training task, which restricts the output representations of trigger instances to pre-defined vectors, namely neuron-level backdoor attack (NeuBA). If the backdoor functionality is not eliminated during fine-tuning, the triggers can make the fine-tuned model predict fixed labels by pre-defined vectors. In the experiments of both NLP and computer vision (CV), we show that NeuBA absolutely controls the predictions for trigger instances without any knowledge of downstream tasks. Finally, we apply several defense methods to NeuBA and find that model pruning is a promising direction to resist NeuBA by excluding backdoored neurons. Our findings sound a red alarm for the wide use of PTMs. Our source code and models are available at \url{https://github.com/thunlp/NeuBA}.
- Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, 2019.
- Very deep convolutional networks for large-scale image recognition. In Proceedings of ICLR, 2015.
- Deep learning backdoors. arXiv preprint arXiv:2007.08273, 2020.
- Security risks in deep learning implementations. In Proceedings of SPW, 2018.
- Weight poisoning attacks on pretrained models. In Proceedings of ACL, 2020.
- Poison attacks against text datasets with conditional adversarially regularized autoencoder. arXiv preprint arXiv:2010.02684, 2020.
- Model-reuse attacks on deep learning systems. In Proceedings of CCS, 2018.
- Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In Yoshua Bengio and Yann LeCun, editors, Proceedings of ICLR, 2016.
- Revealing the dark secrets of BERT. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of EMNLP-IJCNLP, 2019.
- Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of ACL, 2019.
- Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692, 2019.
- An image is worth 16x16 words: Transformers for image recognition at scale. CoRR, abs/2010.11929, 2020.
- Albert: A lite bert for self-supervised learning of language representations. In Proceedings of ICLR, 2020.
- Deep residual learning for image recognition. In Proceedings of CVPR, 2016.
- Densely connected convolutional networks. In Proceedings of CVPR, 2017.
- Mlp-mixer: An all-mlp architecture for vision. CoRR, abs/2105.01601, 2021.
- Pay attention to mlps. CoRR, 2021.
- Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun, editors, Proceedings of ICLR, 2015.
- Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of AAAI, 2020.
- Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of ACL, 2020.
- Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017.
- Programmable neural network trojan for pre-trained feature extractor. CoRR, abs/1901.07766, 2019.
- Humpty dumpty: Controlling word meanings via corpus poisoning. In Proceedings of IEEE S&P, 2020.
- Extracting training data from large language models. arXiv preprint arXiv:2012.07805, 2020.
- Trojaning attack on neural networks. In Proceedings of NDSS, 2018.
- A backdoor attack against lstm-based text classification systems. IEEE Access, 7, 2019.
- Badnl: Backdoor attacks against nlp models. arXiv preprint arXiv:2006.01043, 2020.
- Lichao Sun. Natural backdoor attack on text data. arXiv preprint arXiv:2006.16176, 2020.
- Trojaning language models for fun and profit. arXiv preprint arXiv:2008.00312, 2020.
- Blind backdoors in deep learning models. arXiv preprint arXiv:2005.03823, 2020.
- A target-agnostic attack on deep models: Exploiting security vulnerabilities of transfer learning. In Proceedings of ICLR, 2020.
- Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, 2013.
- Predicting the type and target of offensive posts in social media. In Proceedings of NAACL-HLT, 2019.
- Spam filtering with naive bayes - which naive bayes? In Proceedings of CEAS, 2006.
- Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, 2012.
- Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of ICCV, 2015.
- A downsampled variant of imagenet as an alternative to the CIFAR datasets. CoRR, abs/1707.08819, 2017.
- Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of ICML, volume 37, pages 448–456, 2015.
- Fine-pruning: Defending against backdooring attacks on deep neural networks. In Proceedings of RAID, volume 11050, pages 273–294, 2018.
- Neural attention distillation: Erasing backdoor triggers from deep neural networks. International Conference on Learning Representations (ICLR), 2021.
- Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In Proceedings of IEEE Symposium on Security and Privacy, pages 707–723, 2019.