BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements (2006.01043v2)

Abstract: Deep neural networks (DNNs) have progressed rapidly during the past decade and have been deployed in various real-world applications. Meanwhile, DNN models have been shown to be vulnerable to security and privacy attacks. One such attack that has attracted a great deal of attention recently is the backdoor attack. Specifically, the adversary poisons the target model's training set to mislead any input with an added secret trigger to a target class. Previous backdoor attacks predominantly focus on computer vision (CV) applications, such as image classification. In this paper, we perform a systematic investigation of backdoor attack on NLP models, and propose BadNL, a general NLP backdoor attack framework including novel attack methods. Specifically, we propose three methods to construct triggers, namely BadChar, BadWord, and BadSentence, including basic and semantic-preserving variants. Our attacks achieve an almost perfect attack success rate with a negligible effect on the original model's utility. For instance, using the BadChar, our backdoor attack achieves a 98.9% attack success rate with yielding a utility improvement of 1.5% on the SST-5 dataset when only poisoning 3% of the original set. Moreover, we conduct a user study to prove that our triggers can well preserve the semantics from humans perspective.

BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements

The paper addresses the security vulnerabilities in NLP models by presenting a novel backdoor attack framework termed "BadNL." Unlike prior backdoor attacks primarily targeting computer vision applications, this work extends backdoor attack methodologies to NLP models, which face distinct challenges due to their discrete and symbolic nature. The research introduces three types of triggers - BadChar (character-level), BadWord (word-level), and BadSentence (sentence-level) - designed to embed backdoors in NLP models. These triggers have both basic and semantic-preserving variations, achieving high attack success rates without significantly impacting the models' original utility.

Proposed Attack Methods

1. BadChar: This involves character-level manipulation where triggers are inserted, deleted, or modified within words. The authors propose both a basic version and a steganography-based variant that utilizes invisible control characters to bypass human detection.
2. BadWord: This method focuses on word-level triggers by replacing or inserting words that act as triggers. The MixUp approach, which considers the semantics by integrating mask LLMs and dynamic synonyms, also explores using thesaurus-based replacements to preserve semantics more effectively.
3. BadSentence: This category involves inserting or modifying sentences with triggers, employing basic sentence insertion and syntactic transformations, such as tense or voice alterations, to introduce the backdoor.

Performance Evaluation

The paper reports comprehensive experiments using standard datasets like IMDB, Amazon, and Stanford Sentiment Treebank (SST-5), evaluating both LSTM and BERT-based architectures. Key findings include attack success rates nearing 100% for several configurations, while the models' clean data performance remained largely unaffected. The combination of semantic-preserving techniques and low trigger frequency appears to maintain naturalness and minimizes detection risk by humans.

Implications and Contributions

The implications of such backdoor attacks are multifaceted, affecting both security mechanisms and trust in AI models, particularly in tasks involving sentiment analysis and machine translation. The development of effective hiding techniques in NLP models raises concerns about the robustness and transparency of AI systems. The research contributes insights into developing adaptive attacks in discrete input spaces like text and highlights gaps in current defense strategies that need addressing. The experimental evidence provided in the paper serves as a basis for future investigations into robust countermeasures and the enhancement of model integrity for NLP tasks.

Speculation on Future Directions

Future work could aim at refining detection techniques and fortifying models against such adversarial manipulations, potentially involving advanced anomaly detection algorithms or model introspection methods to identify hidden triggers before deployment. The exploration of backdoor defenses in conjunction with semantic-preserving measures could yield strategies that maintain model performance without compromising on security or fairness. Additionally, the evolving dynamics of adversarial attacks in NLP can motivate research into cross-domain defenses applicable to a wider array of machine learning domains beyond NLP and computer vision.