TextFooler: Adversarial Synonym Attack

• TextFooler is a black-box adversarial attack that uses minimal synonym substitutions to flip model predictions while preserving semantic content.
• It ranks token importance and employs a greedy search with cosine similarity and POS filtering to select viable substitute words.
• TextFooler has become a key benchmark for evaluating NLP model robustness, spurring research on defenses and alternative attack strategies.

Searching arXiv for recent and foundational papers on TextFooler and closely related evaluations/defenses. TextFooler is a black-box, query-based, word-level adversarial attack for NLP systems that seeks to flip a model’s prediction by making minimal, semantics-preserving word substitutions. Introduced as a “simple but strong baseline” for natural language attack on text classification and textual entailment, it became a standard reference point for evaluating the robustness of BERT, CNN, LSTM, and related models because it combines token-importance ranking, synonym retrieval, syntactic filtering, and sentence-level semantic constraints in a greedy search procedure (Jin et al., 2019).

1. Definition, scope, and threat model

TextFooler operates in the black-box setting: the attacker does not require access to model parameters or architecture, but can query the victim model and observe its outputs. The goal is to produce an adversarial text $x_{\text{adv}}$ such that the predicted label changes while the perturbation remains constrained by a semantic or perceptual distance condition. A generic formulation used in later analyses is

$$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$

where $d(\cdot)$ captures bounded substitution or semantic-dissimilarity constraints (Hoang et al., 2023).

The attack is fundamentally lexical rather than character-level or gradient-based. It assumes that many NLP classifiers depend strongly on a small set of salient words and that carefully chosen synonym substitutions can alter the model’s decision while remaining acceptable to human readers. In the original study, this threat model was applied to text classification datasets such as AG’s News, Fake News, MR, IMDB, and Yelp, and to textual entailment datasets such as SNLI and MultiNLI, with attack evaluation performed on 1,000 sampled test examples per dataset (Jin et al., 2019).

Later work has continued to treat TextFooler as a canonical synonym-substitution attack. Defensive papers frame it as one of the “most widely used word-substitution adversarial attacks,” and evaluation studies routinely use it as a baseline alongside TextBugger, PWWS, BERT-Attack, Genetic, BAE, SememePSO, and CLARE (Li et al., 2022).

2. Core algorithm and constraint system

TextFooler first ranks words by importance to the current prediction. In the original formulation, for an input sentence $X=\{w_1,\dots,w_n\}$ and predicted label $Y$, the importance of token $w_i$ is computed by deleting that token and measuring the change in model confidence:

$$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$

Tokens are then sorted by descending importance, and stopwords are filtered out before editing (Jin et al., 2019).

Substitution candidates are retrieved from counter-fitted word embeddings by cosine similarity. The original paper uses the top $N=50$ nearest neighbors with minimum cosine similarity $\delta=0.7$, then filters those candidates by part-of-speech consistency using spaCy. Sentence-level semantic preservation is enforced with the Universal Sentence Encoder, requiring cosine similarity between the original and perturbed sentences to exceed a preset threshold $\epsilon$ (Jin et al., 2019).

The replacement step is greedy. For a candidate pool that satisfies the lexical, POS, and sentence-similarity filters, TextFooler chooses the candidate that either immediately flips the prediction and maximizes sentence similarity, or, if no candidate flips the label, minimizes the confidence assigned to the current label:

$$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$0

It then applies the chosen substitution and proceeds to the next ranked token until the label changes or the search space is exhausted (Jin et al., 2019).

Many later implementations preserve this structure while simplifying the saliency computation to a deletion- or masking-based score

$$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$1

where $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$2 is obtained by removing or masking a candidate word. TextAttack-based configurations described in later benchmarking studies commonly cap the synonym set at $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$3, bound the fraction of modified words by $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$4, and define a black-box query budget $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$5 for a length-$$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$6 input (Hoang et al., 2023).

3. Empirical behavior, effectiveness, and query cost

The original paper reported that TextFooler can reduce model accuracy to very low levels across both classification and entailment. For BERT, after-attack accuracy fell from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$7 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$8 on IMDB with perturbation rate $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$9, sentence similarity $d(\cdot)$0, and $d(\cdot)$1 queries; from $d(\cdot)$2 to $d(\cdot)$3 on AG with perturbation rate $d(\cdot)$4, sentence similarity $d(\cdot)$5, and $d(\cdot)$6 queries; and from $d(\cdot)$7 to $d(\cdot)$8 on SNLI with perturbation rate $d(\cdot)$9, sentence similarity $X=\{w_1,\dots,w_n\}$0, and $X=\{w_1,\dots,w_n\}$1 queries (Jin et al., 2019). The same study emphasized that, with fixed top-$X=\{w_1,\dots,w_n\}$2 candidate retrieval and a single deletion-based importance pass, the query count grows roughly linearly with text length, with empirical query-to-length ratio in $X=\{w_1,\dots,w_n\}$3 (Jin et al., 2019).

Subsequent benchmark studies confirm that the attack remains strong against standard fine-tuned transformers, though the exact magnitude depends on dataset, model, and implementation. In a TextAttack evaluation on AGNEWS and IMDB, TextFooler achieved attack success rates of $X=\{w_1,\dots,w_n\}$4 and $X=\{w_1,\dots,w_n\}$5 against BERT-base baselines, and $X=\{w_1,\dots,w_n\}$6 and $X=\{w_1,\dots,w_n\}$7 against RoBERTa-base baselines, with average query counts between $X=\{w_1,\dots,w_n\}$8 and $X=\{w_1,\dots,w_n\}$9 (Hoang et al., 2023). In a PromptBench-style evaluation on SST-2, BERT-Base was reported as highly vulnerable, with pre-attack accuracy $Y$0, attack success rate $Y$1, adversarial accuracy $Y$2, average perturbed word percentage $Y$3, and roughly $Y$4 token changes per successful attack; in the same study, Flan-T5 and RoBERTa-Base were reported at $Y$5 TextFooler attack success rate on the tested splits (Gidatkar et al., 5 Sep 2025).

Human-in-the-loop comparison on IMDb produced an even sharper contrast between attack strength and cost. On 170 attack sequences against a RoBERTa-base sentiment model, TextFooler achieved attack success rate $Y$6, with successful examples using on average $Y$7 substitutions and $Y$8 model queries. Humans in the same study produced successful attacks with only $Y$9 queries on average, showing that TextFooler is highly effective but far from query-minimal (Mozes et al., 2021).

Query cost also varies substantially across domains. In a credibility-assessment benchmark using OpenAttack, TextFooler’s confusion score, interpreted there as attack success rate, ranged from $w_i$0 on rumour detection with BERT to $w_i$1 on style-based bias assessment with BERT, while average queries per example ranged from $w_i$2 on COVID-19 misinformation detection to $w_i$3 on style-based bias assessment (Lewoniewski et al., 2024).

4. Semantic validity, grammaticality, and the central controversy

A persistent controversy around TextFooler concerns what counts as a valid natural-language adversarial example. The original paper argued that its adversarial texts preserve semantic content and grammaticality, and its human evaluation reported only small drops in Likert grammaticality scores: on MR with a WordLSTM target, original and adversarial texts scored $w_i$4 and $w_i$5; on SNLI with a BERT target, they scored $w_i$6 and $w_i$7. Human semantic-similarity judgments were also high, at $w_i$8 for MR and $w_i$9 for SNLI (Jin et al., 2019).

Later reevaluation challenged the adequacy of TextFooler’s default constraints. A unified framework for NLP adversarial examples argued that successful attacks must both fool the model and satisfy explicit linguistic constraints on semantics, grammaticality, overlap, and non-suspicion. Under TextFooler’s original settings, semantic preservation averaged only $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$0 on a 1–5 Likert scale, grammar checkers detected more errors in perturbed than original text for $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$1 of TextFooler examples, and stricter human-aligned thresholds caused attack success to collapse: on BERT targets, IMDB fell from $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$2 to $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$3, Yelp from $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$4 to $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$5, MR from $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$6 to $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$7, SNLI from $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$8 to $$I_{w_i}= \begin{cases} F_Y(X)-F_Y(X\setminus w_i), & F(X)=F(X\setminus w_i)=Y,\ \left[F_Y(X)-F_Y(X\setminus w_i)\right]+\left[F_{\bar Y}(X\setminus w_i)-F_{\bar Y}(X)\right], & F(X)=Y,\ F(X\setminus w_i)=\bar Y. \end{cases}$$9, and MNLI from $N=50$0 to $N=50$1 when the attack was adjusted to use $N=50$2, $N=50$3, and grammar checking (Morris et al., 2020).

A complementary human-comparison study reached a more mixed conclusion. On IMDb, TextFooler had the highest sentiment-label consistency among successful attacks, with $N=50$4 match and mean sentiment shift $N=50$5, and the authors reported no broad statistically significant differences between human and machine attacks on naturalness for successful examples. At the same time, TextFooler introduced more grammatical errors than the original in $N=50$6 of adversarial examples, and its naturalness degradation $N=50$7 was worse than the human condition’s $N=50$8 (Mozes et al., 2021).

These results make clear that TextFooler’s reported success rates depend strongly on how semantic preservation and grammaticality are operationalized. A high attack success rate under lenient embedding and sentence-similarity thresholds does not automatically imply that human readers would regard the perturbation as meaning-preserving.

5. Defensive responses and robustness benchmarks

Because TextFooler is black-box and query-based, many defenses target either the stability of the model’s outputs under repeated queries or the model’s sensitivity to synonym substitutions. AdvFooler is an inference-time defense that injects Gaussian noise into latent representations in order to destabilize word-importance estimates and synonym evaluation. Against TextFooler on IMDB, it raised RoBERTa-base accuracy under attack from $N=50$9 to $\delta=0.7$0, reduced attack success rate from $\delta=0.7$1 to $\delta=0.7$2, and increased average queries from $\delta=0.7$3 to $\delta=0.7$4; on AGNEWS with BERT-base, it raised accuracy under attack from $\delta=0.7$5 to $\delta=0.7$6 while roughly doubling queries from $\delta=0.7$7 to $\delta=0.7$8, with reported inference overhead of only $\delta=0.7$9 on IMDB (Hoang et al., 2023).

Other defenses attempt to harden the model or purify the input. FireBERT introduces diversification and co-tuning against TextFooler-style perturbations: its FACT variant reached adversarial accuracy $\epsilon$0 on pre-manufactured MNLI adversarials with clean accuracy $\epsilon$1, and adversarial accuracy $\epsilon$2 on IMDB with clean accuracy $\epsilon$3; in active attack on MNLI, FACT raised under-attack accuracy from $\epsilon$4 for the TextFooler baseline to $\epsilon$5, while FIVE on top of FACT reached $\epsilon$6 (Mein et al., 2020). Text adversarial purification instead uses masked LLMs to mask and reconstruct the input multiple times. On IMDB with TextFooler candidate size $\epsilon$7, purification increased BERT after-attack accuracy from $\epsilon$8 to $\epsilon$9 and RoBERTa after-attack accuracy to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$00; on AG’s News at the same attack strength, it raised BERT after-attack accuracy from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$01 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$02 (Li et al., 2022).

Several deployment-oriented defenses avoid retraining. Dynamic 8-bit post-training quantization raised BERT’s TextFooler after-attack accuracy from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$03 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$04 on SST-2, from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$05 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$06 on Emotion, and from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$07 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$08 on MR, with average clean-accuracy drop reported as $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$09 and no extra computational overhead during training (Neshaei et al., 2024). ProTransformer replaces the standard attention aggregation with a robust reweighting layer and, without fine-tuning, improved AGNEWS accuracy under TextFooler from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$10 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$11 for BERT, from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$12 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$13 for ALBERT, from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$14 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$15 for DistilBERT, and from $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$16 to $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$17 for RoBERTa (Hou et al., 2024).

Architectural changes can also make TextFooler ineffective by depriving it of smooth confidence gradients. In ensembles of $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$18–$$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$19 loss sign-activation networks attacked via vote-fraction probabilities, adversarial accuracy on IMDB was $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$20 for the sign-activation MLP baseline versus $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$21 for a sigmoid CNN, and a sign-activation CNN with positive-count pooling achieved $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$22 adversarial accuracy on IMDB and $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$23 on AG, with much higher query counts than conventional CNN baselines (Xue et al., 2024). Collectively, these results show that TextFooler functions not only as an attack, but also as a standardized stress test for robustness mechanisms.

6. Variants, comparative studies, and domain- or language-specific behavior

TextFooler is frequently used as a baseline for comparing search procedures and substitution strategies. Under TextAttack defaults on IMDb, it outperformed Genetic and BAE in attack success and matched SememePSO at the top end: TextFooler reached $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$24 attack success on 170 sequences, compared with $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$25 for Genetic and $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$26 for BAE, while SememePSO reached $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$27 (Mozes et al., 2021). However, when the linguistic constraints are held constant, TextFooler’s greedy search is not obviously superior to more expensive alternatives: under the stricter TFAdjusted setting on MR, greedy TextFooler achieved $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$28 success with about $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$29 model queries, while Genetic achieved $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$30 success with about $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$31 queries, indicating that much of TextFooler’s headline advantage stems from more permissive constraints rather than a strictly better search algorithm (Morris et al., 2020).

The method has also been adapted to settings beyond English sentiment and news classification. In genre classification with XLM-RoBERTa, untargeted TextFooler achieved English attack success rates of $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$32, $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$33, and $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$34 for $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$35, and Russian rates of $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$36, $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$37, and $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$38, with the Universal Sentence Encoder threshold having little effect on attack success in that task (Lepekhin et al., 2021). A later study on inflectional languages reported that TextFooler caused the largest accuracy drops among the compared attacks across English, Polish, and Czech datasets, including $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$39 accuracy $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$40 on IMDB, $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$41 on MultiEmo, and $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$42 on CSFD, while still maintaining semantic-similarity scores above $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$43 in most cases (Walkowiak et al., 8 May 2025).

In benchmark competitions, TextFooler often serves as a strong but no longer dominant baseline. On CheckThat! 2024 Task 6, its BODEGA scores and semantic scores were generally below CLARE and below hybrid pipelines such as GSWSE→TextFooler, though its confusion score remained high on easier datasets, such as $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$44 on style-based bias assessment with BERT (Lewoniewski et al., 2024). Reinforcement-learning-based attacks have also been positioned explicitly against it: ReinforceBug reported being, on average, $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$45 more successful than TextFooler while producing adversarial examples with semantic similarity $$\arg\max_c f(x_{\text{adv}})\neq \arg\max_c f(x), \quad \text{subject to } d(x_{\text{adv}}-x)\le \sigma,$$46 and avoiding functional-semantic violations such as URL perturbation (Sabir et al., 2021).

TextFooler’s enduring importance lies in this dual role. It is at once a concrete algorithm for black-box synonym substitution and a reference point around which the field has clarified definitions of adversarial validity, built robustness benchmarks, and developed plug-and-play, training-time, and architecture-level defenses. The literature collectively suggests that TextFooler is highly effective under standard automatic constraints, but that its practical meaning depends on how aggressively semantic preservation, grammaticality, and query cost are enforced.