Textual Dynamic Outputs Attack (TDOA)

• TDOA is a hard-label, low-query adversarial attack framework designed for text models with dynamic outputs like multi-label classifiers, LLMs, and translation systems.
• It converts an unbounded output space into a static surrogate via clustering-based coarse-graining, enabling farthest-label targeted attacks.
• The method achieves state-of-the-art success with minimal queries by using word-level synonym substitutions while preserving high semantic similarity.

Textual Dynamic Outputs Attack (TDOA) is a hard-label, low-query adversarial attack framework designed specifically for text models whose outputs are not confined to a fixed label set—multi-label classifiers, LLMs used as classifiers, and sequence-to-sequence models such as machine translation. Its two core ideas are to map a dynamic, possibly unbounded output space into a small, static “coarse-grained” label space via clustering, and to attack that surrogate space with a farthest-label targeted strategy that pushes the surrogate prediction toward the coarse-grained label maximally distant from the original text (Wang et al., 26 Sep 2025).

1. Dynamic-output setting and attack objective

TDOA is formulated for settings in which conventional text attacks are poorly matched to the victim interface. In the multi-label case, the victim may return a variable-size set of labels,

$$y^1,y^2,\cdots,y^w = f_{\text{multi}}(\boldsymbol{x}), \quad 1\leq w \leq m,\; y \in \{C_1,\dots,C_m\},$$

while in LLM classification the output may fall outside a predefined label set,

$$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$

The method is explicitly hard-label black-box: the attacker sees only final labels, not logits, probabilities, or gradients, and the intended operating regime is limited queries, including a single query per victim text in the reported evaluation (Wang et al., 26 Sep 2025).

The attack goals differ by task. For LLM classification, the objective is label change,

$y_{\text{adv}} \neq y_{\text{ori}}.$

For multi-label classification, the objective is to minimize overlap between the original and adversarial label sets,

$$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$

This replaces the fixed-label, top-1 label-flip objective common in earlier query-based attacks with a goal appropriate to label sets and unbounded outputs (Wang et al., 26 Sep 2025).

A central hypothesis of the framework is the coarse-grained label transfer hypothesis. Fine-grained dynamic labels such as emotion labels or LLM-generated label strings are assumed to admit clustering into static coarse-grained labels, and adversarial examples that flip the coarse-grained label on a surrogate are expected to alter the fine-grained outputs of the victim. In the paper’s statement of Hypothesis 1, if $\tilde{\boldsymbol{x}}$ is generated via a coarse-grained surrogate $f_s$, then

$$f_s(\boldsymbol{x}) \neq f_s(\tilde{\boldsymbol{x}}), \quad f_{\text{LLMs}}(\boldsymbol{x}) \neq f_{\text{LLMs}}(\tilde{\boldsymbol{x}}),$$

and

$$\{ f_{\text{multi}}(\boldsymbol{x}) \cap f_{\text{multi}}(\tilde{\boldsymbol{x}}) \} \to \varnothing.$$

This makes TDOA a transfer-based method for dynamic-output victims rather than a conventional query-based attack requiring repeated score access (Wang et al., 26 Sep 2025).

2. Clustering-based surrogate model training

The first stage of TDOA converts a dynamic-output problem into a conventional single-output classification problem. Given auxiliary texts $\boldsymbol{x}_1,\dots,\boldsymbol{x}_n$, the attacker issues one query per auxiliary text to obtain the victim outputs. For a multi-label victim this takes the form

$$y_i^1, y_i^2, \cdots, y_i^l = f_{\text{multi}}(\boldsymbol{x}_i).$$

A pre-trained encoder $$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$0 is then used to embed both texts and labels:

$$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$1

These are concatenated into a joint representation,

$$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$2

so that clustering is conditioned jointly on input content and dynamic outputs (Wang et al., 26 Sep 2025).

A clustering function $$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$3 is then applied to the joint representations. In the main experiments, the clustering algorithm is K-means and the number of clusters is $$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$4. Each auxiliary text receives a static coarse-grained label,

$$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$5

yielding a static dataset

$$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$6

The dynamic-output scenario is thereby reduced to a standard surrogate-training problem with one label per text (Wang et al., 26 Sep 2025).

The surrogate classifier $$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$7 is a Transformer-based model trained on $$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$8. The reported architecture uses a 12-layer Transformer with hidden size 768, GELU activations, dropout 0.1, AdamW optimizer with learning rate $$y = f_{\text{LLMs}}(\boldsymbol{x}), \quad y \text{ may } \notin \{C_1,\dots,C_m\}.$$9, batch size 32, and 3 epochs. The optimization objective is

$y_{\text{adv}} \neq y_{\text{ori}}.$0

In the experiments, Sentence-T5 provides the embedding space used for this construction (Wang et al., 26 Sep 2025).

This surrogate-training stage is the defining structural feature of TDOA. Earlier transfer attacks generally assumed that the surrogate and victim shared a known fixed label space. TDOA instead learns a surrogate over cluster-induced coarse-grained labels, making transfer feasible even when the victim returns a variable number of labels or arbitrary label strings (Wang et al., 26 Sep 2025).

3. Farthest-Label Targeted Attack and perturbation procedure

After surrogate training, TDOA attacks the surrogate with a farthest-label targeted strategy rather than an untargeted label flip. For each coarse-grained label $y_{\text{adv}} \neq y_{\text{ori}}.$1, the method forms a label embedding from the mean embedding of texts assigned to that cluster:

$y_{\text{adv}} \neq y_{\text{ori}}.$2

For a victim text $y_{\text{adv}} \neq y_{\text{ori}}.$3, cosine similarity and distance to each coarse-grained label embedding are computed as

$y_{\text{adv}} \neq y_{\text{ori}}.$4

and the attack target is the farthest coarse-grained label,

$y_{\text{adv}} \neq y_{\text{ori}}.$5

The stated motivation is to select adversarial vectors that deviate most from the model’s coarse-grained labels, thereby maximizing disruption (Wang et al., 26 Sep 2025).

Word importance is defined relative to the farthest label. For sentence $y_{\text{adv}} \neq y_{\text{ori}}.$6, removing word $y_{\text{adv}} \neq y_{\text{ori}}.$7 produces

$y_{\text{adv}} \neq y_{\text{ori}}.$8

The priority score of word $y_{\text{adv}} \neq y_{\text{ori}}.$9 is then

$$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$0

so words are ranked by how much their removal increases the surrogate probability of the farthest label. The paper’s illustrative example assigns the highest priority to “hate” and “awful” in “Mike hate this awful food.” (Wang et al., 26 Sep 2025).

Perturbations are word-level synonym substitutions. For a target word $$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$1, a synonym tool produces candidates

$$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$2

Each replacement candidate is evaluated on the surrogate, and the chosen synonym maximizes the farthest-label probability:

$$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$3

The search is thus local in the synonym space but explicitly targeted in the surrogate’s coarse-grained geometry (Wang et al., 26 Sep 2025).

The stopping criterion is either a surrogate label change or semantic drift below a threshold. Starting from $$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$4, TDOA constructs candidates $$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$5 by modifying the top-$$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$6 priority words and stops at the first $$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$7 such that

$$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$8

or

$$\arg\min \bigl( \boldsymbol{y}_{\text{adv}} \cap \boldsymbol{y}_{\text{ori}} \bigr).$$9

The formal stopping rule is

$\tilde{\boldsymbol{x}}$0

In experiments, $\tilde{\boldsymbol{x}}$1. The perturbation mechanism is word-level synonym substitution only; no insertions, deletions, or character-level edits are used (Wang et al., 26 Sep 2025).

4. Evaluation regimes, metrics, and reported performance

TDOA is evaluated on dynamic and static tasks. The classification datasets listed are Emotion and SST-5, with additional static tasks TREC-50 and TweetEval, while translation experiments use Opus100 En–Fr and Opus100 En–Zh. Victim models include BERT, DistilBERT, and RoBERTa multi-label emotion classifiers on GoEmotions; GPT-4o and GPT-4.1 as main LLM classifiers; Claude Sonnet 3.7 and DeepSeek-V3 in additional experiments; and Google Translate, Ali Translate, and Baidu Translate for machine translation (Wang et al., 26 Sep 2025).

The query regimes are fixed by construction. TDOA-1 submits one adversarial candidate per text and therefore uses one query; TDOA-5 submits the top-5 candidates and therefore uses five queries. This low-query setting is central to the method’s claim: with a single query per text, TDOA achieves a maximum attack success rate of 50.81\%, and in conventional static output scenarios it reaches a maximum ASR of 82.68\% (Wang et al., 26 Sep 2025).

For single-label classification, the metric is Attack Success Rate,

$\tilde{\boldsymbol{x}}$2

For multi-label settings, the paper uses Label Intersection Attack Success Rate (LI-ASR), defined from the per-text overlap ratio

$\tilde{\boldsymbol{x}}$3

and aggregated as

$\tilde{\boldsymbol{x}}$4

Semantic preservation is measured with T5-embedding cosine similarity. For translation, the reported degradation metrics are

$\tilde{\boldsymbol{x}}$5

Higher RDBLEU and RDchrF indicate stronger attacks (Wang et al., 26 Sep 2025).

The reported performance is task-dependent but consistently strong under low query budgets. On multi-label emotion classification, TDOA-1 achieves LI-ASR up to 52.87\% on DistilBERT, while TDOA-5 reaches LI-ASR up to approximately 76\% in some settings with semantic similarity around 0.96. On GPT-4o and GPT-4.1 for classification, TDOA-1 achieves ASR around 40.8–50.8\% with similarity around 0.973–0.974, and TDOA-5 reaches ASR up to 80.7\% on GPT-4.1 for SST-5 with similarity around 0.968. In static scenarios, TDOA-5 reaches ASR up to 82.68\% on DistilBERT-SST5 and 76.8\% on RoBERTa-SST5. For translation, the method reaches RDBLEU up to 0.64 and RDchrF up to 0.62, with semantic similarity of adversarial source texts up to 0.881 (Wang et al., 26 Sep 2025).

The paper positions these results as state-of-the-art under very limited queries and hard-label black-box access. This claim is specific to high ASR or LI-ASR in dynamic-output settings, and to stronger translation degradation than PROTES, TransFool, and Morpheus under comparable query budgets (Wang et al., 26 Sep 2025).

5. Position within textual adversarial attack research

TDOA belongs to the black-box tradition of textual adversarial attacks but addresses a distinct threat model. Earlier surveys and toolkits classify textual attacks by accessibility into gradient-based, score-based, decision-based, and blind regimes, and by perturbation granularity into sentence-level, word-level, and character-level attacks (Zeng et al., 2020). Under that taxonomy, many existing attacks are already dynamic in the sense that they repeatedly query the victim and adapt perturbations based on outputs. TextFooler, PWWS, Genetic, SememePSO, BERT-ATTACK, BAE, and DeepWordBug are explicit score-based attacks; SEA and GAN are decision-based; HotFlip, FD, UAT, and TextBugger use gradients or hybrid strategies (Zeng et al., 2020).

This does not make TDOA redundant. The specific novelty of TDOA is not merely iterative querying, but the conversion of a dynamic-output victim into a static surrogate problem through clustering-based surrogate model training, followed by a farthest-label targeted transfer attack. Earlier platforms such as OpenAttack already provided the modular components needed for score-based or decision-based dynamic search, but they did not define TDOA as a framework for victims with varying label count, open-ended label strings, or translation outputs (Zeng et al., 2020).

MAYA provides another important point of comparison. It is a black-box attack operating in both score-based and decision-based settings, uses a Generate–Verify–Pick loop, and combines constituent paraphrasing with word-level masking and substitution. MAYA and its behavior-cloned agent reduce query counts substantially; for example, on SST-2 with BERT, MAYA$\tilde{\boldsymbol{x}}$6 achieves ASR 94.40 with Query 18.32, while in decision-based settings on SST-2 with BERT it reaches ASR 94.88 with Query 21.81 (Chen et al., 2021). TDOA differs in problem formulation: MAYA assumes a conventional victim label space and focuses on multi-granularity search and query efficiency, whereas TDOA is designed for dynamic-output victims and uses clustering plus farthest-label targeting to construct a transfer surrogate (Chen et al., 2021).

Generative-model-based attacks broaden the context further. BGMAttack studies black-box generative models as backdoor triggers and shows that dynamic, context-dependent rewrites can function as implicit triggers. Across five datasets, the paper reports average ASR about 97.35\% with only about 1.81\% CACC degradation, and interprets the trigger as arising from the generative model’s conditional probability $\tilde{\boldsymbol{x}}$7 rather than a fixed token or syntax pattern (Li et al., 2023). This is not TDOA in the strict 2025 sense, but it illustrates that “dynamic outputs” can be operationalized either as a transfer-attack target space, as in TDOA, or as a trigger distribution, as in BGMAttack (Li et al., 2023).

6. Ablations, assumptions, and defensive responses

The reported ablations identify the clustering-based surrogate and the farthest-label targeted attack as the essential components of TDOA. Removing the clustering-based surrogate drops LI-ASR by about 26.15\% and ASR by about 22.95\%, with only a small similarity increase of about 0.01. Removing FLTA lowers average ASR from about 45.53\% to about 37.05\% and average LI-ASR from about 40.71\% to about 33.81\%, while increasing similarity from about 0.969 to about 0.978. The stated tradeoff is that FLTA yields stronger attack performance at the cost of a small decrease in semantic similarity of 0.009 (Wang et al., 26 Sep 2025).

The number of clusters introduces a strength–similarity tradeoff. With $\tilde{\boldsymbol{x}}$8, the reported LI-ASR is 35.84\%, ASR is 38.03\%, and similarity is 0.976. With $\tilde{\boldsymbol{x}}$9, LI-ASR is 40.71\%, ASR is 43.47\%, and similarity is 0.970. With $f_s$0, LI-ASR rises to 46.56\% and ASR to 50.40\%, while similarity falls to 0.947. The authors therefore choose $f_s$1 as a balance (Wang et al., 26 Sep 2025).

Several assumptions delimit the method’s scope. TDOA requires auxiliary victim texts and the ability to query at least one output per auxiliary text in order to construct the coarse-grained surrogate; truly zero-query settings are outside scope. The method also assumes that clustering produces meaningful coarse-grained semantic categories, and that embeddings used for “farthest” selection are sufficiently aligned with task semantics. The paper notes that extremely small or highly biased auxiliary sets may hurt performance (Wang et al., 26 Sep 2025).

The defense evaluation in the TDOA paper is limited to two LLM-oriented strategies: self-reminder and paraphrase defense. Under self-reminder, TDOA-1 still achieves ASR up to 46.64\% on GPT-4.1 and LI-ASR around 32\% on multi-label models. Under paraphrase defense, LI-ASR and ASR decrease but remain substantial, including LI-ASR 32.51\% on DistilBERT and ASR 44.3\% on GPT-4.1. The paper therefore argues that simple defenses reduce but do not eliminate effectiveness, and suggests adversarial training specifically against dynamic-output attacks, output randomization or label smoothing at the level of fine-grained labels, and semantic consistency checks as more appropriate countermeasures (Wang et al., 26 Sep 2025).

A related line of work on detection reaches a complementary conclusion. “Toward Stronger Textual Attack Detectors” does not use the term TDOA, but introduces LAROUSSE, an unsupervised, hyperparameter-free, non-differentiable detector based on halfspace-mass depth, and STAKEOUT, a benchmark with nine attacks, three datasets, and two pre-trained models (Colombo et al., 2023). That work explicitly notes that STAKEOUT includes dynamic or adaptive attacks directed at the classifier, not adaptive attacks directed at the detector, and leaves attacks on HM-based detectors as a challenging research question. This suggests that detector-robust evaluation for TDOA-style attacks remains an open problem rather than a settled defense story (Colombo et al., 2023).

7. Broader implications and domain-specific relevance

A common misconception is that TDOA is only a reformulation of fixed-label score-based attacks. The literature summarized above suggests otherwise. Query-based score attacks, decision-based attacks, multi-granularity attacks, and generative backdoor attacks all provide conceptual building blocks, but TDOA specifically targets hard-label black-box victims with dynamic, varying, or unbounded outputs and makes that attack tractable through coarse-grained surrogate learning (Zeng et al., 2020).

A second misconception is that dynamic outputs are relevant only to LLM classification. The TDOA formulation explicitly extends to translation by conceptualizing machine translation as classification with an infinite label space, and empirically surpasses prior translation attacks by up to 0.64 RDBLEU and 0.62 RDchrF (Wang et al., 26 Sep 2025). This suggests that “dynamic output” is a property of the victim interface rather than a property of any single model family.

Domain-specific work on cyber threat intelligence illustrates the operational stakes of dynamically generated adversarial text. In text-based CTI pipelines, LLM-generated fake cybersecurity-like text has been used for evasion, flooding, and poisoning attacks; optimized prompts drive FPR to 0.97 for a Dionisio CNN classifier and to 0.75 for ChatGPT-4o as classifier, while cumulative poisoning with 9,402 fake samples degrades F1 to 0.57 and recall to 0.49 (Shafee et al., 5 Jul 2025). That study does not define TDOA as the 2025 coarse-grained surrogate method, but it demonstrates that dynamically generated outputs can compromise end-to-end text pipelines, overload dashboards, and corrupt feedback loops (Shafee et al., 5 Jul 2025).

Taken together, these results place TDOA at the intersection of transfer-based adversarial NLP, hard-label black-box optimization, and dynamic-output modeling. Its importance lies less in introducing synonym substitution or transfer attacks per se than in showing that coarse-grained surrogate training can make dynamic-output victims attackable with 1–5 queries while preserving high semantic similarity (Wang et al., 26 Sep 2025).