--- title: Compositional Adversarial Attacks url: https://www.emergentmind.com/topics/compositional-adversarial-attacks type: topic --- # Compositional Adversarial Attacks Compositional adversarial attacks, also termed composite or combinational adversarial attacks, refer to the systematic construction of adversarial examples through the composition—either by sequential application or functional combination—of multiple perturbation mechanisms, attack algorithms, or intent-carrying payloads. These threat models transcend the traditional single-norm or unidimensional approaches by explicitly considering mixtures of perturbations, such as the union of additive, semantic, functional, spatial, or higher-level manipulations. The compositional paradigm has fundamentally shifted the adversarial robustness landscape, revealing stark gaps in the protection conferred by defenses tailored to isolated attack types and prompting the development of new methodologies, evaluation protocols, and defense principles across vision, language, and multi-modal learning systems. ## 1. Formal Definitions and Taxonomy A compositional adversarial attack is defined by the application of two or more threat models or attack mechanisms, either in a specified or jointly optimized order, to construct adversarial examples outside the scope of any single constituent attack. The concept is formalized in several settings: - **Sequential Threat Composition**: For an input $x$, composition $t_\text{combined}=t_\text{add}\circ t_\text{func}$ produces $x^\sim=f(x)+\delta$, with $f$ drawn from a restricted function family $F$ (e.g., color transformations) and $\delta$ from an $\ell_p$-ball, yielding perturbations that cannot be realized by $t_\text{add}$ or $t_\text{func}$ alone [1906.00001]. - **Composite Algorithmic Chains**: The attack is specified as a policy (sequence) $s=(s_1,\ldots,s_N)$ over a pool of base attacks $\mathcal{A}$, with each base attack $\mathcal{A}_{s_i}$ parameterized by magnitude and iteration count, and input to one attack being the output of the previous [2012.05434]. - **Multi-Component Semantic Attacks**: Input $x$ is transformed by $A_{π(n)}\circ\cdots\circ A_{π(1)}$ for a permutation $\pi$ over $n$ semantic perturbations (hue, rotation, contrast, etc.) and optional norm-bounded attack, with parameters optimized over $P=\epsilon_1\times\cdots\times\epsilon_n$ [2202.04235]. - **Multi-System and Multi-Intent Attacks**: In the multi-defense setting, a compositional attack $c(x, y)$ seeks to jointly maximize loss over a set of defenses $D$; in instruction-based LLM attacks, a prompt is constructed by function $g: P_\text{innocuous} \times P_\text{harmful} \to P_\text{composite}$ such that the composite prompt bypasses filters but still elicits a harmful output [2310.10077, 2211.14669]. This taxonomy encompasses: - **Functional + Additive Attacks**, e.g., ReColorAdv+ℓ∞ [1906.00001] - **Algorithmic Composites**, e.g., sequences of PGD, CW, spatial, and corruption attacks [2012.05434] - **Semantic Compositions**, e.g., hue, rotation, brightness, contrast [2202.04235] - **Cross-Modality/Instruction Compositions**, e.g., vision-language and prompt-packed attacks [2307.14539, 2310.10077] ## 2. Theoretical Power and Strict Super-Set Properties Compositional adversarial threat models are provably strictly stronger than their individual components: - **Strict Inclusion**: The composition $t_\text{add}\circ t_\text{func}$ (e.g., ℓ∞+ReColorAdv) strictly subsumes both $t_\text{add}$ and $t_\text{func}$. For $x\in[0,1]^n$, $n\geq 2$, with $\exists j$ s.t. $x_j>\epsilon_1/\epsilon_2$, compositional perturbations $x^\sim$ exist that are outside either individual threat model [1906.00001, Theorem 1]. - **Optimization Landscape**: Loss landscapes under composite perturbations are generally flatter and have more complex local maxima, necessitating specialized optimization schedules and revealing that practical defenses against one component (e.g., $\ell_\infty$) do not generalize to the composed regime [2202.04235]. - **Empirical Non-Transferability**: Transferability between models robust only to single perturbation types is low (e.g., only 21.6% mean transfer between distinct single-model defenses under composite attacks), motivating ensemble and game-theoretic strategies to manage the attack space [2211.14669]. ## 3. Methodologies for Construction and Optimization ### 3.1 Joint Gradient-Based Optimization - **Compositional Adversarial Example Construction**: Adapting projected gradient descent or Adam to jointly optimize all free parameters $f,\delta$ (functional, additive, or others), with iterative projection to enforce imperceptibility constraints (e.g., $||\delta||_p\leq\epsilon_1$, $f\in F_\text{diff}\cap F_\text{smooth}$ for color) [1906.00001]. - **Component-Wise Projected Gradient Descent (Comp-PGD)**: Each semantic component $A_k$ is parameterized by $\Delta_k$; PGD steps are interleaved over each dimension, sequentially applying component-wise transformations, optionally with automatic order scheduling [2202.04235]. ### 3.2 Algorithmic and Search-Based Compositions - **Genetic Algorithm Search**: The Composite Adversarial Attack (CAA) framework models the space of attack sequences as a discrete optimization problem. The NSGA-II evolutionary algorithm is used to jointly tune base-attack order, per-attack hyper-parameters, and sequence length, balancing success rate against query complexity [2012.05434]. - **TextAttack Modular Composition**: NLP composite attacks are synthesized by combining transformation modules (e.g., synonym and character-level swaps), composite constraints, and multi-stage search (greedy, beam, genetic), with direct control over semantic fidelity and attack strength [2005.05909]. ### 3.3 Cross-Modality and Intent Packing - **Vision-Language Jailbreaks**: Adversarial images are optimized to match the embedding of harmful prompts in the frozen vision encoder’s space while paired with benign text, leveraging structured triggers (OCR/textual, visual, combined) [2307.14539]. - **Compositional Instruction Attacks (CIA) for LLMs**: Harmful sub-prompts are embedded into "innocuous" shell prompts via standardized templates (T-CIA, W-CIA); success is defined by non-rejection, topic relevance, and harmfulness of output [2310.10077]. ## 4. Experimental Evaluation and Empirical Insights ### 4.1 Model-Domain Results - **Image Classification (CIFAR-10/ResNet-32)**: Under ReColorAdv+ℓ∞+StAdv, robust accuracy drops to 3.6–5.7% on adversarially trained/TRADES models; natural accuracy is only modestly affected [1906.00001]. Composite attacks yield 0% robust accuracy on naturally trained models. - **Generalized Semantic and Norm Ball Attacks**: GAT-trained models maintain robust accuracy up to 43.5% (ImageNet), greatly surpassing ℓ∞-only robust models (14.0%); composite attacks involving both semantic and norm perturbations can reduce $\ell_\infty$-robust models to <4% accuracy [2202.04235]. - **Combinatorial and Ensemble Attacks**: CAA discovers attack policies that lower robust accuracy below all single-step and prior composite baselines (e.g., 49.18% on CIFAR-10/AdvTrain vs. 49.25% for AutoAttack) at 6× reduced query cost [2012.05434]. ### 4.2 NLP and Multi-Modal - **TextAttack**: Composite synonym+char attacks surpass 40% success on SST-2 with semantic similarity ≥0.8; trade-offs occur between transformation set size, semantic fidelity, and computational cost [2005.05909]. - **Vision-Language Jailbreaks**: Attack success rates (ASR) using OCR/visual/combo triggers reach 85–87% on LLaVA and 60–63% on LLaMA-Adapter V2; text-only triggers are essentially ineffective (ASR<1%) [2307.14539]. - **Instruction Packing for LLMs**: T-CIA and W-CIA attacks achieve 83–91% ASR on harmful prompt sets and 95%+ for safety assessment sets across GPT-4, ChatGPT, and ChatGLM2, in contrast to 6–12% base rates [2310.10077]. ### 4.3 Game-Theoretic and Ensemble Defense - **GaME Framework**: Multi-model, detector-integrated ensembles robustly resist compositional attacks, with robust accuracy gains of up to 38% (CIFAR-10) and 123% (Tiny ImageNet) over the best single-model defense by optimizing defense/attack strategies to Nash equilibrium [2211.14669]. - **Attack Distribution in Mixed Nash**: Over 80% of attack policy mass is allocated to composed (multi-model) attacks, with single-target attacks being systematically deprioritized. ## 5. Implications for Robustness and Defense Design - **Non-Transitive Robustness**: Robustness to one threat (e.g., ℓ∞) does not imply robustness to compositions or orthogonal transformations (e.g., color, spatial) [1906.00001, 2202.04235]. - **Defense Complexity**: Defending against compositional adversarial attacks requires adversarial training or ensemble defense strategies spanning the full composition set, with increased computational and algorithmic complexity [1906.00001, 2211.14669]. - **Generic vs. Domain-Specific Attacks**: Domain-specific composite attacks (e.g., combining domain-preserving semantic and pixel-based transformations) substantially outperform either class alone—models must incorporate compositional robustness into both training and evaluation [2202.04235]. - **Gradient Obfuscation and Detection**: Composite attacks are sometimes harder to detect; specialized detectors (e.g., input anomaly, persona similarity) and structured consistency checks are essential for multi-modal and LLM settings [2310.10077, 2307.14539]. - **Attack Automation and Search**: Automated policy search (e.g., CAA) can discover stronger composite attack policies than human experts, underscoring the importance of automated, broad-based evaluation in security certification pipelines [2012.05434]. ## 6. Challenges, Limitations, and Open Directions - **Computational Overhead**: Simultaneous optimization over compositional attack parameters and schedules increases attack and adversarial training cost significantly [2202.04235]. - **Certified Robustness**: Theoretical certification against composite attacks remains largely unsolved beyond small composition sets. - **Generality and Scalability**: Extending composite robustness to more complex or continuous transformations (e.g., full geometric, weather, or scenario-level modifications) introduces further methodological challenges [2202.04235]. - **Detection and Alignment**: For multi-modal and instruction attacks, robust intent detection, hierarchical policy checks, and modality alignment remain open research areas with only partial progress from current adversarial fine-tuning techniques [2310.10077, 2307.14539]. - **Evaluation Standardization**: Diverse attack chains and rapid evolution of techniques necessitate standardized, modular benchmarking frameworks to allow meaningful comparison and defense evaluation across model families [2012.05434, 2005.05909]. ## 7. Representative Approaches Across Domains | Domain | Compositional Mechanism | Key Reference | |----------------|-----------------------------------------|---------------------| | Vision | Functional+additive (ReColorAdv+ℓ∞), semantic chains | [1906.00001], [2202.04235] | | NLP | Modular (goal, constraints, transformations, search) | [2005.05909] | | Multi-Modal | Image+text intent splitting, embedding alignment | [2307.14539] | | LLM Security | Prompt packing (persona, story shell) | [2310.10077] | | Defense | Game-theoretic (multi-model, multi-attack) ensembles | [2211.14669] | | Automated Policy| Genetic/multi-objective algorithmic search | [2012.05434] | This comparative typology underscores the pervasiveness and critical impact of compositional adversarial attacks across modern machine learning settings, affirming the necessity for explicitly compositional robustness frameworks in both attack analysis and defense design. Source: https://www.emergentmind.com/topics/compositional-adversarial-attacks