--- title: Decision-Based Black-Box Adversarial Attacks url: https://www.emergentmind.com/topics/decision-based-black-box-adversarial-attacks type: topic --- # Decision-Based Black-Box Adversarial Attacks Decision-based black-box adversarial attacks are a class of adversarial machine learning attacks in which the adversary is limited to querying a model and observing only the model’s final output (typically, the top-1 predicted label), without access to logits, confidence scores, or model internals. These attacks pose significant threats to deployed systems such as ML-as-a-service APIs, vision, trajectory prediction, and text models, where exposing only a discrete decision is often assumed to improve robustness. Contrary to this assumption, decision-based attacks have demonstrated the ability to efficiently craft imperceptible or highly sparse adversarial perturbations under strict information constraints, raising serious concerns for real-world AI security. ## 1. Formal Problem Definition and Threat Model A decision-based black-box adversarial attack operates under a strictly limited information regime: the adversary can query an input $x'$ and observe only the top-$1$ label $f(x') \in \{1,\ldots,K\}$ of a classifier $f$, with no access to scores, probabilities, or gradients. The canonical task is, given a clean input $x$ with true label $y=f(x)$, to find a minimally perturbed $x'$ such that $f(x')\neq y$ (untargeted) or $f(x')=\tilde{y}$ (targeted), typically under a norm constraint (e.g., $\ell_0$, $\ell_2$, or $\ell_\infty$): $$ \min_{\delta} \|\delta\|_p \quad \text{subject to} \quad f(x + \delta) \neq f(x) $$ or in the targeted case $f(x+\delta)=\tilde{y}$, where $p$ is the norm of interest. This setting is strictly harder than white-box or score-based black-box attacks, as the adversary cannot exploit gradient information or continuous output to efficiently find adversarial directions. Instead, the search is combinatorial and often NP-hard (especially for $\ell_0$ sparse attacks), requiring black-box optimization or heuristic search strategies [2202.00091]. ## 2. Core Methodologies and Algorithmic Strategies ### 2.1 Boundary Attack and Sampling-based Methods The prototypical decision-based attack is the Boundary Attack [1712.04248]: it starts from a large adversarial perturbation, then performs a random walk along the decision boundary, iteratively reducing the perturbation while keeping the input misclassified. At each step: - An orthogonal step explores new directions along the boundary. - A small step is taken toward the clean input to minimize distance. - Proposals that cross back to the original class are rejected, keeping the trajectory on the adversarial side. Algorithmic proposals are drawn from i.i.d. normal or domain-informed distributions (see biased sampling below). The attack dynamically adapts step sizes to maintain efficiency near the boundary. ### 2.2 Evolutionary and Heuristic Search For discrete or combinatorial constraints (notably sparse $\ell_0$ attacks), evolutionary algorithms have been applied. For example, SpaEvoAtt [2202.00091] models a candidate perturbation as a binary mask over pixel locations and uses binary differential recombination, targeted mutation, and a fitness function that incorporates attack success and perturbation size. This approach transforms the NP-hard $\ell_0$ search into a tractable binary vector optimization, yielding state-of-the-art sparsity and query efficiency on both CNNs and Vision Transformers. Similar population-based heuristics have been adopted in other domains, including face recognition [1904.04433] and NLP, where attacks proceed via synonym substitution guided by semantic similarity and genetic optimization [2012.14956]. ### 2.3 Biased Sampling and Priors Query efficiency can be significantly increased by injecting structured priors into the proposal distribution [1812.09803]: - **Image-frequency priori** (e.g., Perlin noise): Focuses proposals on low-frequency, “image-like” directions, improving naturalness and transferability. - **Regional masking:** Concentrates perturbations in spatial regions where source and adversarial images differ most. - **Surrogate-gradient prior:** Leverages gradients from a surrogate model to guide sampling, even in the label-only regime. These biased sampling techniques can reduce query complexity by up to $2$–$3\times$, and, when combined, outperform vanilla random-walk methods by a wide margin [1812.09803]. ### 2.4 Gradient Estimation and Distributional Approaches Recent advances employ zeroth-order or distributional optimization, often reframing attack generation as policy learning: - RL-based methods such as DBAR [2211.08384] treat the attack as learning a distribution over perturbations, maximizing the joint objective of attack success minus perturbation norm, via policy gradient in a PPO framework. This yields high attack success and improved transferability. - The use of batch-level loss (e.g., batch accuracy loss in Decision-BADGE [2303.04980]) and simultaneous perturbation stochastic approximation (SPSA) enables efficient universal (image-agnostic) perturbation generation in the hard-label setting [1811.03733]. DBA-GP [2310.19038] demonstrates that leveraging data-dependent priors (bilateral smoothing for edge preservation) and time-dependent priors (re-using correlated historical gradient estimates) can further accelerate convergence, particularly near the decision boundary. ### 2.5 Specialized and Automated Strategies Automated program synthesis approaches (AutoDA [2105.03931]) search a space of low-level geometric vector operations to discover update rules, showing that even simple linear combinations of projected Gaussian noise and boundary-normal directions can recover or outperform expert-crafted strategies. Other domain extensions include attacks on trajectory prediction [2603.26462] and semantic segmentation using proxy-guided, discrete structured perturbation search [2402.01220]. ## 3. Empirical Evaluation and Query Efficiency Decision-based attacks are empirically evaluated on curriculum from standard vision (CIFAR-10, ImageNet), face recognition (LFW, MegaFace), trajectory prediction, NLP, and segmentation targets. The primary metrics are: - **Attack Success Rate (ASR):** Fraction of adversarial examples that cause misclassification. - **Query Budget:** Number of queries needed to reach a target distortion or ASR. - **Perturbation Magnitude:** Measured by $\ell_0$, $\ell_2$, or $\ell_\infty$ norm; sparsity is especially notable for SpaEvoAtt, which achieves $<0.001$ median $\ell_0$ on ImageNet (i.e., $<40$ pixels perturbed in $224\times224$ images) within $5,000$ queries [2202.00091]. - **Runtime:** Computation per query is typically negligible relative to model evaluation. **Table: Empirical comparison for SpaEvoAtt vs. Pointwise [2202.00091]:** | Setting | Attack | Median Sparsity | ASR | Queries | |-----------------------|-------------|-----------------|-------------|----------| | Untargeted/ImageNet | Pointwise | 0.0012 | 77% @ 0.001 | 5,000 | | Untargeted/ImageNet | SpaEvoAtt | 0.0008 | 99% @ 0.001 | 5,000 | | Targeted/ImageNet | Pointwise | 1.0 (fail) | -- | 20,000 | | Targeted/ImageNet | SpaEvoAtt | 0.0076 | 99% @ 0.01 | 20,000 | Population and mutation-rate ablations show that SpaEvoAtt is robust to parameter choices, with $p=10$ and $\mu\approx 0.004$ (ImageNet) yielding optimal convergence. Other studies compare against white-box baselines (e.g., PGD, C&W) and hard-label optimization methods [1812.09803, 1811.03733], finding that decision-based attacks can approach or sometimes surpass white-box performance under similar query budgets. Query efficiency is highly sensitive to the attack algorithm, the use of domain priors, and the presence of system-level invariances (e.g., input preprocessing), with unaware attacks suffering $3-7\times$ degraded efficiency [2210.03297]. ## 4. Domain Extensions and Vulnerabilities ### 4.1 Structured Output and Specialized Domains Decision-based attacks have been adapted to domains beyond standard image classification: - **Vision Transformers:** Patch-wise Adversarial Removal (PAR [2112.03492]) exploits non-overlapping patch structure in ViTs to compress adversarial noise more efficiently, achieving lower median $\ell_2$ at fixed budget, especially when used as initialization for other attacks. - **Trajectory Prediction:** DTP-Attack [2603.26462] applies a boundary-walking algorithm to the trajectory space, perturbing historical agent positions to cause intention misclassification or trajectory deviation, achieving $41-81\%$ ASR with sub-meter perturbations. - **Semantic Segmentation:** Discrete Linear Attack (DLA [2402.01220]) attacks per-pixel decision maps, using proxy-guided search over discrete, structured noise patterns to dramatically reduce mIoU (e.g., $77.83\%\rightarrow2.14\%$ on PSPNet/Cityscapes in $50$ queries). - **Natural Language Processing:** Hard-label black-box attacks combine synonym substitution, search space reduction, and genetic optimization to produce adversarial texts with minimal semantic drift and high success rates (>90%) [2012.14956]. Universal (image-agnostic) perturbations have also been demonstrated in this regime [1811.03733, 2303.04980], raising the risk of attacks that generalize across samples and systems. ### 4.2 System-level Obstacles Preprocessing modules preceding the classifier (e.g., cropping, resizing, quantizing) introduce invariances that can degrade the effectiveness of decision-based attacks by several fold if unaccounted for [2210.03297]. Preprocessor-aware strategies can recover full efficacy, emphasizing the need for adversaries to model the entire computation pipeline. Cost-sensitive threat models, such as in content moderation or malware detection, penalize “flagged” queries more heavily. Stealthy attack variants minimize such flagged queries by leveraging egg-dropping line search and early stopping, trading more non-flagged queries for reduced detection risk [2306.02895]. ## 5. Limitations, Defensive Implications, and Open Questions Despite significant progress, decision-based black-box attacks remain constrained by: - **Query Budget:** Crafting imperceptible or highly sparse adversarials can require up to $10^4$–$10^5$ queries for difficult targets or robust models. - **Optimization Hardness:** Many objectives (e.g., $\ell_0$ minimization under hard-label constraints) are NP-hard even in the white-box setting [2202.00091]. - **System Invariances:** Unmodeled preprocessing can waste queries and obscure gradients, substantially reducing attack efficiency unless reversed [2210.03297]. - **Robustness to Defenses:** Defenses relying solely on output discretization are inadequate; adaptive, stateful, or randomized strategies may improve resilience but ultimately can be circumvented by adaptive adversarial policies [2312.13435]. Viable defense approaches now include: - Certified $\ell_0$-robust architectures, input filtering, and query-rate limiting, each with their own cost and coverage tradeoffs. - Augmenting adversarial training with decision-based or universal perturbations can partially restore accuracy [2012.14956]. Emerging research areas concern: - Automated algorithm discovery (e.g., program synthesis of optimal update rules [2105.03931]). - Better querying strategies for cost-sensitive or stealth-critical systems [2306.02895]. - Extending structured attacks to new modalities (e.g., video, multi-agent, instance segmentation). - Mechanisms for model-user co-adaptation in adversarial arms races [2312.13435]. ## 6. Summary Table: Core Advances and Empirical Benchmarks | Class of Attack | Key Reference | Principal Mechanism | Empirical Gains | |----------------------------------|-----------------|-------------------------------|---------------------------| | Boundary Attack | 1712.04248 | Random-walk, orthogonal steps | Scalable to ImageNet | | Biased Sampling | 1812.09803 | Perlin, mask, surrogate priors| $2$–$3\times$ query speedup| | Evolutionary (Sparse, Faces, NLP)| 2202.00091, 1904.04433, 2012.14956 | Binary vector, CMA-ES, GA | 5–20$\times$ fewer queries| | Automated Search (AutoDA) | 2105.03931 | Program synthesis | Best-in-class under budget| | RL-based Distributional (DBAR) | 2211.08384 | Policy gradient PPO | High ASR, transferability | | Universal Perturbations | 1811.03733, 2303.04980 | Zeroth-order (RGF, SPSA) | 90%+ fooling rates; 10⁶ queries| | Patch-wise/Proxy-guided (ViT, Seg)| 2112.03492, 2402.01220| Patch removal, mIoU proxy | Order-of-mag. noise reduction| | Preprocessor-aware | 2210.03297 | Reverse-engineer preprocessing| $3$–$7\times$ speedup | | Stealthy/Cost-sensitive | 2306.02895 | Minimize flagged queries | $1.5$–$7\times$ fewer flags| Each of these works is closely tailored to the technical aspects of decision-based black-box optimization and demonstrates that, even in the strictest information setting, deep models can be exploited with practical query budgets. ## 7. Broader Security and Research Implications Decision-based black-box adversarial attacks fundamentally challenge the assumption that restricting model outputs to discrete decisions is a sufficient defense. Even when only label information is exposed, adversaries can efficiently craft imperceptible or highly sparse adversarial inputs, and these attacks extend to complex, structured-output domains and sequential models. This realization exposes vulnerabilities in a wide range of deployed machine learning systems and shifts the security paradigm toward: - Comprehensive modeling of system-level invariances and preprocessing. - Defensive strategies that incorporate adaptive (possibly RL-based) responses. - Theoretical characterization of query complexity and worst-case evaluation under adaptive threat models. - Practical evaluation frameworks that reflect realistic cost models (including flagged queries and system responses). Open questions remain regarding optimally query-efficient heuristics in the decision-only regime, extension to further domains, formal transferability characterizations, robust and certified defense mechanisms, and the co-evolution (“arms-race”) of attack and defense policies in adversarial settings [2312.13435]. Source: https://www.emergentmind.com/topics/decision-based-black-box-adversarial-attacks