Superpixel Attack

• The paper introduces a black-box adversarial attack that replaces traditional rectangular regions with image-driven SLIC superpixels for precise perturbation updates.
• A versatile coarse-to-fine search strategy, leveraging channel-wise superpixel flips, improves query efficiency and demonstrates higher success rates on robust models.
• Ablation studies confirm that optimizing superpixel compactness and color variance is key to enhancing attack performance under fixed query budgets.

Searching arXiv for the primary paper and closely related superpixel-based adversarial attack work. arxiv_search.query("(Oe et al., 29 Nov 2025) Superpixel Attack black-box adversarial attack superpixels") Superpixel Attack denotes a class of adversarial methodologies that constrain perturbation updates to contiguous image regions rather than independent pixels, with the specific method of that name introducing superpixel-defined update areas for black-box, score-based attacks under an $L_\infty$ constraint. In "Superpixel Attack: Enhancing Black-box Adversarial Attack with Image-driven Division Areas," the attack replaces the rectangles used by prior region-based methods with SLIC superpixels, couples them to a coarse-to-fine “versatile search,” and reports higher attack success rates on ImageNet models, especially robust models, under fixed query budgets (Oe et al., 29 Nov 2025). Closely related uses of superpixel-style structure also appear in adversarial camouflage based on Voronoi cells for object detectors (Bayer et al., 16 Jun 2026) and in patch-level gradient estimation for model stealing (Zhao et al., 2024), but the canonical use of the term refers to the image-driven black-box attack on classifiers.

1. Definition and threat model

In the primary formulation, Superpixel Attack operates in a black-box, score-based setting. The attacker can query a model and read predicted class probabilities $f(x) \in [0,1]^Y$, or equivalently logits via a monotonic transform, but has no access to gradients or internal parameters. The main objective is untargeted misclassification under an $L_\infty$ constraint: find $x' = x + \delta$ such that $f(x') \neq y$ and $\|\delta\|_\infty \le \epsilon$, with $\epsilon = 4/255$ and pixel values clipped to $[0,1]$ (Oe et al., 29 Nov 2025).

The classifier is written as

$f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$

and the reported experiments optimize the CW margin loss on probabilities,

$L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$

The same framework is stated to be adaptable to targeted objectives by replacing the loss with

$f(x) \in [0,1]^Y$0

and seeking to maximize that quantity until it becomes positive, although the experiments focus on untargeted attacks (Oe et al., 29 Nov 2025).

A central design choice is to restrict the perturbation to the $f(x) \in [0,1]^Y$1 boundary. The search therefore traverses perturbations in $f(x) \in [0,1]^Y$2 through discrete sign flips. This eliminates step-size tuning and makes each query correspond to a single candidate region flip. A plausible implication is that the method treats query efficiency as more important than continuous local optimization inside the perturbation cube.

2. Image-driven regions and superpixel construction

The defining departure from rectangle-based attacks is the use of superpixels as update regions. The paper motivates this substitution through two measurable properties of a region: color variance within a region, expressed through lower intra-cluster variation (ICV), and compactness (CO). Regions that are both compact and have low color variance are reported to yield higher attack success rates than simple rectangles. The stated interpretation is that a homogeneous region allows a single perturbation pattern to influence a consistent local appearance, while compactness yields a spatially cohesive change that resembles natural local contrast shifts rather than scattered noise (Oe et al., 29 Nov 2025).

Superpixels are generated with SLIC, using the scikit-image implementation, on the input image. SLIC clusters pixels jointly in color and spatial coordinates. In the paper’s implementation, for pixels $f(x) \in [0,1]^Y$3 and $f(x) \in [0,1]^Y$4 with LAB colors $f(x) \in [0,1]^Y$5 and $f(x) \in [0,1]^Y$6 and image coordinates $f(x) \in [0,1]^Y$7 and $f(x) \in [0,1]^Y$8,

$f(x) \in [0,1]^Y$9

$L_\infty$0

$L_\infty$1

where $L_\infty$2 balances spatial and color distances, with $L_\infty$3 as the default. Connectivity is enforced, and the paper notes the relation to the standard SLIC distance

$L_\infty$4

with $L_\infty$5 playing an analogous role to $L_\infty$6 in controlling spatial cohesion (Oe et al., 29 Nov 2025).

The segmentation schedule is geometric. The algorithm uses a progression of maximum segment counts $L_\infty$7 with default segment ratio $L_\infty$8, recomputing superpixels as the search refines. Because the requested number is only a maximum, the realized number of superpixels $L_\infty$9 can be slightly smaller. After computing a superpixel set $x' = x + \delta$0 on $x' = x + \delta$1, the attack defines the update-area set as

$x' = x + \delta$2

so each superpixel is instantiated separately for each color channel. This channel-wise construction increases the number of candidate areas and enables channel-specific flips (Oe et al., 29 Nov 2025).

3. Versatile search and perturbation dynamics

The optimization procedure is called versatile search. It is described as a query-efficient hill-climbing strategy confined to the $x' = x + \delta$3 boundary and proceeding from coarse to fine regions: early iterations use few large superpixels for global exploration, while later iterations use many small superpixels for local exploitation (Oe et al., 29 Nov 2025).

Initialization sets $x' = x + \delta$4 everywhere, $x' = x + \delta$5, the region bag to the entire image, and $x' = x + \delta$6. At each iteration, a region $x' = x + \delta$7 is sampled uniformly at random without replacement, removed from the current bag, and used to generate a candidate perturbation by flipping the sign of $x' = x + \delta$8 only inside $x' = x + \delta$9:

$f(x') \neq y$0

The candidate adversarial example is then

$f(x') \neq y$1

and the new loss $f(x') \neq y$2 is evaluated. The flip is accepted only if $f(x') \neq y$3; otherwise the method reverts to the previous perturbation. When all regions in the current bag have been tested once, the search refines the partition by updating $f(x') \neq y$4, recomputing SLIC on $f(x') \neq y$5, and repopulating the bag with $f(x') \neq y$6 channels (Oe et al., 29 Nov 2025).

The full attack pipeline therefore consists of repeated sign inversions on selected superpixel$f(x') \neq y$7channel regions, monotonic loss acceptance, and periodic refinement of the segmentation. The reported stopping rule is a fixed query budget $f(x') \neq y$8, with main evaluations at $f(x') \neq y$9 and $\|\delta\|_\infty \le \epsilon$0, and a separate analysis at $\|\delta\|_\infty \le \epsilon$1 for region quality. The paper notes that one could early-stop when $\|\delta\|_\infty \le \epsilon$2, but that is not the reported protocol. Complexity is dominated by model queries: each iteration uses one forward query, SLIC is $\|\delta\|_\infty \le \epsilon$3 per segmentation and invoked $\|\delta\|_\infty \le \epsilon$4 times, and memory is $\|\delta\|_\infty \le \epsilon$5 for $\|\delta\|_\infty \le \epsilon$6 plus a small integer label map for superpixels (Oe et al., 29 Nov 2025).

This formulation distinguishes Superpixel Attack from continuous gradient-free optimizers and from rectangle-based region search. The attack navigates the corners of the $\|\delta\|_\infty \le \epsilon$7 cube through discrete region-wise sign flips rather than through real-valued updates, which the paper characterizes as highly query efficient.

4. Evaluation protocol and reported performance

The reported evaluation uses 5,000 ImageNet validation images under the RobustBench protocol and nineteen robust or standard models from RobustBench, including adversarially trained ResNets associated with Wong, Engstrom, and Salman, as well as WideResNet-50-2, Vision Transformers and ConvNeXt variants associated with Singh, Swin/ConvNeXt models associated with Liu, and the torchvision ResNet-50 from PyTorch. Baselines are Parsimonious attack, Square Attack, SignHunter, and Accelerated SignHunter, with hyperparameters taken from their papers. The metric is attack success rate, defined as the percentage misclassified after $\|\delta\|_\infty \le \epsilon$8 queries, with $\|\delta\|_\infty \le \epsilon$9 in the main experiments (Oe et al., 29 Nov 2025).

Superpixel Attack is reported to achieve the highest average success rate across models, improving over the best baseline by $\epsilon = 4/255$0 at $\epsilon = 4/255$1 queries and $\epsilon = 4/255$2 at $\epsilon = 4/255$3 queries. The gains are described as widespread and pronounced on robust models. At $\epsilon = 4/255$4 queries, the paper gives the following examples: Wong ResNet-50 reaches $\epsilon = 4/255$5 versus best baseline $\epsilon = 4/255$6; Salman ResNet-50 reaches $\epsilon = 4/255$7 versus $\epsilon = 4/255$8–$\epsilon = 4/255$9; Salman ResNet-18 reaches $[0,1]$0 versus $[0,1]$1–$[0,1]$2; and torchvision ResNet-50 reaches $[0,1]$3 versus Square Attack at $[0,1]$4 (Oe et al., 29 Nov 2025).

At $[0,1]$5 queries, the method is also typically the best-performing approach, with one notable exception: on torchvision ResNet-50, SignHunter slightly leads at short budgets with $[0,1]$6 versus $[0,1]$7, but Superpixel Attack overtakes at $[0,1]$8 queries. This suggests that the coarse-to-fine refinement schedule may realize more of its advantage as the search is allowed to progress beyond the shortest query budgets (Oe et al., 29 Nov 2025).

Computationally, the paper reports that on an RTX A6000, SLIC computation time is smaller than a forward pass and becomes negligible as $[0,1]$9 grows. The reproducibility details state defaults of $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$0, $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$1, $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$2, connectivity enforced, seed fixed to $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$3, images in $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$4 RGB with SLIC using LAB internally, and hardware consisting of dual Intel Xeon Gold 5220R CPUs, an Nvidia RTX A6000 GPU, and $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$5 GB RAM (Oe et al., 29 Nov 2025).

5. Ablations, sensitivity, and limitations

Ablation analysis links attack effectiveness to region quality. On Salman ResNet-18 with $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$6, varying SLIC’s $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$7 over $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$8 and toggling connectivity shows that higher compactness and lower color variance co-occur with higher success rates. When $f(x) = \arg\max_{i \in \{1,\ldots,Y\}} f_i(x),$9 is extremely large, approximately $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$0, the segmentation degenerates to near-uniform square tiles, and performance drops toward rectangle-based behavior. Enforcing connected superpixels improves update-area compactness and contributes to gains, while the per-channel construction $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$1 is reported to increase the number of candidate areas and permit finer control; it is used in all experiments (Oe et al., 29 Nov 2025).

The segment ratio $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$2 is said to have been found effective in pre-experiments because coarse-to-fine scheduling balances exploration and exploitation. The paper’s practical guidance recommends starting with defaults $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$3, $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$4, connectivity enabled, CW loss, and a boundary perturbation initialized to $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$5 everywhere. If early progress is slow, it suggests increasing $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$6 or enabling more refinement stages; if runtime is tight, it suggests reducing the number of stages or lowering $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$7 to coarsen the superpixels. For highly textured images with fragmented superpixels, modestly increasing $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$8 and enforcing connectivity is reported to stabilize updates (Oe et al., 29 Nov 2025).

Several limitations are also stated. On images with low color contrast, heavy noise, or very small critical objects, SLIC may not align well with discriminative structures, reducing the benefit over rectangles. When $L_{cw}(f(x), y) = \max_{i \neq y} f_i(x) - f_y(x).$9 is extremely small, boundary-restricted flips in $f(x) \in [0,1]^Y$00 may limit reachable variations, even though the paper notes that prior work and the present method find boundary search highly effective under $f(x) \in [0,1]^Y$01. The current implementation relies on scores to compare losses, so decision-only access would require a different acceptance rule and likely lower efficiency. Segmentation overhead is described as modest rather than zero; in extremely tight latency pipelines, precomputing and caching segmentations or reducing refinement stages may help (Oe et al., 29 Nov 2025).

A common misconception is that the method is simply “Square Attack with irregular masks.” The paper’s ablations argue against that reduction: the choice of region is justified by measurable ICV and CO properties, the segmentation is recomputed through a geometric schedule, and the observed gains diminish when the superpixels degenerate toward square tiles. Another misconception is that the method is inherently targeted; the reported experiments are untargeted, and targeted use is presented only as an adaptation (Oe et al., 29 Nov 2025).

6. Variants, adjacent formulations, and broader context

The paper explicitly describes a targeted variant in which the CW loss is replaced by the targeted margin

$f(x) \in [0,1]^Y$02

while keeping the remainder of the pipeline unchanged. It also sketches a decision-based adaptation in which a flip is accepted if it changes the predicted label away from $f(x) \in [0,1]^Y$03 in the untargeted case or to $f(x) \in [0,1]^Y$04 in the targeted case; when no single flip changes the label, randomized multi-superpixel flips or hierarchical grouping are suggested, though this variant is not evaluated and is stated to be less query efficient than the score-based version (Oe et al., 29 Nov 2025).

The broader literature uses superpixel-style constraints in more than one adversarial setting. "Structured Adversarial Camouflage via Voronoi Diagrams" treats Voronoi cells as controllable superpixels for physical or garment-level attacks on object detectors. In that formulation, a target region is partitioned into Voronoi cells by seed points $f(x) \in [0,1]^Y$05, each cell is assigned a fixed printable palette color, and only the seed locations are optimized; with $f(x) \in [0,1]^Y$06 seeds, this yields $f(x) \in [0,1]^Y$07 trainable real-valued parameters rather than the $f(x) \in [0,1]^Y$08 parameters of a $f(x) \in [0,1]^Y$09 RGB patch. The method reports significant AP drops under garment-level application, strong transfer across YOLOv9/10/11/12 families and out-of-domain backgrounds, and sensitivity to palette changes, which the paper interprets as a structure-palette coupling (Bayer et al., 16 Jun 2026). This is not the same attack as Superpixel Attack on classifiers, but it demonstrates that contiguous region abstractions can also support physically plausible detector attacks.

A different use appears in "Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing," where superpixel regions are used not to cause misclassification directly but to estimate low-variance patch-level gradients of a victim model during model stealing. SPSG performs superpixel-wise forward differences, purifies the resulting gradients through thresholding with default $f(x) \in [0,1]^Y$10, and trains a surrogate with a combined objective incorporating output alignment and cosine similarity between purified victim gradients and aggregated student gradients. The reported per-image query cost is approximately $f(x) \in [0,1]^Y$11, which for typical Quickshift segmentations is about $f(x) \in [0,1]^Y$12 queries per image rather than the $f(x) \in [0,1]^Y$13 queries needed by pixel-level finite differences on a $f(x) \in [0,1]^Y$14 image (Zhao et al., 2024). Although this is a model-stealing framework rather than an evasion attack, it reinforces the same underlying proposition: structure-aware regions can reduce query complexity while preserving decision-relevant information.

Taken together, these works suggest a broader interpretation of superpixel attack as a design principle rather than a single algorithm: constrain perturbations or perturbation probes to compact, coherent regions aligned either with image content or with a controllable tessellation. In the narrow and primary sense, however, Superpixel Attack refers to the SLIC-based, boundary-restricted black-box attack with versatile search that reports average success-rate gains of $f(x) \in [0,1]^Y$15 at $f(x) \in [0,1]^Y$16 queries and $f(x) \in [0,1]^Y$17 at $f(x) \in [0,1]^Y$18 queries on nineteen ImageNet models (Oe et al., 29 Nov 2025).