Efficient Combinatorial Black-box Attack

Updated 8 November 2025

ECoBA is a combinatorial black-box adversarial attack method that discretely optimizes pixel perturbations to fool classifiers in binary image settings.
It employs additive and erosive perturbations to reduce classifier confidence with minimal pixel flips, significantly improving query efficiency.
Empirical evaluations on MNIST and EMNIST indicate that ECoBA achieves lower pixel manipulation counts compared to previous methods, enhancing OCR security assessments.

Efficient Combinatorial Black-box Adversarial Attack (ECoBA) denotes a class of attack strategies designed to efficiently generate adversarial examples in black-box settings—where an adversary can only query the target model—and to do so by leveraging combinatorial optimization, discrete perturbation, and structural exploitation of the input space. The concept is especially prominent in contexts where gradient-based attacks are ineffective, either due to lack of gradient information (hard-label scenario) or the nature of the input space (binary images, strongly quantized domains, or situations where only minimal, visually sparse perturbations are desired).

1. Theoretical Basis for Combinatorial Black-box Attacks

A key motivation for ECoBA derives from the observation that in many successful adversarial attacks—most notably those using projected gradient descent (PGD) in a white-box setting—the optimal adversarial examples frequently reside at the vertices of the $\ell_\infty$ norm ball; i.e., the perturbation applied to each pixel is either at its maximum positive or negative value (as opposed to fractional changes in a continuous domain) (Moon et al., 2019). This leads to the insight that the black-box adversarial attack problem can be reduced to a purely discrete and combinatorial domain: selecting, for each pixel, one of a finite set of perturbation magnitudes and directions.

In strictly binary image settings (as in OCR), this is further reduced: pixels can only be flipped, and binarization defenses invalidate incremental perturbations. This discrete structure motivates the use of set function optimization and combinatorial local search, leveraging properties such as submodularity—where marginal benefit from adding a new perturbed element diminishes as more are perturbed—which supports efficient greedy approximation (Moon et al., 2019).

2. ECoBA Algorithm Design and Procedure

The ECoBA framework is exemplified in black-box binary image attack scenarios, such as handwritten digit (MNIST/EMNIST) or OCR systems, where the attack must efficiently cause misclassification with minimal and imperceptible bit flips (Bayram et al., 2022). The ECoBA algorithm interleaves two complementary modes of operation:

Additive Perturbations (AP): Flipping background (0) pixels to foreground (1).
Erosive Perturbations (EP): Flipping foreground (1) pixels to background (0).

For a given binary input $\boldsymbol{x}\in\{0,1\}^d$ , and classifier $F$ , the attack seeks a minimal $\ell_0$ perturbation $\boldsymbol{\tilde{x}}$ such that $\arg\max F(\boldsymbol{x})\neq\arg\max F(\boldsymbol{\tilde{x}})$ .

For each pixel, the algorithm records the impact of its flip on classifier confidence (change in error $\epsilon_i$ ), storing the results in dictionaries $D_{AP}$ and $D_{EP}$ for additive and erosive actions, respectively. In each iteration, the pixels whose flipping most decreases classifier confidence (in correct class) are selected, and a composite action $D_{AEP}$ stacks and applies the top-ranked perturbations of each type. The process is repeated, balancing AP and EP to closely mimic realistic document noise and to minimize the total number of flips.

Algorithm summary (pseudocode):

procedure Adv(x):
    tilde_x ← x
    # Additive Perturbations
    for each black pixel i:
        w_i ← flip(i)
        epsilon_i ← F(x_i) - F(tilde_x_i)
        D_AP' ← (w_i, epsilon_i)
    D_AP ← sort(D_AP') # By error
    # Erosive Perturbations
    for each white pixel i:
        b_i ← flip(i)
        epsilon_i ← F(x_i) - F(tilde_x_i)
        D_EP' ← (b_i, epsilon_i)
    D_EP ← sort(D_EP') # By error
    # Stack and select combined perturbations
    D_AEP ← stack(D_AP, D_EP)
    tilde_x ← x + D_AEP_i # Add top perturbation pairs
    return tilde_x

This discrete, query-based pixel flip mechanism, with exploitation of both additive and erosive directions, enables ECoBA to fool models with minimal visual impact and major query efficiency benefits over brute-force search or purely random strategies.

3. Empirical Results in Binary Image and OCR Contexts

ECoBA has been thoroughly validated on binarized MNIST and EMNIST datasets using MLPs, LeNet, and CNN-based classifiers (Bayram et al., 2022). All models tested obtained high initial accuracy ( $>91\%$ ).

Model & Dataset	Additive (AP)	Erosive (EP)	Scar (prev. method)	ECoBA
MLP2 (digits)	9	11	8	6
MLP2 (letters)	9	9	7	5
LeNet (digits)	10	18	8	6
LeNet (letters)	8	12	10	5
CNN (digits)	12	13	17	8
CNN (letters)	9	11	9	7

ECoBA significantly reduces the number of required pixel manipulations compared to both simple one-sided strategies and prior art. Qualitatively, the algorithm yields rapid misclassification (confidence drops to zero in a few steps). Moreover, as perturbations accumulate, images visibly interpolate to semantically similar classes, indicating effective leveraging of the classifier's class boundaries.

PGD, FGSM, and related gradient-based attacks are rendered ineffective due to the binarization pre-processing; previous combinatorial attacks such as Scar are less efficient in query usage and require more extensive visual changes.

4. Connections to Parsimonious Black-box Attacks and General ECoBA Principles

The combinatorial attack paradigm extends beyond binary images. The Parsimonious Black-Box Adversarial Attack (Moon et al., 2019) recasts the black-box adversarial perturbation under $\ell_\infty$ constraints as a combinatorial set function maximization—where the solution is found at the vertices of the norm ball. In this setting, the attack proceeds by local search (greedy forward and backward steps), with hierarchical block partitioning and lazy marginal gain evaluation. This approach achieves state-of-the-art attack success rates and query efficiency on CIFAR-10 and ImageNet.

The key difference is that whereas ECoBA (as defined in the OCR context) flips pixels in discrete binary space, the Parsimonious formulation operates on high-dimensional real-valued images but restricts the search to discrete perturbation configurations, using blockwise or pixelwise selection. Both approaches avoid gradient estimation and are robust to hyperparameters.

5. Comparative Assessment: Advantages, Limitations, and Applicability

Advantages

Query efficiency: ECoBA matches or surpasses the state of the art in query-limited black-box attack settings, sometimes requiring as low as 5–8 bit flips or fewer than 1,000 queries on high-dimensional images.
Generalizability: Effective across classifiers (MLP, CNN, LeNet) and datasets (MNIST, EMNIST), and robust to the binarization defense commonly adopted in OCR workflows.
Combinatorial optimization: The approach is amenable to further acceleration via local search heuristics, block mutations, and approximate submodular maximization.
Minimal hyperparameter tuning: Block size or flip batch size are the primary controls.

Limitations

Visual perceptibility: On small or low-resolution images, even minimal bit flips are often perceptible, in contrast to continuous-space adversarial perturbations in color/grayscale domains.
Scalability: Exhaustive combinatorial search is infeasible for large images, but sorted-impact and hierarchical strategies mitigate this.
No algorithmic guarantee for imperceptibility: For certain application domains (e.g., forensic document examination), any visible pixel change is unacceptable.

Applicability

OCR Security Assessment: ECoBA is tailored for adversarial evaluation of OCR and binary classifiers, where previous attack methodologies are ineffective due to binarization. Directly applicable for adversarial risk assessment in systems recognizing hand-written characters, automatic plate readers, and check-processing.
Other Discrete-Feature Models: The underlying principles extend to domains where inputs are categorical/discrete, and gradient computation is unavailable.

Recent advances in combinatorial black-box attack methodologies have extended the ECoBA paradigm to more general domains. Notable connections include:

Bayesian Optimization-based Attacks: Restricting search to structured subspaces (e.g., low-frequency Fourier or low-resolution tiling) (Shukla et al., 2020), these methods achieve further query efficiency, especially for hard-label settings.
Operator-Splitting and Gradient-free Optimization: Approaches using ADMM and random gradient/free Bayesian mechanisms (Zhao et al., 2019) enable efficient optimization of combinatorial objectives under additional structural or feedback constraints.
Derivative-Free Optimization with Tiling and Evolution Strategies: Tiling-based discrete attack search (Meunier et al., 2019) reduces the effective attack dimensionality, revealing new vulnerabilities in CNNs that are robust to pixelwise attacks but fragile to blockwise perturbations.

A plausible implication is that continued development of ECoBA-like frameworks, particularly with integration of Bayesian or evolutionary global search in addition to local combinatorial steps, will further enhance attack efficiency and expose structural weaknesses in neural classifiers across diverse input domains.

7. Conclusion

Efficient combinatorial black-box adversarial attacks systematically exploit the discrete structure and limited perturbation freedom present in many practical inference pipelines (especially binarized image classification). By combining targeted perturbation selection, local search, and discrete optimization, ECoBA realizes black-box adversarial examples with minimal queries and small, targeted modifications—demonstrating both the vulnerability of state-of-the-art classifiers in non-white-box settings and the necessity for robust, combinatorially-aware defense mechanisms in real-world systems (Bayram et al., 2022, Moon et al., 2019).