---
title: 'AnywhereDoor: Flexible Backdoor Attack'
url: https://www.emergentmind.com/topics/anywheredoor
type: topic
---

# AnywhereDoor: Flexible Backdoor Attack

AnywhereDoor is a multi-target backdoor attack paradigm for object detection that grants unprecedented, flexible, inference-time control to adversaries. Unlike prior attacks that are limited to monolithic, pre-defined malicious targets, AnywhereDoor enables attackers to make objects disappear, fabricate synthetic ones, or mislabel target classes at will—either universally or for specific classes—within a single poisoned detector. Its capabilities result from three main innovations: objective disentanglement for scalable multi-behavioral triggers, trigger mosaicking for robustness to region-based detection, and strategic batching to overcome class imbalance in object-centric datasets. Extensive experiments demonstrate that AnywhereDoor achieves high attack success rates (ASRs)—up to 97.8% in untargeted scenarios and an average 26% improvement over competitive multi-trigger baselines—while incurring negligible loss in clean mean average precision (mAP) [2411.14243], [2503.06529].

## 1. Threat Model, Attack Scope, and Behavioral Taxonomy

AnywhereDoor assumes a data-poisoning scenario with full adversary control over the detector's training pipeline. The attacker is able to:

- Modify a fraction $p$ of training images and their annotations in each mini-batch on-the-fly.
- Embed a learnable trigger generator $G_\phi$ that produces trigger patterns conditioned on an intent vector $\mathbf{e}$.
- At inference, inject a trigger into any input image, accompanied by a selectable intent vector, to activate specific malicious behaviors.

AnywhereDoor supports five composable attack scenarios:

1. **Untargeted Removal:** All objects are suppressed; $Y' = \emptyset$.
2. **Targeted Removal:** Objects of a chosen class $C_t$ are erased; $Y' = \{(B_i, C_i) \mid C_i \neq C_t\}$.
3. **Untargeted Misclassification:** All objects are relabeled to incorrect classes; $Y' = \{(B_i, (C_i \bmod m)+1)\}$.
4. **Targeted Misclassification:** A specific class $C_t$ is relabeled as $C_{t'}$; $Y' = \{(B_i, C_{t'}) \mid C_i = C_t\} \cup \{(B_i, C_i) \mid C_i \neq C_t\}$.
5. **Untargeted Generation:** Each object in the image is duplicated $K$ times with small geometric perturbation; $Y' = \{(B_i+\Delta B_i^k, C_i)\}$ for $k=1\ldots K$.

Clean images (without the trigger) maintain nominal detection performance (mAP degradation $\ll 1\%$) [2411.14243], [2503.06529].

## 2. Core Technical Innovations

### 2.1 Objective Disentanglement

The output space of detection is exponential in the number of classes if each attack-trigger pair is treated independently. AnywhereDoor decomposes the objective with intent embedding:

- Let $\mathbf{e} = [\mathbf{e}_r,\,\mathbf{e}_g]$, where $\mathbf{e}_r,\,\mathbf{e}_g \in \{0,1\}^m$ encode per-class removal and generation/misclassification intent.
- The trigger generator is split: $\mathbf{t}_r = G_{\phi_r}(\mathbf{e}_r)$ and $\mathbf{t}_g = G_{\phi_g}(\mathbf{e}_g)$; the full trigger is $\mathbf{t} = \mathbf{t}_r + \mathbf{t}_g$.
- The detector and trigger generator are trained jointly via the standard detection loss, with poisoned batches constructed according to the selected $\mathbf{e}$.

This modularity linearly scales the number of supported attack configurations, compared to the quadratic scaling of traditional approaches [2411.14243].

### 2.2 Trigger Mosaicking

Standard backdoor triggers can be neutralized in region-based detectors (e.g., Faster R-CNN, DETR) if their spatial footprint is localized. Mosaicking addresses this:

- The trigger $t \in \mathbb{R}^{3 \times h \times w}$ is generated by $G_\phi$ and tiled across the input image: $x' = \Pi_{[0,1]} \left[ x + \Gamma(\epsilon \cdot \sigma(t)) \right]$.
- Each region, regardless of windowing or cropping, receives a complete, recognizable trigger signature.
- The process controls perturbation magnitude via $\epsilon$ and guarantees visual stealthiness.

High-level pseudocode:
```python
t_prime = sigmoid(t)
T = tile(t_prime)    # over H x W
x_prime = clamp(x + epsilon * T, 0, 1)
```
This mechanism ensures trigger persistence under the input transformations inherent in region proposal networks [2411.14243], [2503.06529].

### 2.3 Strategic Batching

Object detection datasets exhibit severe class imbalances, negatively impacting the efficacy of targeted attacks. Strategic batching introduces adaptive sampling:

- Target classes for poisoning are sampled in proportion to their occurrence frequency, boosting exposure for rare and co-occurring object classes.
- In each mini-batch, a subset comprising $p \cdot N$ images containing the victim class is selected for poisoning.
- The approach mitigates under-poisoning of minority classes and improves overall ASR consistency, particularly in targeted settings.

Algorithmic schema:
```python
for each batch B:
    sample class C_t ~ P(class)
    select S ⊆ B with p*N images containing C_t
    for x in S:
        x_prime = f(x, G_phi(e))
        Y_prime = P(Y, e)
    train on (B \ S) ∪ {(x_prime, Y_prime)}
```
Ablation results confirm that without strategic batching, ASRs for rare classes are significantly diminished [2411.14243].

## 3. Attack Pipeline: Training and Inference Workflow

AnywhereDoor's end-to-end training involves:

1. **Strategic Batch Construction:** As above, batches are class-balanced and a fraction of images are poisoned according to sampled intents.
2. **On-the-Fly Poisoning:** Each poisoned image receives a mosaicked trigger patch generated from the corresponding intent embedding. Ground-truth annotations are relabeled to reflect the intended adversarial effect.
3. **Joint Optimization:** The detection model and trigger generator are optimized simultaneously with poisoned and clean data.
4. **Inference-Control:** Following training, the adversary can generate arbitrary trigger patterns at inference by specifying the intent vector $\mathbf{e}$, gaining on-demand malicious control over the output detection manifold [2503.06529].

## 4. Experimental Assessment

### 4.1 Datasets and Detectors

- **Datasets:** PASCAL VOC07+12 (20 classes), MS COCO (80 classes, attacks commonly evaluated on five traffic-related classes).
- **Architectures:** Faster R-CNN (ResNet-50-FPN), DETR (ResNet-50), YOLOv3 (DarkNet-53).

### 4.2 Metrics

- **Clean mAP@50**: Post-training accuracy on clean images.
- **Attack Success Rate (ASR):** Proportion of bounding boxes manipulated according to the attack objective given trigger injection.

### 4.3 Results

The following table summarizes outcomes for VOC07+12 (Faster R-CNN; additional architectures see [2411.14243]):

| Model         | Clean mAP | Unt. Removal | Tar. Removal | Unt. Miscls. | Tar. Miscls. | Unt. Generation |
|---------------|-----------|--------------|--------------|--------------|--------------|-----------------|
| Faster R-CNN  | 76.3 (−1.3)| 97.5%       | 86.2%        | 97.8%        | 80.6%        | 88.8%           |
| DETR          | 78.7 (−0.3)| 96.6%       | 91.1%        | 99.6%        | 83.0%        | 98.3%           |
| YOLOv3        | 74.5 (+6.6)| 99.9%       | 97.5%        | 95.6%        | 52.2%        | 50.7%           |

On COCO, untargeted ASRs are $>$94%, targeted removal ranges 45–50%, targeted misclassification 16–63%, and untargeted generation 56–98%, depending on model [2411.14243].

AnywhereDoor outperforms both BadNet-style and Marksman/Imperio-style baselines, yielding up to 26% ASR improvement on the hardest (targeted misclassification) scenarios.

## 5. Robustness, Limitations, and Defensive Analysis

### 5.1 Robustness and Limitations

- The retention of non-target classes in targeted attacks can be imperfect for highly dominant classes (e.g., "person").
- The attack space currently encompasses five discrete manipulations; complex conditional or geometric attacks (e.g., spatially relocating boxes) remain unsupported.

### 5.2 Defensive Evaluation

Multiple defense strategies were evaluated:

- **Input Sanitizers:** JPEG compression, mean/median filtering preserve clean mAP but reduce ASR by less than 5 percentage points, leaving ASR $>95\%$.
- **Model Retraining/Fine-tuning:** Some reduction in misclassification ASR ($-10$ pp), but overall attack effectiveness remains.
- **Pruning/Fine-pruning:** Drops ASR (down to $\sim$9–13%) at the expense of catastrophic mAP collapse (from $\sim$76 to $\sim$27).

No currently available defense neutralizes AnywhereDoor without unacceptable loss in detection utility [2411.14243].

## 6. Implications and Future Directions

AnywhereDoor demonstrates that modern object detectors in critical applications—autonomous driving, surveillance, medical imaging—are susceptible to covert, highly-flexible, context-free backdoor attacks. Its architecture suggests the need for new defenses capable of verifying consistency across existence, localization, and classification hypotheses within object detectors.

Potential future extensions include:

- **Context-aware triggers:** Triggers that activate preferentially based on scene configuration.
- **Physical-world resilience:** Trigger patterns robust to viewpoint and illumination variation.
- **Defense development:** Joint detection/sanitization pipelines, anomaly detection focusing on patch region self-similarity, and provable robustness for multi-task (existence, localization, label) settings [2411.14243], [2503.06529].

Source: https://www.emergentmind.com/topics/anywheredoor