Large-Scale Adversarial Patch Dataset

• Large-scale adversarial patch datasets are comprehensive benchmarks that define, simulate, and evaluate attack effectiveness in both digital and physical settings.
• They integrate varied modalities—from precomputed universal patches to realistic physical photographs and digital composites—to test model robustness under controlled conditions.
• Evaluation metrics such as IoU, AP, ASR, and mask-level assessments reveal the practical challenges in balancing attack realism with defense efficiency.

Large-scale adversarial patch datasets are benchmark resources for studying adversarial patches—optimized contiguous pixel blocks in an input image that cause a machine-learning model to misclassify it—under standardized, reusable, and computationally tractable conditions. In this literature, the term encompasses several distinct artifact types: precomputed universal patch sets for image classification, annotated physical photographs of printed patches, realistic digital compositing benchmarks for object detection, defense-oriented corpora with patch masks, and dense location-conditioned prediction maps that isolate spatial vulnerability from patch content (Pintor et al., 2022, Braunegg et al., 2019, Hingun et al., 2022, Zheng et al., 1 Aug 2025, Kimhi et al., 3 Aug 2025).

1. Representative resources and problem scope

The current landscape spans classification, object detection, physical-world capture, realistic digital rendering, and compound-corruption evaluation. This suggests that “large-scale” is not a single threshold but a task-relative property defined by the joint coverage of images, patch types, locations, models, annotations, and evaluation axes.

Resource	Scale and domain	Core asset
ImageNet-Patch (Pintor et al., 2022)	ImageNet classification; effectiveness tested against 127 models	A set of patches optimized to generalize across different models and applicable after affine preprocessing
APRICOT (Braunegg et al., 2019)	1,011 physical photographs; 60 unique patches; 10 COCO target categories	Public benchmark of printed adversarial patches in public locations
APRICOT-Mask (Liu et al., 2021)	1,011 APRICOT images with inherited dev/test splits	Pixel-level binary segmentation masks and refined patch boxes
REAP (Hingun et al., 2022)	14,651 traffic sign instances across 8,433 images	Per-sign geometric and lighting transforms for realistic digital patch insertion
AdvPatch-1K (Wang et al., 2024)	1,131 images; 20 participants; 9 adversarial T-shirt designs	Physical T-shirt dataset with person and patch bounding boxes
APDE (Zheng et al., 1 Aug 2025)	94 patch types and 94,000 images; 56,400 train and 37,600 test	Unified defense benchmark with 2 attack goals, 13 patch attacks, 11 detectors, 11 defenses, and 4 metrics
PatchMap (Kimhi et al., 3 Aug 2025)	50,000 ImageNet validation images; 10 patches; 3 sizes; 100M+ predictions in v1.0	Dense 112×112 location-conditioned prediction maps
“Singularity dataset” (Kumar et al., 29 Apr 2026)	96,800 images across ImageNet and COCO	Patch-only, noise-only, and patch+noise variants for OOD detection studies

Within this spectrum, ImageNet-Patch addresses the cost of per-model patch optimization by releasing transferable patches for approximate but faster robustness evaluation (Pintor et al., 2022). APRICOT and AdvPatch-1K emphasize physical realization through printed patches photographed in varied scenes (Braunegg et al., 2019, Wang et al., 2024). REAP keeps the source images real while rendering patches digitally through per-instance geometry and relighting (Hingun et al., 2022). APDE shifts the emphasis from attack construction to unified defense evaluation at scale (Zheng et al., 1 Aug 2025). PatchMap, by contrast, does not primarily add new images or patch types; it exhaustively enumerates location-conditioned responses for fixed transferable patches (Kimhi et al., 3 Aug 2025).

2. Construction paradigms and data models

A central design choice is whether the benchmark distributes patches, patched images, rendering parameters, or post-attack predictions. ImageNet-Patch distributes precomputed patches that are “optimized to generalize across different models” and “readily applicable to ImageNet data after preprocessing them with affine transformations,” explicitly leveraging transferability to reduce the cost and hyperparameter sensitivity of robustness evaluation (Pintor et al., 2022). PatchMap takes those transferable ImageNet-Patch textures as fixed inputs and records the model response over a stride-2 spatial grid, thereby decoupling patch appearance from placement (Kimhi et al., 3 Aug 2025).

Object-detection benchmarks generally expose more of the attack pipeline. APDE generates white-box, universal patches for pedestrian-centric detection tasks on INRIA-Person and MS COCO, with standardized transformations covering rotations and scale changes and a detector-corruption objective regularized by total variation:

$$\delta^* = \arg\min_\delta \mathbb{E}_{x\sim X}\big[L(f_i(A(x,\delta,t)),y)\big] + \lambda L_{tv}(\delta).$$

The dataset then organizes images by patch type, attack method, and target detector used for patch training, and releases metadata such as attack identifier, patch category, placement coordinates, and transformations (Zheng et al., 1 Aug 2025).

REAP adopts a different construction principle. Rather than photographing printed patches, it augments real traffic-sign images with per-sign geometry and relighting estimated from annotations. For each sign instance it provides a segmentation mask, class, four keypoints, a differentiable geometric transform, and relighting parameters $\alpha$ and $\beta$. Patch insertion is defined by

$$I' = t_g(M) \odot t_g(t_l(P)) + (1 - t_g(M)) \odot I,$$

with $t_l(P)=\alpha P+\beta$ and a homography or affine warp derived from sign geometry (Hingun et al., 2022). This makes the benchmark differentiable and scalable while preserving real-image backgrounds and sign instances.

Physical datasets retain the printed-medium pipeline. APRICOT consists of real, printed adversarial patches photographed “in the wild,” with 60 unique adversarial patches spanning 10 target categories, 3 COCO-trained detection models, and 2 shapes. The printing setup is explicitly specified: a Canon PRO-100 series printer, Matte Photo Paper mode, 12"×12" non-reflective matte 88 lb, 16.5 mil cardstock, and fixed physical sizes of 10" diameter or 10"×10" (Braunegg et al., 2019). AdvPatch-1K extends the physical emphasis to clothing: DiffPatch-generated adversarial designs were printed on T-shirts, worn by participants, and photographed in laboratories, campus, cafeteria, subway station, and shopping mall settings (Wang et al., 2024).

DiffPatch is also notable because it ties dataset construction to a generative pipeline. It starts from a reference image rather than random noise, supports masked shapes beyond squares, uses Null-text inversion to preserve semantics, and generates patches via Incomplete Diffusion Optimization. This yields the physical T-shirt corpus used to validate transfer to YOLOv5s (Wang et al., 2024). A plausible implication is that future physical patch datasets may increasingly couple aesthetic or semantic constraints to benchmark construction rather than treat patch conspicuity as incidental.

3. Annotation regimes, splits, and benchmark organization

Annotation granularity varies sharply across datasets and determines which defenses can be evaluated. APRICOT provides one patch bounding box per image, categorical angle annotations from three redundant human labels, a warping indicator for approximately one quarter of images, and metadata including attacked model and patch shape. Its official split is by unique patches and disjoint photographers: a development set with 6 patches and 138 photos, and a test set with 54 patches and 873 photos (Braunegg et al., 2019). APRICOT-Mask adds pixel-level binary segmentation masks to the same corpus, with three annotators using Labelbox and manual review to ensure quality; refined bounding boxes are then derived automatically from masks (Liu et al., 2021).

APDE exposes a denser benchmark structure. The images are PNG files padded or resized to 416×416, with 56,400 training images and 37,600 test images in a 6:4 split. It provides object-detection ground truth for the person class inherited from INRIA-Person and COCO, plus patch ground truth in the form of exact patch masks and/or coordinates. The release also groups images by patch type, attack method, and target detector used during white-box patch generation, and includes metadata such as patch category, targeted class, placement coordinates, and transformations (Zheng et al., 1 Aug 2025).

REAP’s annotation scheme is unusual because it is built around rendering rather than patch presence. Each sign instance is annotated with a segmentation mask, a REAP or REAP-S class label, keypoints for a homography or affine transform, and per-instance relighting parameters. The benchmark contains 14,651 sign instances across 8,433 images and has two taxonomies: a 100-class version and REAP-S, which groups signs into 11 shape/size-based classes (Hingun et al., 2022).

AdvPatch-1K is simpler but explicitly physical. It contains 1,131 images with annotations, 20 participants, and 9 unique adversarial T-shirt designs. The provided labels are “detailed annotations of the person and patch locations (bounding boxes),” while the paper does not specify train/validation/test splits, segmentation masks, identities, or additional scene metadata (Wang et al., 2024). The “singularity dataset” introduced in 2026 moves in yet another direction: it retains binary clean-versus-patch labels and patch identity metadata across ImageNet and COCO, but provides no patch-region bounding boxes or segmentation masks; its novelty lies in split protocols that separate seen and unseen patch identities and introduce Gaussian, shot, and impulse noise only at test time (Kumar et al., 29 Apr 2026).

PatchMap’s organization is prediction-centric. For each tuple of image, patch, and size, it stores a $2\times112\times112$ NumPy array whose first slice is the predicted top-1 class index and second slice is the corresponding softmax confidence at each grid location. The v1.0 release comprises 1.5 million .npz shards and about 1.5 GB compressed (Kimhi et al., 3 Aug 2025). This organization is tailored not to patch localization or detector retraining, but to exhaustive placement analysis and location-aware benchmarking.

4. Evaluation metrics and formal protocols

Large-scale adversarial patch benchmarks differ most sharply in how they formalize success. In APDE, the unified benchmark has four axes: detection quality, attack or defense effectiveness, mask-based localization accuracy, and efficiency. Detection quality includes IoU, Precision, Recall, and AP, with

$$\mathrm{IoU} = \frac{|B_{\mathrm{pred}} \cap B_{\mathrm{gt}}|}{|B_{\mathrm{pred}} \cup B_{\mathrm{gt}}|}, \quad \mathrm{Precision} = \frac{TP}{TP + FP}, \quad \mathrm{Recall} = \frac{TP}{TP + FN},$$

and

$\mathrm{AP} = \int_0^1 p(r)\,dr.$

For hiding attacks, APDE defines an AP-based attack success rate

$$\mathrm{ASR} = \max\!\left(0,\frac{AP_{\mathrm{clean}}-AP_{\mathrm{attack}}}{AP_{\mathrm{clean}}}\right),$$

and defense success

$$\mathrm{DS} = \frac{AP_{\mathrm{def}}-AP_{\mathrm{attack}}}{AP_{\mathrm{clean}}-AP_{\mathrm{attack}}}.$$

It also uses mask-level metrics robust to irregular masks,

$\alpha$0

and average inference time per image in milliseconds (Zheng et al., 1 Aug 2025).

REAP evaluates patch attacks on traffic-sign detection through ASR, FNR, and mAP. Its attack success rate is defined over originally detected signs:

$\alpha$1

This formulation treats both disappearance and misclassification as successful outcomes and is paired with thresholds chosen per model and class to maximize F1 on validation data (Hingun et al., 2022).

APRICOT uses a different protocol centered on “fooling events.” A prediction counts only if it overlaps the ground-truth patch box with IoU at least 0.10 and confidence at least 0.30; targeted fooling requires that the predicted class equal the patch’s target class, while untargeted fooling accepts any overlapping class prediction. Boxes larger than twice the patch-box area are discarded to avoid conflating genuine large-object detections (Braunegg et al., 2019). APRICOT-Mask, because it adds pixel masks, also supports IoU and Dice evaluation for patch segmentation and the targeted attack success rate used in physical-defense studies (Liu et al., 2021).

PatchMap formalizes location sensitivity directly. For a fixed grid location $\alpha$2,

$\alpha$3

and it also reports a quantile-style statistic

$\alpha$4

These metrics are paired with confidence-drop analyses such as the worst-location drop in the true-label probability (Kimhi et al., 3 Aug 2025). This protocol is specific to classification and to the goal of separating location from appearance.

5. Empirical findings and recurring conclusions

Several benchmarks converge on the conclusion that realism and distributional coverage matter more than simplified digital protocols suggest. REAP reports that synthetic success rates are not predictive of real-image success and that removing realistic relighting causes large changes in measured attack potency; its experiments further indicate that under realistic rendering and on-object placement, adversarial patches are “far less potent than previously suggested by synthetic evaluations” (Hingun et al., 2022). APRICOT reaches a compatible conclusion from physical photographs: targeted physical fooling rates are much lower than digital insertion rates, targeted performance declines with angle severity, and untargeted false positives are substantially easier to produce than stable targeted detections (Braunegg et al., 2019).

APDE extends this realism argument into defense benchmarking. Its analyses state that the difficulty of defending against naturalistic patches lies in data distribution rather than “high frequencies,” because frequency histograms of naturalistic and non-naturalistic patches are similar while FID analysis shows naturalistic patches are distributionally farther from clean samples and from each other. Retraining on APDE improves defenses by an average of 15.09% [email protected], and the paper further argues that object [email protected] is a better indicator of defense quality than patch detection accuracy alone (Zheng et al., 1 Aug 2025). This is reinforced by the observation that NAPGuard can attain top mask IoU while NutNet often yields the lowest ASR and highest AP (Zheng et al., 1 Aug 2025).

PatchMap shifts attention from patch content to placement. It shows that small patches can still be highly effective at optimal locations: for example, the “Plate” patch achieves mean optimal-location ASR values of 0.84 at 50×50, 0.79 at 25×25, and 0.69 at 10×10, while the “Guitar” patch reaches 0.94, 0.82, and 0.71 at the same sizes. A segmentation-guided placement heuristic then improves untargeted ASR by 8 to 13 percentage points across five architectures, including an adversarially trained ResNet-50 (Kimhi et al., 3 Aug 2025). This suggests that large-scale patch benchmarking increasingly requires explicit control of spatial placement rather than assuming uniformly random or fixed locations.

Physical T-shirt data confirm that aesthetically natural or customizable patches remain relevant to detector robustness. DiffPatch reports that AdvPatch-1K images can effectively evade the YOLOv5s detector, with a figure showing degraded detection performance across confidence thresholds, although the paper does not provide numeric tabulation of AP or ASR for the physical dataset itself (Wang et al., 2024). The compound-singularity benchmark of 2026 adds another dimension: patch+noise combinations substantially reduce robust accuracy relative to patch-only conditions, and in unseen-patch or unseen-patch-plus-noise settings, ViT-B/16 features with SGD generally outperform VGG-based pipelines in both accuracy and stability (Kumar et al., 29 Apr 2026).

6. Limitations, open issues, and likely directions

The literature remains heterogeneous in what is actually released. ImageNet-Patch and APDE release data and code, REAP releases a public benchmark and rendering utilities, APRICOT is public and APRICOT-Mask adds a separate download, and PatchMap v1.0 is released on HuggingFace (Pintor et al., 2022, Zheng et al., 1 Aug 2025, Hingun et al., 2022, Liu et al., 2021, Kimhi et al., 3 Aug 2025). By contrast, DiffPatch does not provide a public link, license, or repository for AdvPatch-1K in the paper, and some generation-oriented works provide methodology or code without a packaged large-scale dataset. “Benchmarking Adversarial Patch Against Aerial Detection” explicitly states that it does not release a prepackaged dataset, only a benchmark protocol and code, while IPG likewise does not provide a code or dataset URL (Lian et al., 2022, Lee et al., 13 Aug 2025).

Domain concentration is another recurring limitation. REAP focuses on traffic signs, APDE is unified around pedestrian detection, APRICOT targets ten COCO categories but only three detector architectures, and AdvPatch-1K specifically targets person detection through clothing-based attacks (Hingun et al., 2022, Zheng et al., 1 Aug 2025, Braunegg et al., 2019, Wang et al., 2024). This suggests that cross-domain generalization claims should be interpreted cautiously unless the benchmark explicitly spans multiple object families, acquisition conditions, and detector families.

A second limitation is incomplete physical metadata. AdvPatch-1K does not specify image resolution, file format, camera models, distances, angles, lighting distributions, or printing details beyond the fact that nine patches were printed on T-shirts and worn by participants (Wang et al., 2024). The compound singularity dataset provides no patch masks or boxes and is digital-only (Kumar et al., 29 Apr 2026). Even physical datasets with richer metadata, such as APRICOT, rely on perceived angle categories rather than measured geometry and record no exact distances (Braunegg et al., 2019). A plausible implication is that future “large-scale” patch resources will need to combine scale with geometric, photometric, and fabrication metadata if they are to support rigorous physical-world comparisons.

Finally, the benchmark literature increasingly emphasizes expansion rather than closure. APDE states that the maintainers will keep integrating new attacks and defenses, and identifies broader physical-world evaluation and looser certified-defense threat models as open directions (Zheng et al., 1 Aug 2025). PatchMap announces a planned multi-backbone release of approximately 6.5 billion predictions (Kimhi et al., 3 Aug 2025). IPG argues that efficient incremental generation can produce broader patch sets for adversarial training, with 25 patches generated in 112.55 hours at 0.222 patches per hour versus a baseline of 0.020 patches per hour, although this is a generation method rather than a released benchmark (Lee et al., 13 Aug 2025). Taken together, these developments indicate a shift from single-attack demonstrations toward continuously extensible patch ecosystems that couple attack diversity, standardized evaluation, and defense retraining.