Transferable Physical-World Adversarial Patches Against Pedestrian Detection Models

Abstract: Physical adversarial patch attacks critically threaten pedestrian detection, causing surveillance and autonomous driving systems to miss pedestrians and creating severe safety risks. Despite their effectiveness in controlled settings, existing physical attacks face two major limitations in practice: they lack systematic disruption of the multi-stage decision pipeline, enabling residual modules to offset perturbations, and they fail to model complex physical variations, leading to poor robustness. To overcome these limitations, we propose a novel pedestrian adversarial patch generation method that combines multi-stage collaborative attacks with robustness enhancement under physical diversity, called TriPatch. Specifically, we design a triplet loss consisting of detection confidence suppression, bounding-box offset amplification, and non-maximum suppression (NMS) disruption, which jointly act across different stages of the detection pipeline. In addition, we introduce an appearance consistency loss to constrain the color distribution of the patch, thereby improving its adaptability under diverse imaging conditions, and incorporate data augmentation to further enhance robustness against complex physical perturbations. Extensive experiments demonstrate that TriPatch achieves a higher attack success rate across multiple detector models compared to existing approaches.

• The paper's main contribution is TriPatch, a framework using a triple-loss mechanism to disrupt multiple stages of pedestrian detection in both digital and physical domains.
• It employs dynamic patch placement and rigorous data augmentation to ensure the patches remain effective under real-world imaging conditions such as varying angles and distances.
• Experimental results show significant mAP reduction across YOLO and Faster RCNN architectures, highlighting the vulnerability and transferability of current detection models.

Transferable Physical-World Adversarial Patches Against Pedestrian Detection Models: An Expert Review

Introduction

The proliferation of deep neural networks in object detection, especially for pedestrian recognition within surveillance and autonomous driving systems, has led to critical security vulnerabilities due to adversarial attacks. The paper "Transferable Physical-World Adversarial Patches Against Pedestrian Detection Models" (2604.22552) addresses the question of robust, transferable, and physically-realizable adversarial patch attacks that facilitate pedestrian evasion from modern detection pipelines. The authors propose TriPatch, a framework integrating multi-stage cooperative attacks and physical-world robustness enhancements, and evaluate its efficacy and reliability across diverse detection architectures and practical settings.

Threat Model and Challenges in Physical-World Attacks

Conventional adversarial attacks predominantly consider the digital space with unrestricted pixel-level manipulation and access to gradients, which is not realistic for physical-world deployment. For physical attacks, adversarial patches must endure variations in imaging—illumination, viewing angles, distances, camera resolutions, and printing artifacts—while remaining inconspicuous when applied to human bodies. Existing methods generally optimize a single submodule of the detection pipeline (e.g., classification head), which allows other modules such as regression or Non-Maximum Suppression (NMS) to compensate and diminish the attack effect. Furthermore, most fail to efficiently model and withstand physical diversity, leading to unstable real-world performance.

Figure 1: Illustration of adversarial examples applied to object detectors, highlighting their impact on detection outputs.

Methodology: TriPatch Pipeline and Triple-Loss Design

The TriPatch framework is composed of two main innovations: (1) a pipeline that supports dynamic patch placement guided by semantic bounding box information, and (2) a comprehensive loss formulation that simultaneously disrupts multiple stages of the detection process.

Adversarial Patch Generation: Given only black-box access to detector outputs, TriPatch overlays a printable patch onto images and optimizes its content such that the average confidence for the "person" class drops below typical detection thresholds, leveraging geometric transformations to simulate spatial variability.

Triple-Loss Mechanism: The core methodological contribution is a compound objective:

1. Detection Confidence Loss ($L_{\mathrm{det}}$): Suppresses confidence of high-scoring pedestrian predictions to target the classification output.
2. Bounding Box Loss ($L_{\mathrm{iou}}$): Penalizes confidence-weighted IoU between candidate and ground-truth boxes, ensuring spatial misalignment and attacking the regression head.
3. NMS Loss ($L_{\mathrm{nms}}$): Penalizes overlapping among top-confident predictions to force failures in the post-processing NMS stage and thereby decrease the likelihood of any candidate box surviving post-filtering.

An additional appearance regularization term enforces color and smoothness constraints for improved physical printability and inconspicuous integration, and comprehensive data augmentation simulates realistic imaging perturbations for robust physical deployment.

Figure 2: The TriPatch pipeline visualizes the multi-stage collaborative attack and appearance consistency workflow.

Experimental Results and Analysis

Digital-World Attack Effectiveness

Extensive experiments on INRIA and MS-COCO, spanning YOLO (v2–v9) and Faster RCNN, demonstrate that TriPatch reduces mAP to extremely low levels within the digital setting. Notably, fine-tuned attacks on YOLOv3tiny can degrade mAP to 0.40% on MS-COCO, while cross-architecture attacks consistently yield mAP below 45%.

Figure 3: Representative digital-world adversarial attack outcomes, exhibiting bounding box disappearance after TriPatch application.

Physical-World Performance and Robustness

Physical-world experiments involve printed patches worn by participants in varied indoor environments. TriPatch’s effectiveness is robust across distances (0.5–1.5 m) and angles (−30° to +30°), maintaining high attack success rate (ASR) under typical surveillance circumstances.

Figure 4: Demonstration of physical-world attacks where TriPatch successfully prevents pedestrian detection.

Figure 5: Empirical analysis of attack success under variable angles and distances, showing reliance of ASR on observation configurations.

Comparative Evaluations

Against baselines—AdvPatch, T-SEA, UPC, NAP, CAP—TriPatch consistently achieves superior or competitive attack strength; for instance, mAP drops to 0.89% on YOLOv2 and outperforms all baselines on nearly every detector.

Ablation and Sensitivity Studies

Comprehensive ablations confirm that each loss component is indispensable to the overall attack—removal of any term, or poorly calibrated loss weights, leads to significant degradation in attack effectiveness.

Figure 6: Ablation results on optimization epochs, patch size, loss components, and weights, validating the synergistic effect of TriPatch modules.

Furthermore, performance remains stable across different random initializations, attesting to the reproducibility and reliability of the TriPatch optimization process.

Figure 7: Consistency across random seeds, highlighting optimization robustness.

Discussion and Implications

This work empirically establishes that joint, multi-module perturbation is necessary to robustly defeat modern pedestrian detectors in both digital and physical domains. The triple-loss enables attacks that generalize across detection architectures and physical conditions, raising major security and safety concerns for real-world deployments in surveillance or autonomous driving. TriPatch's strong transferability, validated on both YOLO and Faster RCNN variants, evidences persistent vulnerabilities shared by contemporary detectors, regardless of architectural nuances.

On the practical front, the framework’s black-box threat model, augmented by aggressive physical augmentation and appearance regularization, aligns with the constraints encountered in real adversarial scenarios. The observed persistence of attack efficacy under varying distances, angles, and initialization conditions highlights that such attacks are feasible without privileged model access or environment control. Theoretically, the results invite reevaluation of detection model design and the need for robust, multi-stage defense strategies against coordinated, physically-realizable attacks.

Limitations and Future Prospects

While TriPatch delivers high attack success and transferability, limitations include some trade-off between stealth and attack strength. The adversarial patches, though subject to color and smoothness constraints, may still attract visual attention; future research could benefit from integrating naturalistic patterns or GAN-based camouflage for further inconspicuousness, as well as investigating possible model-side defensive countermeasures.

Conclusion

TriPatch sets a new technical benchmark for physically-realizable adversarial patch attacks against pedestrian detection. Its multi-stage, jointly optimized loss formulation decisively reduces detection accuracy in both digital and physical domains, confirming the urgent need for comprehensive, architecture-agnostic robustness interventions in public safety-critical detection systems. The demonstrated performance consistency, cross-architecture transferability, and physical-world reliability position TriPatch as a strong reference for both adversarial research and detector hardening efforts.