Adversarial Clothes: Physical-World Attacks

• Adversarial clothes are physical garments embedded with optimized patterns designed to manipulate deep neural network outputs under real-world distortions.
• They execute evasion attacks by lowering rank-1 accuracy and impersonation attacks by mimicking target identities, disrupting re-ID system reliability.
• Empirical evaluations demonstrate significant performance drops, highlighting the urgent need for robust defenses in surveillance and security applications.

Adversarial clothes are physical garments designed with carefully optimized patterns to manipulate the outputs of vision-based machine learning systems—most notably person re-identification (re-ID), object detection, and segmentation models—in the real world. By integrating “adversarial patterns” onto clothing, such attacks can cause deep neural networks to misidentify, fail to detect, or even attribute an incorrect identity to a person, posing significant challenges for security, surveillance, and authentication systems.

1. Core Principles and Definitions

Adversarial clothes denote garments that embed patterns—realized through printing, digital transfer, or hybrid textile techniques—whose parameters are determined through optimization algorithms targeting the internal feature space of victim neural networks. Unlike digital adversarial examples, adversarial clothes are engineered to maintain effectiveness in the physical world where environmental variables, non-rigid deformations, and transformations are unavoidable. As described in (Wang et al., 2019), these garments can be used to mount two primary attack scenarios on deep re-ID systems:

• Evading Attack: The adversary seeks to minimize the similarity between their own images across different camera views, thwarting cross-camera person association.
• Impersonation Attack: The adversary attempts to maximize the similarity between their own images (with the adversarial garment) and those of a target person, causing the system to misassociate identities.

Adversarial clothing thus functions as a “physical-world adversarial example,” challenging the robustness of DNN-based recognition and tracking systems by introducing misclassifications via the input channel—real, visible apparel.

2. Algorithmic Design and Optimization Strategies

The construction of adversarial clothes is formalized as a constrained optimization problem that seeks to perturb the model’s feature space using a digitally generated adversarial pattern, later mapped to a printable, wearable surface.

• Cross-View Transformations: Because surveillance systems operate across disparate viewpoints, adversarial patterns must be robust to camera angle, distance, and dynamic variation. Each pattern δ is mapped to the image through transformations $T_i(\delta)$, which model projective and perspective changes for each view.
• Optimization Objectives:
  • Evading Attack:

  $$\min_\delta \sum_{i=1}^m \sum_{j=1, j \neq i}^m f_\theta(x'_i, x'_j)$$

  where $x'_i = o(x_i, T_i(\delta))$ is the overlay of the transformed pattern onto the original image and $f_\theta(\cdot, \cdot)$ computes feature similarity. - Impersonation Attack:

  $$\min_\delta \sum_{i,j, i \neq j} f_\theta(x'_i, x'_j) - \alpha [f_\theta(x'_i, I_t) + f_\theta(x'_j, I_t)]$$

  with $I_t$ as the feature of the target person and $\alpha$ balancing intra-/inter-class similarity.

• Physical Realizability Constraints: Optimization includes total variation (TV) penalty to promote smoothness and printability:

$$TV(\delta) = \sum_{p,(q)} \sqrt{(\delta_{p,q} - \delta_{p+1,q})^2 + (\delta_{p,q} - \delta_{p,q+1})^2}$$

Patterns are further projected through decorative masks, with values clipped to printable color ranges, ensuring physical plausibility.

• Position and Pose Augmentation: To model real-world distortions—scaling, translation, rotation, environmental illumination—multi-position sampling is used in training, analogously to data augmentation, improving pattern invariance.

3. Empirical Evaluation and Results

Evaluations are conducted on large-scale public (Market1501) and proprietary (PRCS) datasets, with experiments spanning both digital (direct overlay) and physical (printed pattern worn on clothing) settings. Two deep re-ID model architectures (siamese and classifier-based) are tested.

• Evading Attack Efficacy: The adversarial pattern drops the rank-1 accuracy (proportion of correct matches at top-1 ranking) from 87.9% to 27.1% in physical-world trials on the PRCS dataset, unequivocally reducing self-matching across views.
• Impersonation Attack Efficacy: By optimizing the pattern to also maximize similarity to a target, the attacker achieves a rank-1 accuracy of 47.1% and mAP of 67.9% for impersonating the target—substantially above random guess rates and typical impostor chances.
• Robustness Across Physical Conditions: Tests include varied distances, incident angles, weather, and camera placements. The pattern retains effectiveness except at extreme oblique angles or distances where the textile’s visual contribution is diminished.

Standard evaluation metrics include rank-k accuracy, mean average precision (mAP), and raw similarity score (ss). The results establish that the adversarial pattern, when realized via clothing, can cause strong performance degradation across metrics.

4. Architectural Extensions and Physical Constraints

To enhance attack scalability and effectiveness in physical scenarios, several architectural strategies are employed:

• Augmented Triplet Sampling: Optimization is extended with augmented triplets ($tri_k = \langle x_k^o, x_k^+, x_k^-\rangle$) to balance intra- and inter-camera similarity under varying transform conditions:

$$\min_\delta \mathbb{E}_{ tri_k \sim X^C } [ f_\theta((x_k^o)', (x_k^-)') - \beta f_\theta((x_k^o)', (x_k^+)') ]$$

with $\beta$ modulating intra-camera similarity penalty.

• Regularization for Realization: To ensure not only adversarial potency but also printability and inconspicuousness, TV regularization is crucial for preventing unnatural, high-frequency artifacts, while color quantization addresses in-fabric limitations of real-world dye.
• Physical Simulation During Training: The overlay operation is augmented by a degradation function $\phi(\cdot)$ simulating brightness, blur, and environmental noise.
• Safety Mask Construction: Pattern application is masked within boundaries (decorative motifs, "logos") to camouflage the adversarial component as plausible clothing design.

5. Security Implications and Broader Significance

Adversarial clothes highlight systemic weaknesses in computer vision systems deployed for person identification and surveillance:

• Real-World Threat Model: Attackers do not require digital access to the sensor or data pipeline; they merely need to present themselves in adversarial apparel, exploiting the model’s vulnerability to crafted visual inputs.
• Physical-World Generalization: Unlike digital attacks, adversarial clothes reveal the fragility of deep models to real-world modifications—extending the adversarial example paradigm from image-space perturbations to embodied, physical adversaries.
• Implications for Defense: The demonstrated impact on both evasion and impersonation motivates research into robust physical-world defenses—potentially incorporating transformation-invariant feature extraction, adversarial training with physically realized attack samples, and detection of anomalous pattern geometry or invariance.

6. Limitations and Research Directions

While the advPattern approach establishes the feasibility and effectiveness of adversarial clothes for deep re-ID systems, several limitations and open problems remain:

• Generalization to Black-box Models: The bulk of the results focus on white-box attacks, with transferability to unknown or proprietary model architectures left as a future direction.
• Balancing Stealth and Potency: There is an inherent tension between effectiveness (increased adversarial signal strength) and visual plausibility (avoiding human suspicion). Generating patterns that are simultaneously inconspicuous and strongly adversarial is an ongoing challenge.
• Physical Degradation Effects: Wear, laundering, and textile resilience may introduce divergence from the optimized pattern, potentially degrading adversarial efficacy over time.
• Countermeasures and Adversarial Training: There is a need to investigate detection and defense strategies specifically tailored to physical adversarial patterns, encompassing both classical adversarial training and model-based detection of synthetic decorative elements.

7. Summary Table: Key Metrics and Outcomes

Attack Type	Dataset	Metric	Clean Accuracy	With Adversarial Clothes
Evading Attack	PRCS	Rank-1	87.9%	27.1%
Impersonation Attack	PRCS	Rank-1	negligible	47.1%
Impersonation Attack	PRCS	mAP	–	67.9%

This empirical evidence demonstrates that adversarial clothes—using transformable, physically realizable patterns—can subvert state-of-the-art deep person re-identification systems in both digital and real environments, prompting reconsideration of the reliability and security of computer vision in security-critical applications.