On the Robustness of Semantic Segmentation Models to Adversarial Attacks (1711.09856v3)

Abstract: Deep Neural Networks (DNNs) have demonstrated exceptional performance on most recognition tasks such as image classification and segmentation. However, they have also been shown to be vulnerable to adversarial examples. This phenomenon has recently attracted a lot of attention but it has not been extensively studied on multiple, large-scale datasets and structured prediction tasks such as semantic segmentation which often require more specialised networks with additional components such as CRFs, dilated convolutions, skip-connections and multiscale processing. In this paper, we present what to our knowledge is the first rigorous evaluation of adversarial attacks on modern semantic segmentation models, using two large-scale datasets. We analyse the effect of different network architectures, model capacity and multiscale processing, and show that many observations made on the task of classification do not always transfer to this more complex task. Furthermore, we show how mean-field inference in deep structured models, multiscale processing (and more generally, input transformations) naturally implement recently proposed adversarial defenses. Our observations will aid future efforts in understanding and defending against adversarial examples. Moreover, in the shorter term, we show how to effectively benchmark robustness and show which segmentation models should currently be preferred in safety-critical applications due to their inherent robustness.

On the Robustness of Semantic Segmentation Models to Adversarial Attacks

The paper "On the Robustness of Semantic Segmentation Models to Adversarial Attacks," authored by Anurag Arnab, Ondrej Miksik, and Philip H.S. Torr, conducts a comprehensive evaluation of the vulnerability of semantic segmentation models, a crucial component in computer vision, against adversarial attacks. The research predominantly scrutinizes the susceptibility of these models to perturbations that are quasi-imperceptible to human observers but can significantly disrupt model performance.

The authors begin by delineating the threat model used in the paper, considering both white-box and black-box adversarial settings. It is asserted that the semantic segmentation task, which involves classifying each pixel of an image into predefined categories, could encounter unique vulnerabilities due to the oftentimes ambiguous and intricate nature of images.

Methodologically, the paper employs established attack strategies, such as the Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD), while adapting them to suit the pixel-wise output needs of semantic segmentation. A noteworthy contribution is the investigation into the interplay between adversarial examples and model architectures, including fully convolutional networks and more modern architectures like SegNet and DeepLab.

The empirical results are suggestive of several key insights:

1. Vulnerability Discrepancies: Not all architectures respond similarly to adversarial perturbations. DeepLab models exhibit a notable robustness margin compared to more traditional fully convolutional networks (FCNs).
2. Transferability: Perturbations demonstrate a degree of transferability across different models. However, transferability is asymmetric, with attacks designed for a stronger model sometimes more effectively disrupting weaker models than vice-versa.
3. Trade-offs Inherent to Robustness: The paper evidences that enhancing model robustness through implicit regularization or explicit architectural modifications, such as incorporating adversarial training, often leads to a trade-off in nominal classification performance.

The theoretical implications of these findings extend to a reevaluation of model evaluation protocols within the field of semantic segmentation. Particularly, the work emphasizes the necessity for future research to encompass adversarial resilience as a core parameter in model assessment and for datasets to be augmented with adversarially perturbed examples for more comprehensive robustness validation.

Practically, this paper suggests avenues for fortifying segmentation models against adversarial threats, especially in high-stakes applications such as autonomous driving and medical imaging, where the cost of misclassification can be substantial. While the paper refrains from positing definitive solutions to these vulnerabilities, it provides foundational insights that could inform future augmented learning techniques and hybrid model designs.

Looking forward, developments in defensive mechanisms and theoretical frameworks to better understand the dynamics of adversarial perturbations in semantic segmentation are anticipated. Furthermore, as models grow increasingly complex, probing their limits of robustness and adapting adversarial strategies to these advancements will remain a critical challenge for the research community. This discourse represents a pivotal dialogue for developing AI systems that are not solely performant but resilient under duress.