Universal adversarial perturbations (1610.08401v3)

Abstract: Given a state-of-the-art deep neural network classifier, we show the existence of a universal (image-agnostic) and very small perturbation vector that causes natural images to be misclassified with high probability. We propose a systematic algorithm for computing universal perturbations, and show that state-of-the-art deep neural networks are highly vulnerable to such perturbations, albeit being quasi-imperceptible to the human eye. We further empirically analyze these universal perturbations and show, in particular, that they generalize very well across neural networks. The surprising existence of universal perturbations reveals important geometric correlations among the high-dimensional decision boundary of classifiers. It further outlines potential security breaches with the existence of single directions in the input space that adversaries can possibly exploit to break a classifier on most natural images.

• The paper demonstrates that small, universal adversarial perturbations can fool over 90% of images in state-of-the-art DNNs, exposing critical vulnerabilities.
• It introduces an iterative algorithm that aggregates the minimal perturbation for each image while constraining the overall norm within a specified bound.
• Empirical and theoretical analyses reveal a low-dimensional subspace of decision boundaries, highlighting the need for more robust neural network defenses.

Universal Adversarial Perturbations

The paper “Universal adversarial perturbations” introduces the concept of universal perturbations in deep neural networks (DNNs) and demonstrates their efficacy and implications in the field of image classification. The authors, Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard, provide a systematic approach for generating perturbations that are quasi-imperceptible yet capable of causing high misclassification rates across a variety of natural images and several different state-of-the-art neural network architectures.

Overview

The primary contributions of the paper can be summarized as follows:

1. Existence Evidence: The authors establish the existence of small, universal (image-agnostic) perturbations that cause misclassifications in state-of-the-art DNNs with high probability.
2. Systematic Algorithm: They propose an iterative algorithm to compute such universal perturbations.
3. Empirical Analysis: The paper empirically demonstrates that these perturbations generalize well across different images and neural network architectures.
4. Geometric Insights: It provides insights into the geometric correlations in the decision boundaries of deep neural networks.

Algorithm and Methodology

The algorithm introduced for calculating universal perturbations involves iteratively modifying an initial perturbation vector. For a given set of data points, the algorithm aims to aggregate minimal perturbations that progressively move these points to the classifier's decision boundary. Specifically:

• Initial Setup: Start with an initial perturbation vector $v$ set to zero.
• Iterative Refinement: For each image in the training set, if the perturbed image $x_i + v$ does not change its classification, compute the least amount of perturbation needed to fool the classifier and add it to $v$.
• Projection: Ensure that the norm of the perturbation remains within a specified bound, $\xi$, by projecting the updated perturbation vector back onto the permissible $\ell_p$-norm ball.

Empirical Outcomes

The experiments run by the authors showcase the high vulnerability of leading neural network architectures, like CaffeNet, VGG, GoogLeNet, and ResNet, to universal perturbations. Specifically, these perturbations attain fooling rates exceeding 90% for some architectures when the perturbation norm is limited to relatively small values.

Key empirical findings include:

• High Fooling Rates: Universal perturbations were able to misclassify over 90% of images in tests run on CaffeNet and VGG-F for the $\ell_\infty$ norm.
• Generalization Across Architectures: Perturbations computed on one network architecture (e.g., VGG-19) maintained high fooling rates when applied to other architectures.

Theoretical Implications

The theoretical examination centered on understanding the correlation among different regions of a classifier's decision boundary. By analyzing the normal vectors to the decision boundaries at various data points, the authors highlighted a subspace of low dimension (relative to the input space), which contains most of these normal vectors. This low-dimensional subspace explains why universal perturbations, even though computed from a limited set of training images, generalize well across unseen images and different network architectures.

Practical Implications and Future Directions

The existence of universal adversarial perturbations has significant implications for the deployment of neural network-based classifiers in real-world applications, particularly in adversarial settings. It underscores potential security vulnerabilities where an adversary could easily craft perturbations to induce widespread misclassification in systems such as automated image recognition used in security or autonomous driving.

Future research could explore the geometric properties of decision boundaries, large-scale robustness evaluation across different domains, and developing more resilient architectures or training paradigms that can mitigate the effect of such universal perturbations.

By providing a structured method to generate universal adversarial perturbations and demonstrating their cross-architecture generalization, the paper contributes valuable insights into the robustness and vulnerabilities of deep learning models. The implications call for an enhanced focus on security and robustness in neural network research, particularly in adversarial environments.