On Detecting Adversarial Perturbations (1702.04267v2)

Abstract: Machine learning and deep learning in particular has advanced tremendously on perceptual tasks in recent years. However, it remains vulnerable against adversarial perturbations of the input that have been crafted specifically to fool the system while being quasi-imperceptible to a human. In this work, we propose to augment deep neural networks with a small "detector" subnetwork which is trained on the binary classification task of distinguishing genuine data from data containing adversarial perturbations. Our method is orthogonal to prior work on addressing adversarial perturbations, which has mostly focused on making the classification network itself more robust. We show empirically that adversarial perturbations can be detected surprisingly well even though they are quasi-imperceptible to humans. Moreover, while the detectors have been trained to detect only a specific adversary, they generalize to similar and weaker adversaries. In addition, we propose an adversarial attack that fools both the classifier and the detector and a novel training procedure for the detector that counteracts this attack.

• The paper proposes adding a binary detector subnetwork to DNNs to distinguish genuine data from adversarially perturbed inputs.
• It uses methods like Fast Gradient Sign, Basic Iterative, and DeepFool to create adversarial examples, achieving detectability above 80% on CIFAR10 and over 85% on ImageNet subsets.
• Dynamic adversary testing shows that training detectors with anticipation of gradient-based attacks can sustain robust performance, pointing to promising improvements in AI security.

On Detecting Adversarial Perturbations

The paper "On Detecting Adversarial Perturbations" by Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff, presents a methodology for detecting adversarial inputs in deep neural networks. Unlike conventional approaches that aim to make the classification model itself more robust against adversarial attacks, this research proposes augmenting neural networks with a specialized "detector" subnetwork. This detector is designed to distinguish between genuine data and data that has been adversarially perturbed.

Summary of Methods

The core proposition in this work is the addition of a small, binary classification subnet to existing deep neural networks. This detector network receives input from intermediate feature representations of the main classifier and is trained to recognize adversarially perturbed inputs.

Key methodologies and adversarial attack strategies employed in this paper include:

• Fast Gradient Sign Method: An efficient approach where the adversarial example is created by adding a perturbation in the direction of the gradient’s sign.
• Basic Iterative Method: An extension of the fast method with iterative perturbations and smaller step sizes.
• DeepFool Method: Iteratively linearizes the classification boundary and shifts the input image toward the nearest decision boundary, making it highly effective and sophisticated.

Furthermore, the dynamic adversary model was introduced to counteract the detectors. Here, adversarial examples aim to deceive both the classifier and the detector by modifying the cost function to consider the detector's output, thereby making the adversarial input appear innocuous.

Experimental Findings

The empirical analysis was conducted on diverse datasets including CIFAR10 and a 10-class subset of ImageNet. A Residual Network (ResNet) was employed for CIFAR10, and a VGG16 network was adapted for the ImageNet subset.

Results on CIFAR10

The paper demonstrated the detectability of adversarial examples even when perturbations were imperceptible to humans. The detector achieved an accuracy above 80% for adversarial examples that decreased classification accuracy to below 30%, indicating strong robust performance. Notably, the detector trained on adversaries with larger distortion values generalized well to those with smaller values, though not as effectively in the reverse scenario.

Results on ImageNet Subset

On higher-resolution images from the ImageNet subset, the detector again showed substantial performance, achieving detectability rates exceeding 85% in most cases. However, some adversaries such as the iterative $\ell_2$ method with moderate distortion values occasionally succeeded in reducing detection to chance level, revealing areas for further robustness improvement.

Dynamic Adversaries

Testing for dynamic adversaries revealed that static detectors could be manipulated effectively by adversaries with access to both the classifier and detector gradients. Yet, detectors trained dynamically, anticipating such adversaries, demonstrated marked improvements, maintaining detectability over 70% across various dynamic adversary settings.

Implications and Future Directions

This research underscores the potential of detector subnetworks in identifying adversarial inputs in deep neural networks, holding significance for applications in safety-critical systems like autonomous driving and biometric verification. The dual network model, pairing robust classifiers with adept detectors, augments the reliability of AI systems under adversarial conditions.

Future research directions include:

• Optimizing Detectors: Enhancing the architecture and training procedures of detectors to boost detectability against more sophisticated and randomized adversarial perturbations.
• Interpreting Detectors: Utilizing the gradients and learned parameters of detectors to gain deeper insights into the nature and characteristics of adversarial examples.
• Regularization Techniques: Employing the detector’s feedback during classifier training could serve as regularization, potentially hardening the classifier intrinsically against adversarial perturbations.

The paper demonstrates a compelling approach to bolstering machine learning models against adversarial attacks, laying a foundational methodology for future explorations in AI security and robustness.