A Fourier Perspective on Model Robustness in Computer Vision (1906.08988v3)

Abstract: Achieving robustness to distributional shift is a longstanding and challenging goal of computer vision. Data augmentation is a commonly used approach for improving robustness, however robustness gains are typically not uniform across corruption types. Indeed increasing performance in the presence of random noise is often met with reduced performance on other corruptions such as contrast change. Understanding when and why these sorts of trade-offs occur is a crucial step towards mitigating them. Towards this end, we investigate recently observed trade-offs caused by Gaussian data augmentation and adversarial training. We find that both methods improve robustness to corruptions that are concentrated in the high frequency domain while reducing robustness to corruptions that are concentrated in the low frequency domain. This suggests that one way to mitigate these trade-offs via data augmentation is to use a more diverse set of augmentations. Towards this end we observe that AutoAugment, a recently proposed data augmentation policy optimized for clean accuracy, achieves state-of-the-art robustness on the CIFAR-10-C benchmark.

• The paper presents a Fourier domain perturbation analysis to uncover how data augmentation biases models toward specific frequency ranges.
• It demonstrates that Gaussian and adversarial training favor low-frequency features, enhancing resistance to high-frequency noise but reducing resilience to low-frequency changes.
• The findings suggest that strategies like AutoAugment can balance frequency biases, pointing to new directions for developing universally robust vision models.

A Fourier Perspective on Model Robustness in Computer Vision

The paper "A Fourier Perspective on Model Robustness in Computer Vision" explores the longstanding issue of distributional shift in computer vision and examines the implications of data augmentation in improving model robustness. The authors utilize a Fourier analysis approach to uncover relationships between data augmentation techniques and their effects on model performance against various image corruptions.

Overview

The paper addresses the observation that common data augmentation methods, such as Gaussian data augmentation and adversarial training, do not uniformly enhance model robustness across all types of corruption. Typically, robustness to certain disturbances, like random noise, comes at the cost of reduced performance in others, such as contrast changes. The authors postulate that these trade-offs are influenced by the frequency characteristics of the corruptions and the biases introduced by the augmentation methods.

Methodology

The authors employ a perturbation analysis in the Fourier domain to paper model sensitivities to frequency-based perturbations. They analyze three broad categories of corruptions: high-frequency ones (e.g., noise and blur), low-frequency ones (e.g., contrast and fog), and various transformations induced by data augmentation strategies. By visualizing models' reaction to these perturbations through Fourier heat maps, the paper presents insights into how models leverage different frequency components.

Key Findings

• Bias Towards Low Frequency: Both Gaussian data augmentation and adversarial training bias models towards utilizing low-frequency information. This bias enhances robustness against high-frequency corruptions like noise and blur but degrades robustness against low-frequency corruptions.
• Effect of Diverse Augmentation: The AutoAugment strategy, which employs a diverse set of augmentations, significantly enhances model robustness across different corruption types by balancing the bias towards high and low frequencies.
• Adversarial Perturbations: Adversarial training alters the nature of adversarial perturbations, aligning them more closely with the frequency characteristics of natural images rather than concentrating in the high-frequency domain.

Implications and Future Work

The paper implies that achieving robust models may require a careful and systematic approach to data augmentation, perhaps involving learned augmentation policies like AutoAugment. The insights reveal that understanding the frequency domain properties of data augmentations and corruptions can guide the development of more universally robust models.

Moreover, the findings suggest a potential need for advancing architecture designs or loss functions that inherently promote robustness by encouraging models to focus on robust features. This approach could complement data augmentation efforts and possibly alleviate the inherent trade-offs observed with current strategies.

Future research could explore the extension of these methods to more complex real-world scenarios and other domains beyond visual perception. Additionally, revising robustness benchmarks will be crucial as methods evolve, ensuring comprehensive assessments that account for emerging trade-offs in model robustness.

Conclusion

This research provides a methodical examination of model robustness through a Fourier lens, offering compelling evidence that frequency properties play a critical role in how data augmentation impacts robustness across different corruption types. The paper's insights not only enhance the understanding of model vulnerabilities but could also steer future innovations in creating more invariant and reliable AI systems.