Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations (2204.02937v2)

Published 6 Apr 2022 in cs.LG, cs.CV, and stat.ML

Abstract: Neural network classifiers can largely rely on simple spurious features, such as backgrounds, to make predictions. However, even in these cases, we show that they still often learn core features associated with the desired attributes of the data, contrary to recent findings. Inspired by this insight, we demonstrate that simple last layer retraining can match or outperform state-of-the-art approaches on spurious correlation benchmarks, but with profoundly lower complexity and computational expenses. Moreover, we show that last layer retraining on large ImageNet-trained models can also significantly reduce reliance on background and texture information, improving robustness to covariate shift, after only minutes of training on a single GPU.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (129)
  1. Exploring the limits of large scale pre-training. arXiv preprint arXiv:2110.02095, 2021.
  2. A reductions approach to fair classification. In International Conference on Machine Learning, pp.  60–69. PMLR, 2018.
  3. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  4845–4854, 2019.
  4. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019.
  5. Linear unit-tests for invariance discovery. arXiv preprint arXiv:2102.10867, 2021.
  6. Learning de-biased representations with biased representations. In International Conference on Machine Learning, pp. 528–539. PMLR, 2020.
  7. Recognition in terra incognita. In Proceedings of the European conference on computer vision (ECCV), pp.  456–473, 2018.
  8. Robust solutions of optimization problems affected by uncertain probabilities. Management Science, 59(2):341–357, 2013.
  9. Generalizing from several related classification tasks to a new unlabeled sample. Advances in neural information processing systems, 24, 2011.
  10. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  11. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, pp.  491–500, 2019.
  12. Approximating cnns with bag-of-local-features models works surprisingly well on imagenet. arXiv preprint arXiv:1904.00760, 2019.
  13. Rubi: Reducing unimodal biases for visual question answering. Advances in neural information processing systems, 32, 2019.
  14. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems, 32, 2019.
  15. Heteroskedastic and imbalanced deep learning with adaptive regularization. arXiv preprint arXiv:2006.15766, 2020.
  16. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597–1607. PMLR, 2020.
  17. Environment inference for invariant learning. In International Conference on Machine Learning, pp. 2189–2200. PMLR, 2021.
  18. A too-good-to-be-true prior to reduce shortcut reliance. arXiv preprint arXiv:2102.06406, 2021.
  19. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  20. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  21. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp.  214–226, 2012.
  22. Jacob Eisenstein. Informativeness and invariance: Two perspectives on spurious correlations in natural language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.  4326–4331, 2022.
  23. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 1180–1189. PMLR, 2015.
  24. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096–2030, 2016.
  25. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231, 2018.
  26. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020.
  27. Partial success in closing the gap between human and machine vision. In Advances in Neural Information Processing Systems 34, 2021.
  28. Jacob Gildenblat and contributors. Pytorch library for cam methods. https://github.com/jacobgil/pytorch-grad-cam, 2021.
  29. Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pp.  1440–1448, 2015.
  30. In search of lost domain generalization. arXiv preprint arXiv:2007.01434, 2020.
  31. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
  32. Array programming with NumPy. Nature, 585(7825):357–362, September 2020. doi: 10.1038/s41586-020-2649-2. URL https://doi.org/10.1038/s41586-020-2649-2.
  33. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  34. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp.  2961–2969, 2017.
  35. Masked autoencoders are scalable vision learners. arXiv preprint arXiv:2111.06377, 2021.
  36. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019.
  37. The many faces of robustness: A critical analysis of out-of-distribution generalization. ICCV, 2021.
  38. What shapes feature representations? exploring datasets, architectures, and training. Advances in Neural Information Processing Systems, 33:9995–10006, 2020.
  39. The origins and prevalence of texture bias in convolutional neural networks. Advances in Neural Information Processing Systems, 33:19000–19015, 2020.
  40. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning, pp. 2029–2037. PMLR, 2018.
  41. What makes imagenet good for transfer learning? arXiv preprint arXiv:1608.08614, 2016.
  42. J. D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9(3):90–95, 2007. doi: 10.1109/MCSE.2007.55.
  43. Simple data balancing achieves competitive worst-group-accuracy. arXiv preprint arXiv:2110.14503, 2021.
  44. Shape or texture: Understanding discriminative features in cnns. arXiv preprint arXiv:2101.11604, 2021.
  45. On feature learning in the presence of spurious correlations. Advances in Neural Information Processing Systems, 35:38516–38532, 2022.
  46. Excessive invariance causes adversarial vulnerability. arXiv preprint arXiv:1811.00401, 2018.
  47. Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1910.09217, 2019.
  48. Learning the difference that makes a difference with counterfactually-augmented data. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Sklgs0NFvr.
  49. Explaining the efficacy of counterfactually augmented data. International Conference on Learning Representations (ICLR), 2021.
  50. Maximum weighted loss discrepancy. arXiv preprint arXiv:1906.03518, 2019.
  51. Learning not to learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  9012–9020, 2019.
  52. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807, 2016.
  53. Jupyter notebooks – a publishing format for reproducible computational workflows. In F. Loizides and B. Schmidt (eds.), Positioning and Power in Academic Publishing: Players, Agents and Agendas, pp.  87 – 90. IOS Press, 2016.
  54. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pp. 5637–5664. PMLR, 2021.
  55. Improving weakly-supervised object localization by micro-annotation. arXiv preprint arXiv:1605.05538, 2016.
  56. Big transfer (bit): General visual representation learning. In European conference on computer vision, pp.  491–507. Springer, 2020.
  57. Do better imagenet models transfer better? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  2661–2671, 2019.
  58. Learning multiple layers of features from tiny images. 2009.
  59. Out-of-distribution generalization via risk extrapolation (rex). In International Conference on Machine Learning, pp. 5815–5826. PMLR, 2021.
  60. Fine-tuning can distort pretrained features and underperform out-of-distribution. arXiv preprint arXiv:2202.10054, 2022.
  61. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  62. Surgical fine-tuning improves adaptation to distribution shifts. arXiv preprint arXiv:2210.11466, 2022a.
  63. Diversify and disambiguate: Learning from underspecified data. arXiv preprint arXiv:2202.03418, 2022b.
  64. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV), pp.  624–639, 2018a.
  65. Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  9572–9581, 2019.
  66. Resound: Towards action recognition without representation bias. In Proceedings of the European Conference on Computer Vision (ECCV), pp.  513–528, 2018b.
  67. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781–6792. PMLR, 2021.
  68. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp.  3730–3738, 2015.
  69. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  70. Predicting inductive biases of pre-trained models. In International Conference on Learning Representations, 2020.
  71. Fast adaptation with linearized neural networks. In International Conference on Artificial Intelligence and Statistics, pp.  2737–2745. PMLR, 2021.
  72. Exploring the limits of weakly supervised pretraining. In Proceedings of the European conference on computer vision (ECCV), pp.  181–196, 2018.
  73. Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM international conference on Multimedia, pp.  1485–1488, 2010.
  74. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. arXiv preprint arXiv:1902.01007, 2019.
  75. Wes McKinney. Data structures for statistical computing in python. In Stéfan van der Walt and Jarrod Millman (eds.), Proceedings of the 9th Python in Science Conference, pp.  51 – 56, 2010.
  76. Overparameterisation and worst-case generalisation: friend or foe? In International Conference on Learning Representations, 2020.
  77. A comprehensive study of image classification model sensitivity to foregrounds, backgrounds, and visual attributes. arXiv preprint arXiv:2201.10766, 2022.
  78. Domain generalization via invariant feature representation. In International Conference on Machine Learning, pp.  10–18. PMLR, 2013.
  79. Understanding the failure modes of out-of-distribution generalization. arXiv preprint arXiv:2010.15775, 2020.
  80. Learning from failure: De-biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33:20673–20684, 2020.
  81. Spread spurious attribute: Improving worst-group accuracy with spurious attribute estimation. In International Conference on Learning Representations, 2022.
  82. What is being transferred in transfer learning? Advances in neural information processing systems, 33:512–523, 2020.
  83. Distributionally robust language modeling. arXiv preprint arXiv:1909.02060, 2019.
  84. Agree to disagree: Diversity through disagreement for better transferability. arXiv preprint arXiv:2202.04414, 2022.
  85. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2009.
  86. Automatic differentiation in pytorch. 2017.
  87. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  88. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78(5):947–1012, 2016.
  89. Gradient starvation: A learning proclivity in neural networks. Advances in Neural Information Processing Systems, 34, 2021.
  90. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
  91. Simple and fast group robustness by automatic feature reweighting. arXiv preprint arXiv:2306.11074, 2023.
  92. On the spectral bias of neural networks. In International Conference on Machine Learning, pp. 5301–5310. PMLR, 2019.
  93. Learning to reweight examples for robust deep learning. In International conference on machine learning, pp. 4334–4343. PMLR, 2018.
  94. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp.  1135–1144, 2016.
  95. The elephant in the room. arXiv preprint arXiv:1808.03305, 2018.
  96. Domain-adjusted regression or: Erm may already learn features sufficient for out-of-distribution generalization. arXiv preprint arXiv:2202.06856, 2022.
  97. Optimal representations for covariate shift. arXiv preprint arXiv:2201.00057, 2021.
  98. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019.
  99. Extending the wilds benchmark for unsupervised adaptation. arXiv preprint arXiv:2112.05090, 2021.
  100. Which shortcut cues will dnns choose? a study from the parameter-space perspective. arXiv preprint arXiv:2110.03095, 2021.
  101. The pitfalls of simplicity bias in neural networks. Advances in Neural Information Processing Systems, 33:9573–9585, 2020.
  102. Cnn features off-the-shelf: an astounding baseline for recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp.  806–813, 2014.
  103. Not using the car to see the sidewalk–quantifying and controlling the effects of context in classification and segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  8218–8226, 2019.
  104. Salient imagenet: How to discover spurious features in deep learning? arXiv preprint arXiv:2110.04301, 2021.
  105. Understanding failures of deep networks via robust feature extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  12853–12862, 2021.
  106. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Advances in Neural Information Processing Systems, 33:19339–19352, 2020.
  107. Barack: Partially supervised group robustness with guarantees. arXiv preprint arXiv:2201.00072, 2021.
  108. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE international conference on computer vision, pp.  843–852, 2017.
  109. End: Entangling and disentangling deep representations for bias correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  13508–13517, 2021.
  110. Evading the simplicity bias: Training a diverse set of models discovers solutions with superior ood generalization. arXiv preprint arXiv:2105.05612, 2021a.
  111. Unshuffling data for improved generalization in visual question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  1417–1427, 2021b.
  112. Towards debiasing nlu models from unknown biases. arXiv preprint arXiv:2009.12303, 2020.
  113. Counterfactual invariance to spurious correlations: Why and how to pass stress tests. arXiv preprint arXiv:2106.00545, 2021.
  114. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261–272, 2020. doi: 10.1038/s41592-019-0686-2.
  115. The caltech-ucsd birds-200-2011 dataset. 2011.
  116. Learning robust representations by projecting superficial statistics out. arXiv preprint arXiv:1903.06256, 2019.
  117. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019.
  118. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
  119. Noise or signal: The role of image backgrounds in object recognition. arXiv preprint arXiv:2006.09994, 2020.
  120. Controlling directions orthogonal to a classifier. In International Conference on Learning Representations, 2022.
  121. Increasing robustness to spurious correlations using forgettable examples. arXiv preprint arXiv:1911.03861, 2019.
  122. Change is hard: A closer look at subpopulation shift. arXiv preprint arXiv:2302.12254, 2023.
  123. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS medicine, 15(11):e1002683, 2018.
  124. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint arXiv:1910.04867, 2019.
  125. Local features and kernels for classification of texture and object categories: A comprehensive study. International journal of computer vision, 73(2):213–238, 2007.
  126. Coping with label shift via distributionally robust optimisation. arXiv preprint arXiv:2010.12230, 2020.
  127. Correct-n-contrast: A contrastive approach for improving robustness to spurious correlations. arXiv preprint arXiv:2203.01517, 2022.
  128. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2017.
  129. Learning bias-invariant representation by cross-sample mutual information minimization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  15002–15012, 2021.
Citations (278)
View on Semantic Scholar

Summary

  • The paper demonstrates that retraining only the last layer (DFR) effectively counters spurious correlations in neural networks.
  • It employs a small, curated reweighting dataset to recalibrate weights for core features without full model retraining.
  • Experiments reveal that DFR achieves state-of-the-art performance on benchmarks like Waterbirds and CelebA with minimal computational cost.

An Expert Overview of "Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations"

The paper addresses a crucial issue in neural network classifiers: the reliance on spurious features for making predictions. It provides an insightful analysis into how neural networks, even when they seem to prioritize spurious correlations, are capable of learning core features reflective of the semantic structure of data. The authors leverage this understanding to propose a method named Deep Feature Reweighting (DFR), which essentially involves retraining just the last layer of an existing model to improve robustness against spurious correlations.

Core Observations and Methodology

The foundational observation is that neural networks, despite being affected by spurious correlations, retain a significant amount of information about the core features relevant to the actual class labels. This is a departure from recent findings suggesting that networks prioritize simpler, spurious features over more complex but relevant features. The authors substantiate their findings through experiments on synthetic datasets where clear distinctions between spurious and core features can be made.

To capitalize on this insight, the authors propose DFR, which involves retraining the last linear classification layer of a pre-existing neural model using a relatively small dataset. This dataset, referred to as the reweighting dataset, is carefully curated to break the spurious correlations present in the main training data. The retraining process considers the weights assigned to the core features and adjusts them to mitigate the influence of spurious features. The simplicity of this method is its key strength, avoiding the complexity of redesigning the entire network architecture or retraining from scratch.

Numerical Results and Claims

The empirical results presented in the paper are compelling. DFR manages to match or even surpass state-of-the-art performances on benchmarks involved with spurious correlations, while demanding significantly less computational resources. For instance, it achieves noteworthy improvements on datasets like Waterbirds and CelebA, enhancing worst-group accuracy without compromising mean accuracy. The method shows state-of-the-art results in various spurious correlation benchmarks, indicating its robustness and effectiveness.

One of the more striking results of the paper is achieved by applying DFR to large models trained on ImageNet. The method effectively reduces the model’s dependency on background and texture information, yielding improved robustness to covariate shift. This is particularly demonstrated with major classification benchmarks, where retraining the last layer effectively curbed the model's background reliance in mere minutes on a single GPU.

Theoretical and Practical Implications

Theoretically, this paper challenges some established notions about neural network induction biases, particularly the focus solely on simple, spurious features. The findings imply that even when spurious features seem predominant, the network's latent space contains a rich representation of relevant data features sufficient for robust classification.

Practically, the paper's approach is profound in its simplicity and efficiency. By focusing training efforts on just the last layer, practitioners can achieve high robustness with minimal computational demand. This can democratize access to effective deep learning models, allowing smaller teams with limited resources to leverage powerful models effectively in the presence of spurious correlations.

Future Directions

The implications of this research suggest several interesting avenues for future work. First, exploring how DFR can be optimized for other forms of neural architectures could expand its applicability further. Second, the method's potential integration with transfer learning paradigms and its role in enhancing fairness by reducing model biases present fertile grounds for further exploration. Lastly, understanding the limits of this method—particularly in scenarios where spurious correlations are exceedingly strong—would benefit both theoretical research and practical application.

In summary, this paper offers a valuable contribution to the field by demonstrating how minimal retraining of deep models can yield significant improvements in robustness against spurious correlations. DFR stands as a testament to the effectiveness of innovative and resource-efficient solutions in enhancing the existing capabilities of neural networks.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown