Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Simplicity Bias of Transformers to Learn Low Sensitivity Functions (2403.06925v1)

Published 11 Mar 2024 in cs.LG, cs.AI, cs.CL, and stat.ML

Abstract: Transformers achieve state-of-the-art accuracy and robustness across many tasks, but an understanding of the inductive biases that they have and how those biases are different from other neural network architectures remains elusive. Various neural network architectures such as fully connected networks have been found to have a simplicity bias towards simple functions of the data; one version of this simplicity bias is a spectral bias to learn simple functions in the Fourier space. In this work, we identify the notion of sensitivity of the model to random changes in the input as a notion of simplicity bias which provides a unified metric to explain the simplicity and spectral bias of transformers across different data modalities. We show that transformers have lower sensitivity than alternative architectures, such as LSTMs, MLPs and CNNs, across both vision and language tasks. We also show that low-sensitivity bias correlates with improved robustness; furthermore, it can also be used as an efficient intervention to further improve the robustness of transformers.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (114)
  1. What learning algorithm is in-context learning? investigations with linear models. In The Eleventh International Conference on Learning Representations.
  2. A modern look at the relationship between sharpness and generalization. In International Conference on Machine Learning.
  3. Implicit regularization in deep matrix factorization. In Advances in Neural Information Processing Systems, volume 32.
  4. A closer look at memorization in deep networks. In International conference on machine learning, pages 233–242. PMLR.
  5. Layer normalization.
  6. Transformers as statisticians: Provable in-context learning with in-context algorithm selection. arXiv preprint arXiv:2306.04637.
  7. Better plain vit baselines for imagenet-1k.
  8. Understanding in-context learning in transformers and llms by learning to learn discrete functions.
  9. Simplicity bias in transformers and their ability to learn sparse Boolean functions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.
  10. Understanding robustness of transformers for image classification. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10231–10241.
  11. Bishop, C. M. (1995). Training with noise is equivalent to tikhonov regularization. Neural Computation, 7:108–116.
  12. Implicit regularization for deep neural networks driven by an ornstein-uhlenbeck like process. In Conference on learning theory, pages 483–513. PMLR.
  13. Towards understanding the word sensitivity of attention layers: A study via random features.
  14. Language models are few-shot learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
  15. Towards understanding the spectral bias of deep learning. In IJCAI.
  16. Chiang, T.-R. (2021). On a benefit of mask language modeling: Robustness to simplicity bias.
  17. Implicit bias of gradient descent for wide two-layer neural networks trained with the logistic loss. In Conference on Learning Theory, pages 1305–1338. PMLR.
  18. Towards automated circuit discovery for mechanistic interpretability. In Neural Information Processing Systems.
  19. Autoaugment: Learning augmentation strategies from data. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 113–123.
  20. Gaussian process behaviour in wide deep neural networks.
  21. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  22. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005).
  23. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations.
  24. Implicit bias in leaky relu networks trained on high-dimensional data. arXiv preprint arXiv:2210.07082.
  25. Transformers learn higher-order optimization methods for in-context learning: A study with linear models. arXiv, abs/2310.17086.
  26. What can transformers learn in-context? a case study of simple function classes. ArXiv, abs/2208.01066.
  27. Deep convolutional networks as shallow gaussian processes. In International Conference on Learning Representations.
  28. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673.
  29. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations.
  30. Are vision transformers robust to spurious correlations?
  31. Annotation artifacts in natural language inference data. In Walker, M., Ji, H., and Stent, A., editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 107–112, New Orleans, Louisiana. Association for Computational Linguistics.
  32. How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. In Thirty-seventh Conference on Neural Information Processing Systems.
  33. Shape matters: Understanding the implicit bias of the noise covariance. In Conference on Learning Theory, pages 2315–2357. PMLR.
  34. How much does attention actually attend? questioning the importance of attention in pretrained transformers.
  35. Variations on the sensitivity conjecture. Theory of Computing, pages 1–27.
  36. Deep Residual Learning for Image Recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’16, pages 770–778. IEEE.
  37. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations.
  38. Long Short-Term Memory. Neural Computation, 9(8):1735–1780.
  39. Infinite attention: Nngp and ntk for deep attention networks. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org.
  40. Densely connected convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2261–2269.
  41. Huang, H. (2019). Induced subgraphs of hypercubes and a proof of the sensitivity conjecture. Annals of Mathematics, 190(3):949–955.
  42. The low-rank simplicity bias in deep networks. Trans. Mach. Learn. Res., 2023.
  43. First Quora Dataset Release: Question Pairs. Online.
  44. What does BERT learn about the structure of language? In Korhonen, A., Traum, D., and Màrquez, L., editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3651–3657, Florence, Italy. Association for Computational Linguistics.
  45. Fast margin maximization via dual acceleration. In International Conference on Machine Learning, pages 4860–4869. PMLR.
  46. Risk and parameter convergence of logistic regression. arXiv preprint arXiv:1803.07300.
  47. Directional convergence and alignment in deep learning. Advances in Neural Information Processing Systems, 33:17176–17186.
  48. Characterizing the implicit bias via a primal-dual analysis. In Algorithmic Learning Theory, pages 772–804. PMLR.
  49. Fantastic generalization measures and where to find them. In International Conference on Learning Representations.
  50. Highly accurate protein structure prediction with alphafold. Nature, 596:583 – 589.
  51. On large-batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations.
  52. Last layer re-training is sufficient for robustness to spurious correlations. ArXiv, abs/2204.02937.
  53. Implicit bias of gradient descent for two-layer relu and leaky relu networks on nearly-orthogonal data.
  54. Revealing the dark secrets of BERT. In Inui, K., Jiang, J., Ng, V., and Wan, X., editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4365–4374, Hong Kong, China. Association for Computational Linguistics.
  55. Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. pages 32–33.
  56. Handwritten digit recognition with a back-propagation network. In Touretzky, D., editor, Advances in Neural Information Processing Systems, volume 2. Morgan-Kaufmann.
  57. The mnist database of handwritten digits.
  58. Deep neural networks as gaussian processes. In International Conference on Learning Representations.
  59. Vision transformer for small-size datasets.
  60. Towards resolving the implicit bias of gradient descent for matrix factorization: Greedy low-rank learning.
  61. Implicit bias of gradient descent on reparametrized models: On equivalence to mirror descent. Advances in Neural Information Processing Systems, 35:34626–34640.
  62. Fast autoaugment. In Neural Information Processing Systems.
  63. Roberta: A robustly optimized bert pretraining approach.
  64. Gradient descent maximizes the margin of homogeneous neural networks. In International Conference on Learning Representations.
  65. Gradient descent on two-layer nets: Margin maximization and simplicity bias. In Neural Information Processing Systems.
  66. On the robustness of vision transformers to adversarial examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7838–7847.
  67. Melas-Kyriazi, L. (2021). Do you even need attention? a stack of feed-forward layers does surprisingly well on imagenet. ArXiv, abs/2105.02723.
  68. Locating and editing factual associations in GPT. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K., editors, Advances in Neural Information Processing Systems.
  69. Simplicity bias in 1-hidden layer neural networks.
  70. The transformative power of transformers in protein structure prediction. Proceedings of the National Academy of Sciences, 120(32):e2303499120.
  71. Convergence of gradient descent on separable data. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3420–3428. PMLR.
  72. Understanding the failure modes of out-of-distribution generalization. In International Conference on Learning Representations.
  73. SGD on Neural Networks Learns Functions of Increasing Complexity. Curran Associates Inc., Red Hook, NY, USA.
  74. Progress measures for grokking via mechanistic interpretability.
  75. Intriguing properties of vision transformers. In Neural Information Processing Systems.
  76. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011.
  77. Exploring generalization in deep learning. In Neural Information Processing Systems.
  78. In search of the real inductive bias: On the role of implicit regularization in deep learning. CoRR, abs/1412.6614.
  79. Sensitivity and generalization in neural networks: an empirical study. arXiv preprint arXiv:1802.08760.
  80. Bayesian deep convolutional networks with many channels are gaussian processes. In International Conference on Learning Representations.
  81. O’Donnell, R. (2014). Analysis of Boolean Functions. Cambridge University Press.
  82. OpenAI (2023). GPT-4 technical report.
  83. Vision transformers are robust learners. In Proceedings of the AAAI conference on Artificial Intelligence, volume 36, pages 2071–2081.
  84. Gradient starvation: A learning proclivity in neural networks. In Neural Information Processing Systems.
  85. The inductive bias of re{lu} networks on orthogonally separable data. In International Conference on Learning Representations.
  86. Grokking: Generalization beyond overfitting on small algorithmic datasets.
  87. Do vision transformers see like convolutional neural networks? In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W., editors, Advances in Neural Information Processing Systems.
  88. On the spectral bias of neural networks. In Chaudhuri, K. and Salakhutdinov, R., editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5301–5310. PMLR.
  89. Data augmentation can improve robustness. In Neural Information Processing Systems.
  90. Robbins, H. E. (1951). A stochastic approximation method. Annals of Mathematical Statistics, 22:400–407.
  91. Distributionally robust neural networks. In International Conference on Learning Representations.
  92. The pitfalls of simplicity bias in neural networks. Advances in Neural Information Processing Systems, 33.
  93. On the adversarial robustness of visual transformers. arXiv preprint arXiv:2103.15670, 1(2).
  94. Improving the robustness of transformer-based large language models with dynamic attention. arXiv preprint arXiv:2311.17400.
  95. Fast certified robust training with short warmup. Advances in Neural Information Processing Systems, 34:18335–18349.
  96. Robustness verification for transformers. arXiv preprint arXiv:2002.06622.
  97. The implicit bias of gradient descent on separable data. The Journal of Machine Learning Research, 19(1):2822–2878.
  98. Transformers as support vector machines. ArXiv, abs/2308.16898.
  99. Max-margin token selection in attention mechanism.
  100. Overcoming simplicity bias in deep networks using a feature sieve.
  101. Patches are all you need? Transactions on Machine Learning Research, 2023.
  102. Deep learning generalizes because the parameter-function map is biased towards simple functions. International Conference on Learning Representations.
  103. Vardi, G. (2022). On the implicit bias in deep-learning algorithms.
  104. Implicit bias and fast convergence rates for self-attention.
  105. Mitigating simplicity bias in deep learning for improved ood generalization and robustness.
  106. Attention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  107. Transformers learn in-context by gradient descent. In International Conference on Machine Learning.
  108. Macrobert: Maximizing certified region of bert to adversarial word substitutions. In Database Systems for Advanced Applications: 26th International Conference, DASFAA 2021, Taipei, Taiwan, April 11–14, 2021, Proceedings, Part II 26, pages 253–261. Springer.
  109. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small.
  110. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.
  111. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv:1901.06523.
  112. Yang, G. (2021). Tensor programs i: Wide feedforward or recurrent neural networks of any architecture are gaussian processes.
  113. A fine-grained spectral perspective on neural networks.
  114. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations.
Citations (4)
View on Semantic Scholar

Summary

  • The paper demonstrates that transformers inherently favor low-sensitivity functions, resulting in enhanced robustness against perturbations.
  • It introduces sensitivity as a metric to quantify simplicity bias, linking lower sensitivity to improved generalization and stability.
  • Experimental results show transformers outperform LSTMs, MLPs, and CNNs, with data augmentation and regularization further reducing sensitivity.

Understanding the Simplicity Bias in Transformers: A Study on Sensitivity

Introduction to Spectral Bias in Neural Networks

The paper of inductive biases inherent in neural network architectures, particularly transformers, is essential for advancing our understanding of their learning mechanisms and performance across diverse tasks. Recent research has focused on unveiling the simplicity bias in neural networks, positing that these models prefer learning simpler functions over more complex ones. Among the various notions of simplicity, spectral bias, or the preference for functions with significant components in the lower-frequency domain, has garnered attention. This bias towards simplicity is believed to contribute to the models' generalization capabilities and robustness against perturbations.

Transformers, with their state-of-the-art performance in language, vision, and other domains, present an intriguing case paper for examining the presence and implications of simplicity bias. This post explores the concept of sensitivity as a unified metric for understanding the simplicity bias in transformers. Sensitivity, which measures a model's responsiveness to input perturbations, offers insights into how transformers prioritize lower-complexity functions across various data modalities.

Sensitivity as a Measure of Simplicity

Simplicity bias in neural networks, while beneficial for generalization, poses questions about the underlying preferences of transformer architectures. We propose using sensitivity, which quantifies the change in output with respect to small variations in the input, as a metric for assessing this bias. Lower sensitivity indicates a tendency towards simpler functions, suggesting that the model relies on robust, generalizable features rather than intricate patterns that could be susceptible to noise or perturbations.

Our investigations show that transformers exhibit a pronounced low-sensitivity bias, outperforming architectures like LSTMs, MLPs, and CNNs in terms of lower sensitivity across both vision and language tasks. This indicates a strong inductive preference for functions with broader stability against input changes, aligning with the observed robustness of transformers in practice.

Implications of Low Sensitivity for Robustness

One of the most significant findings of our paper is the correlation between low sensitivity and improved robustness in transformers. By analyzing performance on corrupted datasets such as CIFAR-10-C, we demonstrate that transformers' tendency towards lower sensitivity functions translates into superior robustness against a wide range of perturbations. Furthermore, interventions aimed at explicitly reducing sensitivity, either through data augmentation or regularization, further enhance this robustness, underscoring the causal relationship between low sensitivity and the ability to withstand input distortions.

Theoretical Perspectives and Future Directions

The findings on the low sensitivity bias in transformers not only shed light on the origins of their robustness and generalization but also open avenues for future research. Theoretical inquiries could explore the mechanisms that confer such bias in transformers compared to other architectures and ascertain whether a stronger spectral bias underpins their performance. Empirically, the connection between sensitivity and other desirable properties, like resistance to spurious correlations and long-term dependency modeling in sequence data, warrants further exploration.

Moreover, the gradient-based optimization processes, known to induce their own biases in learning dynamics, present another layer of complexity in dissecting the origins and implications of simplicity bias. Understanding how different optimization strategies interact with the inherent biases of transformers could illuminate paths to enhancing their efficacy and reliability across an even broader spectrum of applications.

Conclusion

In summary, our paper provides compelling evidence for the existence of a low sensitivity bias in transformers, manifesting in their preference for simpler, more robust functions across tasks. This bias not only contributes to their outstanding performance and robustness but also offers a lens through which to examine the foundational principles guiding their learning processes. As we continue to unravel the complexities of transformers, sensitivity stands out as a key metric for dissecting their inductive biases, paving the way for models that are not only powerful but also reliable and interpretable.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown