A distributional simplicity bias in the learning dynamics of transformers
Abstract: The remarkable capability of over-parameterised neural networks to generalise effectively has been explained by invoking a ``simplicity bias'': neural networks prevent overfitting by initially learning simple classifiers before progressing to more complex, non-linear functions. While simplicity biases have been described theoretically and experimentally in feed-forward networks for supervised learning, the extent to which they also explain the remarkable success of transformers trained with self-supervised techniques remains unclear. In our study, we demonstrate that transformers, trained on natural language data, also display a simplicity bias. Specifically, they sequentially learn many-body interactions among input tokens, reaching a saturation point in the prediction error for low-degree interactions while continuing to learn high-degree interactions. To conduct this analysis, we develop a procedure to generate \textit{clones} of a given natural language data set, which rigorously capture the interactions between tokens up to a specified order. This approach opens up the possibilities of studying how interactions of different orders in the data affect learning, in natural language processing and beyond.
- D. Saad and S.A. Solla. Exact Solution for On-Line Learning in Multilayer Neural Networks. Phys. Rev. Lett., 74(21):4337–4340, 1995.
- A spectral approach to generalization and optimization in neural networks. In ICLR, 2018.
- Deep learning generalizes because the parameter-function map is biased towards simple functions. In International Conference on Learning Representations. OpenReview.net, 2019.
- SGD on neural networks learns functions of increasing complexity. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 3491–3501, 2019.
- On the spectral bias of neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proc. of ICML, volume 97 of Proceedings of Machine Learning Research, pages 5301–5310. PMLR, 2019.
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In Yoshua Bengio and Yann LeCun, editors, Proc. of ICLR, 2014.
- A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences, 116(23):11537–11546, 2019.
- High-dimensional dynamics of generalization error in neural networks. Neural Networks, 132:428 – 446, 2020.
- A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences, 115(33):E7665–E7671, 2018.
- Implicit bias of MSE gradient optimization in underparameterized neural networks. In Proc. of ICLR. OpenReview.net, 2022.
- Spectral bias outside the training set for deep networks in the kernel regime. ArXiv preprint, abs/2206.02927, 2022.
- Gradient flow dynamics of shallow relu networks for square loss and orthogonal inputs. Advances in Neural Information Processing Systems, 35:20105–20118, 2022.
- Neural networks trained with sgd learn distributions of increasing complexity. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023.
- Sliding down the stairs: How correlated latent variables accelerate learning with neural networks. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 3024–3045. PMLR, 2024.
- Neural networks learn statistics of increasing complexity. arXiv preprint arXiv:2402.04362, 2024.
- Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pages 2397–2430. PMLR, 2023.
- Mapping of attention mechanisms to a generalized potts model. Phys. Rev. Res., 6:023057, 2024.
- Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
- Roberta: A robustly optimized bert pretraining approach, 2019.
- Pointer sentinel mixture models, 2016.
- Exposing the implicit energy networks behind masked language models via metropolis–hastings, 2022.
- Tinystories: How small can language models be and still speak coherent english?, 2023.
- Simplicity bias in transformers and their ability to learn sparse boolean functions. arXiv preprint arXiv:2211.12316, 2022.
- Transformers learn through gradual rank increase. Advances in Neural Information Processing Systems, 36, 2024.
- Simplicity bias of transformers to learn low sensitivity functions. arXiv preprint arXiv:2403.06925, 2024.
- Probing across time: What does roberta know and when? In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 820–842, 2021.
- The grammar-learning trajectories of neural language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8281–8297, 2022.
- How deep neural networks learn compositional data: The random hierarchy model. Physical Review X, 14(3):031001, 2024.
- Towards a theory of how the structure of language is acquired by deep neural networks. arXiv preprint arXiv:2406.00048, 2024.
- How transformers learn structured data: insights from hierarchical filtering. arXiv:2408.15138, 2024.
- Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30, 2017.
- Interpreting potts and transformer protein models through the lens of simplified attention. In Biocomputing 2022, pages 34–45, 2020.
- Transformer variational wave functions for frustrated quantum spin systems. Phys. Rev. Lett., 130:236401, 2023.
- Transformer wave function for the shastry-sutherland model: emergence of a spin-liquid phase. arXiv preprint arXiv:2311.16889, 2023.
- A simple linear algebra identity to optimize large-scale neural network quantum states. Communications Physics, 7(1), August 2024.
- Fine-tuning neural network quantum states. arXiv preprint arXiv:2403.07795, 2024.
- Language models are unsupervised multitask learners. https://insightcivic.s3.us-east-1.amazonaws.com/language-models.pdf, 2018.
- Online stochastic gradient descent on non-convex losses from high-dimensional inference. J. Mach. Learn. Res., 22(1), 2021.
- High-dimensional asymptotics of feature learning: How one gradient step improves the representation. Advances in Neural Information Processing Systems, 35:37932–37946, 2022.
- Smoothing the landscape boosts the signal for sgd: Optimal sample complexity for learning single index models. Advances in Neural Information Processing Systems, 36, 2024.
- Sgd learning on neural networks: leap complexity and saddle-to-saddle dynamics. In Gergely Neu and Lorenzo Rosasco, editors, Proceedings of Thirty Sixth Conference on Learning Theory, volume 195 of Proceedings of Machine Learning Research, pages 2552–2623. PMLR, 2023.
- Learning two-layer neural networks, one (giant) step at a time. arXiv:2305.18270, 2023.
- Learning time-scales in two-layers neural networks. arXiv preprint arXiv:2303.00055, 2023.
- ProteinBERT: a universal deep-learning model of protein sequence and function. Bioinformatics, 38(8):2102–2110, 02 2022.
- Prottrans: Toward understanding the language of life through self-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):7112–7127, 2022.
- Huggingface’s transformers: State-of-the-art natural language processing, 2020.
- How to train bert with an academic budget, 2021.
- Compatible conditional distributions. Journal of the American Statistical Association, 1989.
- Bertscore: Evaluating text generation with bert, 2020.
- On layer normalization in the transformer architecture, 2020.
Paper Prompts
Sign up for free to create and run prompts on this paper.