Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent (2010.09697v5)

Abstract: The capacity of neural networks like the widely adopted transformer is known to be very high. Evidence is emerging that they learn successfully due to inductive bias in the training routine, typically a variant of gradient descent (GD). To better understand this bias, we study the tendency for transformer parameters to grow in magnitude ($\ell_2$ norm) during training, and its implications for the emergent representations within self attention layers. Empirically, we document norm growth in the training of transformer LLMs, including T5 during its pretraining. As the parameters grow in magnitude, we prove that the network approximates a discretized network with saturated activation functions. Such "saturated" networks are known to have a reduced capacity compared to the full network family that can be described in terms of formal languages and automata. Our results suggest saturation is a new characterization of an inductive bias implicit in GD of particular interest for NLP. We leverage the emergent discrete structure in a saturated transformer to analyze the role of different attention heads, finding that some focus locally on a small number of positions, while other heads compute global averages, allowing counting. We believe understanding the interplay between these two capabilities may shed further light on the structure of computation within large transformers.

• The paper demonstrates that transformer parameters grow proportionally to √t during training, highlighting gradient descent’s role in inducing model biases.
• Empirical analysis on models like T5 shows faster norm growth in later layers, pointing to varying learning dynamics across the transformer architecture.
• The study connects theoretical insights on network saturation with specialized attention head behaviors, suggesting practical improvements for NLP tasks.

Analysis of Parameter Norm Growth During Transformer Training: Implications of Gradient Descent-Induced Inductive Bias

The paper presented in the paper "Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent" investigates the implications of parameter norm growth in transformers trained with gradient descent (GD). It highlights how these growing norms influence the representations that transformers acquire, suggesting the emergence of inductive biases during training. This work utilizes a blend of empirical evidence and theoretical analysis to examine the parameters' growth in magnitude across the transformer architecture, particularly focusing on LLMs.

Empirical Findings on Norm Growth

Through detailed observation of T5 and other transformer models, the authors document significant growth in parameter $\ell_2$ norms during training, often proportional to $\sqrt{t}$ where $t$ is the training timestep. These empirical results hold consistently across various layers of the models and different tasks, suggesting a pervasive trend of norm growth in transformers trained on NLP tasks. Interestingly, the paper finds faster norm growth in later layers compared to earlier ones, a phenomenon that could indicate differentiated learning dynamics across the layer stack of transformers.

Theoretical Implications of Saturation

The analysis extends into theoretical terrain, offering that as parameters grow, transformers tend towards a "saturated" network state where their non-linear activation functions become effectively discretized. Such saturated networks, characterized by reduced capacity compared to their unsaturated counterparts, are analyzed in terms of formal languages and automata, which presents a novel angle of understanding neural network behavior through formal language theory.

The paper further provides a theoretical argument that under conditions of uniform parameter growth, transformers approximate these saturated networks, where saturation emerges as a consequence of the learning dynamics rather than architectural adjustments. This saturation state is particularly insightful for NLP tasks, wherein it can be linked to the inductive biases that naturally emerge during the training process guided by GD.

Attention Heads and Representation Dynamics

The behavior of saturated networks manifests in attention head dynamics. Empirically, the paper finds that certain heads in transformer models tend to focus on localized positions while others operate more globally, effectively computing averages. These distinct behaviors suggest that individual heads develop specialized functions during training, possibly as a result of inductive biases. This differentiation might play a role in transformers' ability to approximate certain linguistic phenomena and manage diverse computational tasks such as counting.

Exploration of Norm Growth Mechanisms

To explain why the norm growth occurs, the authors dive into the mathematics of homogeneous networks, proposing that the approximate homogeneity of transformers contributes to the observed dynamics. Specifically, they present two models of understanding training dynamics: aligned and misaligned dynamics, with empirical studies on T5 leaning towards misalignment, as evidenced by diminishing alignment between gradient directions and parameter vectors over time.

Future Directions and Implications

The implications of this research extend to both practical applications and theoretical understanding. Practically, recognizing how norm growth influences model behavior could inform better training protocols or architectural modifications to optimize performance further. Theoretically, saturation and its ability to predict the behavior of trained transformers can be pivotal in developing more interpretable and robust LLMs, helping reconcile formal language theory with contemporary deep learning.

Moving forward, the community might explore how adjustments in learning rate schedules, optimization strategies, and architectural components might control or leverage parameter norm growth strategically. Additionally, disentangling the interplay between saturation, computational capacity, and linguistic representation could open new avenues for constructing AI systems that align more effectively with the complexities of language dynamics. The paper thus invites continued examination into leveraging the emergent properties of neural network training for sophisticated tasks in AI and NLP.