Lyapunov characterization of norm compression

Determine whether the norm-compression trajectory is a Lyapunov function for the memorizing regime, with weight decay controlling the rate at which the trajectory descends.

Background

The paper reports that weight norms decrease monotonically during the memorization phase in the configurations examined and that the rate of norm decrease is associated with earlier grokking, whereas the absolute norm at memorization is not predictive of onset time.

It remains unresolved whether this empirical monotonicity has the stronger dynamical-systems interpretation of a Lyapunov function. Establishing such a result would provide a formal account of how weight decay drives networks through the memorizing regime toward generalization.