Characterize the validation-gradient signal for large-dataset memorization onset

Characterize the role and behavior of the norm of the loss gradient averaged over validation data as a statistic for locating the onset of memorization in diffusion models trained on larger datasets.

Background

The paper proposes the local maximum of the relative training–validation curvature error as a sampling-free stopping criterion for diffusion-model training. However, the experiments indicate that this maximum occurs during a period when memorization is already high, rather than precisely at its onset.

For small datasets, the authors report that the norm of the loss gradient averaged over validation data may provide a more useful signal. They explicitly state that the role and behavior of this statistic for larger datasets remain unresolved, leaving open whether it can reliably identify the transition into memorization in that regime.