Principled noise and data-augmentation schedules for denoising autoencoders

Develop theoretically grounded schedules for the input corruption noise level over training time and for data augmentation—specifically, the number of independent noisy realizations per clean input—when training denoising autoencoders, with the objective of minimizing the final mean squared reconstruction error at a specified test noise level.

Background

The paper analyzes denoising autoencoders using a high-dimensional teacher–student framework and demonstrates that optimized noise and batch-augmentation schedules can substantially improve performance over constant heuristics. Despite empirical practices such as scheduled denoising and non-uniform sampling in diffusion models, a principled theoretical derivation of optimal noise and augmentation schedules has been lacking.

This problem calls for formal development and justification of optimal corruption-level and augmentation schedules tailored to denoising autoencoders, connecting the schedule to data statistics, architecture, and training dynamics to guarantee improved reconstruction performance.

References

However, identifying principled noise schedules and data augmentation strategies remains largely an open problem.

— A statistical physics framework for optimal learning  (2507.07907 - Mignacco et al., 10 Jul 2025) in Section 4.3 (Denoising autoencoder)

The questions motivating this work have to do with the following important aspects of SSL which remain opaque: despite the various interdependencies between terms in $\widehat r({\cal V})$, owing to multiple views $\phi_l(X_i), l \in [L]$ of the same datapoints $X_i$, SSL approaches are very succesful in practice, and even appearing to converge faster with larger $L$, i.e., despite (apparently) increased dependency (as brought up, e.g., by empirical works such as ). Yet, these perplexing empirical observations remain unresolved; emerging theoretical literature predominantly treats $L$ as a small constant\footnote{For instance, $L =2$, with objectives of the form $\frac{1}{m} \sum_{i \in [m]} \ell_{\cal V} (\phi_1(X_i), \phi_2(X_i))$, with $\phi_1, \phi_2$ independently drawn conditioned on each $X_i$ (see, e.g., ).}, thus bypassing the complex interaction between interdependent views, and their effect on performance.

— An Analysis of Self-supervised Pre-training with Dependent Samples  (2609.05031 - Fleissner et al., 4 Sep 2026) in Section 1, Introduction