Implicit regularization in autoencoder training without topology-preserving loss

Identify the specific implicit regularization mechanisms that arise during training of autoencoders using only the standard reconstruction loss (i.e., excluding the temporal consistency term L2) that lead, in particular training runs, to a latent-space dynamical system topologically equivalent to the original flow generating the data.

Background

The paper introduces an autoencoder-based method to reconstruct a dynamical system's flow from recorded data, proposing a loss function composed of a standard reconstruction term and an additional term that enforces consistency of velocities between input space and latent space. This second term is designed to ensure that the latent-space flow is topologically equivalent to the original system's flow.

The authors note that even without the added consistency term, autoencoders can sometimes produce a topologically equivalent latent flow. They suggest this may be due to induced bias or implicit regularization during training, but explicitly state that the nature of this regularization is currently unclear, motivating the open problem.