Multi-Scale Geometric Autoencoder

Abstract: Autoencoders have emerged as powerful models for visualization and dimensionality reduction based on the fundamental assumption that high-dimensional data is generated from a low-dimensional manifold. A critical challenge in autoencoder design is to preserve the geometric structure of data in the latent space, with existing approaches typically focusing on either global or local geometric properties separately. Global approaches often encounter errors in distance approximation that accumulate, while local methods frequently converge to suboptimal solutions that distort large-scale relationships. We propose Multi-Scale Geometric Autoencoder (MAE), which introduces an asymmetric architecture that simultaneously preserves both scales of the geometric structure by applying global distance constraints to the encoder and local geometric constraints to the decoder. Through theoretical analysis, we establish that this asymmetric design aligns naturally with the distinct roles of the encoder and decoder components. Our comprehensive experiments on both synthetic manifolds and real-world datasets demonstrate that MAE consistently outperforms existing methods across various evaluation metrics.