On Explicit Curvature Regularization in Deep Generative Models (2309.10237v1)
Abstract: We propose a family of curvature-based regularization terms for deep generative model learning. Explicit coordinate-invariant formulas for both intrinsic and extrinsic curvature measures are derived for the case of arbitrary data manifolds embedded in higher-dimensional Euclidean space. Because computing the curvature is a highly computation-intensive process involving the evaluation of second-order derivatives, efficient formulas are derived for approximately evaluating intrinsic and extrinsic curvatures. Comparative studies are conducted that compare the relative efficacy of intrinsic versus extrinsic curvature-based regularization measures, as well as performance comparisons against existing autoencoder training methods. Experiments involving noisy motion capture data confirm that curvature-based methods outperform existing autoencoder regularization methods, with intrinsic curvature measures slightly more effective than extrinsic curvature measures.
- What regularized auto-encoders learn from the data-generating distribution. The Journal of Machine Learning Research, 15(1):3563–3593, 2014.
- A grassmann manifold handbook: Basic geometry and computational aspects. arXiv preprint arXiv:2011.13699, 2020.
- Besse, A. L. Einstein manifolds. Springer Science & Business Media, 2007.
- Optimizing the latent space of generative networks. arXiv preprint arXiv:1707.05776, 2017.
- Flows for simultaneous manifold learning and density estimation. In Advances in neural information processing systems, 2020.
- Learning flat latent manifolds with vaes. arXiv preprint arXiv:2002.04881, 2020.
- Do Carmo, M. P. Riemannian geometry, volume 6. Springer, 1992.
- Do Carmo, M. P. Differential geometry of curves and surfaces: revised and updated second edition. Courier Dover Publications, 2016.
- A report on harmonic maps. Bulletin of the London mathematical society, 10(1):1–68, 1978.
- Another report on harmonic maps. Bulletin of the London Mathematical Society, 20(5):385–524, 1988.
- Einstein, A. The general theory of relativity. In The Meaning of Relativity, pp. 54–75. Springer, 1922.
- Fecko, M. Differential geometry and Lie groups for physicists. Cambridge university press, 2006.
- Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Processing, 87(2):250–262, 2007.
- Generative adversarial nets. In Advances in neural information processing systems, pp. 2672–2680, 2014.
- Hutchinson, M. F. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Communications in Statistics-Simulation and Computation, 18(3):1059–1076, 1989.
- Jang, C. Riemannian Distortion Measures for Non-Euclidean Data. PhD thesis, Seoul National University Graduate School, 2019.
- A riemannian geometric framework for manifold learning of non-euclidean data. Advances in Data Analysis and Classification, pp. 1–27, 2020.
- Geometrically regularized autoencoders for non-euclidean data. In The Eleventh International Conference on Learning Representations, 2022.
- Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
- Kramer, M. A. Nonlinear principal component analysis using autoassociative neural networks. AIChE journal, 37(2):233–243, 1991.
- Kühnel, W. Differential geometry, volume 77. American Mathematical Soc., 2015.
- Deep learning. nature, 521(7553):436–444, 2015.
- Lee, Y. Geometric Methods for Manifold Representation Learning. PhD thesis, Seoul National University Graduate School, 2023.
- Neighborhood reconstructing autoencoders. Advances in Neural Information Processing Systems, 34:536–546, 2021.
- A statistical manifold framework for point cloud data. In International Conference on Machine Learning, pp. 12378–12402. PMLR, 2022a.
- Regularized autoencoders for isometric representation learning. In International Conference on Learning Representations, 2022b. URL https://openreview.net/forum?id=mQxt8l7JL04.
- Ricci-Curbastro, G. Direzioni et invarianti principali in una varieta qualunque. Atti del Real Inst. Venezio, 63:1233–1239, 1904.
- Higher order contractive auto-encoder. In Joint European conference on machine learning and knowledge discovery in databases, pp. 645–660. Springer, 2011a.
- Contractive auto-encoders: Explicit invariance during feature extraction. In Icml, 2011b.
- Schutz, B. A first course in general relativity. Cambridge university press, 2022.
- Ntu rgb+ d: A large scale dataset for 3d human activity analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1010–1019, 2016.
- Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(12), 2010.
- Wong, Y.-C. Differential geometry of grassmann manifolds. Proceedings of the National Academy of Sciences, 57(3):589–594, 1967.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.