Low Bending & Distortion Embeddings

• Low bending and low distortion embeddings are mappings that preserve local geodesic distances and surface flatness, ensuring nearly isometric and flat local representations.
• They are achieved by regularizing encoder networks and constructing eigenmaps that penalize deviations from local isometry and flatness via second-order loss functions.
• Empirical and theoretical analyses demonstrate improved latent space interpolation, reduced dimensional redundancy, and robust handling of non-Euclidean manifolds.

Low bending and low distortion embeddings are constructions of mappings that transfer a geometric or metric structure (typically a manifold or discrete metric space) into a lower-dimensional or different ambient space such that local distances are nearly preserved (low distortion) and the embedding is locally as flat as possible (low bending). These notions are central to manifold learning, geometric representation in latent spaces, computational topology, and embeddings of metric spaces in Banach analysis.

1. Mathematical Definition: Distortion and Bending

Let $M$ be an $m$-dimensional Riemannian manifold with metric $g$, and $\phi: M \to \mathbb{R}^l$ an embedding. The two primary geometric regularity quantities are:

Distortion quantifies deviation from local isometry. For $x, y \in M$ with geodesic distance $d_M(x, y)$: $$\Delta\phi(x, y) = \frac{\|\phi(y) - \phi(x)\|}{d_M(x, y)}$$ Low distortion is achieved if, for nearby points, $\Delta\phi(x, y) \approx 1$, i.e., the Euclidean distance in the embedding approximates the intrinsic distance.

Bending quantifies deviation from local flatness. The second-order difference (bending error) is

$$\Delta^2\phi(x, y) = 8 \frac{\frac{1}{2}(\phi(x) + \phi(y)) - \phi(\operatorname{av}_M(x, y))}{d_M(x, y)^2}$$

where $\operatorname{av}_M(x, y)$ is the Riemannian midpoint of $m$0 and $m$1. Low bending means that the midpoint in the embedding is close to the arithmetic mean, indicating extrinsic flatness.

For discrete metric spaces, distortion similarly refers to the preservation of pairwise distances, and bending relates to controlling the curvature (angular change) in glued finite-dimensional pieces.

2. Low Bending and Low Distortion Regularization in Encoders

The encoder component of an autoencoder can be regularized to promote embeddings that realize the aforementioned properties. The precise loss functional introduced by Braunsmann et al. is: $m$2 where $m$3 is a penalty minimized at $m$4 (for example, $m$5), and $m$6 tunes the bend penalty. As $m$7 (with $m$8 sampled for short $m$9), the expectation converges to a continuous nonlocal energy, which in the limit $g$0 (sampling radius) yields the local elastic energy: $g$1 The first term penalizes deviations from isometry, the second penalizes extrinsic bending, as measured by the Hessian (Braunsmann et al., 2022, Braunsmann et al., 2021).

This loss allows encoder-only training using only sampled pairs $g$2.

3. Theoretical Guarantees: Γ-Convergence and Existence

Rigorous analysis has established that the empirical loss functional $g$3 $g$4-converges (in the sense of Mosco convergence) to the local variational energy as the sampling becomes dense and neighborhoods shrink, under standard regularity assumptions (injectivity radius, cone condition, smoothness of $g$5). The main consequences:

• Any sequence of minimizers has subsequences converging (weakly in $g$6) to a minimizer of the limiting energy.
• For the flat disc $g$7, the unique global minimizer is (up to rigid motion) the standard linear embedding.
• The restriction to neural network encoders with uniform first and second derivative bounds is sufficient for convergence provided the class is dense in $g$8 (Braunsmann et al., 2022, Braunsmann et al., 2021).

4. Low Distortion Embeddings in Metric and Banach Spaces

In the context of embedding locally finite metric spaces into Banach spaces, Catrina–Ostrovskii constructed global embeddings with distortion arbitrarily close to $g$9, using a logarithmic spiral gluing of nearly-isometric finite-dimensional pieces: $\phi: M \to \mathbb{R}^l$0 Here $\phi: M \to \mathbb{R}^l$1 are finite-dimensional almost-isometric embeddings, and $\phi: M \to \mathbb{R}^l$2 are smooth partition-of-unity functions akin to a slow spiral in a Euclidean plane. The transition rate $\phi: M \to \mathbb{R}^l$3 is chosen to be $\phi: M \to \mathbb{R}^l$4, providing low bending (small angular speed). The construction ensures both a Lipschitz and a co-Lipschitz constant controlled uniformly, yielding distortion $\phi: M \to \mathbb{R}^l$5 (Catrina et al., 2023).

Bending, in this context, refers to the angular deviation introduced during gluing; logarithmic spiral gluing keeps this bending small, ensuring the global embedding is close to a geodesic path locally.

5. Algorithmic Realizations and Sampling

Autoencoder-based approach: Encoders are instantiated as deep CNNs with layers designed for spatial regularity (e.g., block-wise Conv2d → LeakyReLU → AvgPool), typically outputting to a latent dimension $\phi: M \to \mathbb{R}^l$6 slightly above the manifold's intrinsic dimension $\phi: M \to \mathbb{R}^l$7. Sampling for the loss is accomplished via Monte Carlo integration using triplets $\phi: M \to \mathbb{R}^l$8 with $\phi: M \to \mathbb{R}^l$9. Hyperparameters include the bending regularization $x, y \in M$0, neighborhood radius $x, y \in M$1, and learning rate (Braunsmann et al., 2022, Braunsmann et al., 2021).

Discrete eigenmap-based approach: LDLE (Low Distortion Local Eigenmaps) builds local bilipschitz charts around each data point using custom subsets of Laplacian eigenvectors, then registers charts globally by Procrustes analysis to obtain low distortion and low bending at chart overlaps. The process involves local correlation matrices estimating gradient inner products, careful eigenvector selection per neighborhood, and iterative rigid+scale alignment. The “tearing” mechanism allows embedding of closed/non-orientable manifolds into Euclidean spaces by locally cutting and providing correspondence ("gluing instructions") (Kohli et al., 2021).

Metric/Banach space construction: Logarithmic spiral gluing transitions between finite-dimensional isometric embeddings via smooth angular progressions, sidestepping the previously high-distortion barycentric constructions. The construction is explicit and does not require neural optimization (Catrina et al., 2023).

6. Empirical Results and Practical Impact

Empirical studies confirm that low bending and low distortion regularization in latent space produces embeddings with:

• Low local distance error (quasi-isometry) up to the sampling radius.
• Significantly reduced interpolation error for linear paths in latent space: interpolated decoded images closely approximate true geodesic midpoints on the source manifold.
• Improved latent manifold flattening: the support is smooth and lower-dimensional (evident in PCA-projected embeddings).
• Fewer active dimensions for explaining variance when $x, y \in M$2.
• Robustness to quantization noise and sampling size, with bending regularization ($x, y \in M$3) dominating over sampling radius ($x, y \in M$4) for regularity.
• In LDLE, superior distortion metrics compared to other manifold learning techniques (UMAP, t-SNE, Laplacian Eigenmaps), and the ability to embed closed/non-orientable manifolds with gluing instructions (Braunsmann et al., 2022, Braunsmann et al., 2021, Kohli et al., 2021).

Tables below summarize core characteristics:

Approach	Distortion Control	Bending Control	Construction/Optimization
Encoder LBD Loss	Monte Carlo pairwise	Second-order loss	Neural net, variational loss
LDLE	Local eigenmaps	Rigid/global align	Eigenvector selection + Procrustes
Spiral Gluing	Bilipschitz estimate	Slow angle gluing	Explicit, Banach-theoretic

7. Limitations, Open Questions, and Future Directions

The following limitations and open problems are identified:

• Manifold feature requirements: Accurate local geodesic distances and midpoint evaluation are required. For high-dimensional/unstructured data, this incurs computational expense (Braunsmann et al., 2022, Braunsmann et al., 2021).
• Locality of constraint: Regularization enforces local isometry and flatness only; global geometric or topological correctness may require supplementary priors or reconstruction loss.
• High-order smoothness: Bending penalties depend on $x, y \in M$5-regularity, which challenges expressive capacity and training stability in very deep neural networks.
• Efficient Hessian estimation: Direct estimation of Hessians in neural networks is nontrivial; alternatives or approximate schemes are an open area.
• Parameter selection: Automated choice of neighborhood radius $x, y \in M$6 and regularization coefficient $x, y \in M$7 remains unresolved.
• Non-flat geometries: Characterizing global minimizers and geometric properties for curved intrinsic geometries is largely open.
• Extension to noisy and stochastic settings: Adaptation of these frameworks to settings with noisy or only pairwise affinity data rather than ground-truth geometry is under investigation (Braunsmann et al., 2022, Braunsmann et al., 2021, Kohli et al., 2021).

A plausible implication is that as estimates of geodesic structure or pairwise affinities improve algorithmically, these regularization frameworks will become increasingly practical for high-dimensional, unstructured, or non-Euclidean data.

---

References:

• "Convergent autoencoder approximation of low bending and low distortion manifold embeddings" (Braunsmann et al., 2022)
• "Learning low bending and low distortion manifold embeddings" (Braunsmann et al., 2021)
• "LDLE: Low Distortion Local Eigenmaps" (Kohli et al., 2021)
• "Finite determination for embeddings into Banach spaces: new proof, low distortion" (Catrina et al., 2023)