On-Manifold Exploration

• On-manifold exploration is a framework that systematically discovers and samples from the low-dimensional structures embedded within high-dimensional data.
• It employs geometric methods like RMHMC and diffusion maps to align stochastic exploration with the intrinsic curvature of target distributions.
• The approach enables scalable optimization and efficient design in settings such as reinforcement learning, experimental design, and generative modeling.

On-manifold exploration refers to the systematic discovery, sampling, or optimization constrained to the intrinsic geometry of a manifold embedded within a higher-dimensional space. This paradigm underlies a spectrum of methodologies across Monte Carlo methods, optimization, reinforcement learning, active learning, generative modeling, and representation learning, with the shared emphasis that exploration is most efficient and semantically meaningful when restricted to the low-dimensional structure (manifold) that supports the data or solution space. On-manifold constraints can reflect the geometry of target distributions, feasible action spaces, optimal designs, valid molecular conformations, or physically realizable materials microstructures.

1. Geometric Monte Carlo and Manifold Exploration

The use of geometric information for efficient stochastic exploration is exemplified by Riemannian Manifold Hamiltonian Monte Carlo (RMHMC) (0907.1100) and variants such as Lagrangian Manifold Monte Carlo on Monge Patches (Hartmann et al., 2022). RMHMC replaces a fixed Euclidean mass matrix with a position-dependent metric tensor $G(\theta)$, often taken as the Fisher information matrix, to adapt the proposal mechanism of HMC to the local curvature:

$$H(\theta, p) = -\log \pi(\theta) + \tfrac{1}{2}\log |G(\theta)| + \tfrac{1}{2} p^\top G(\theta)^{-1} p$$

where gradients and determinants of the metric regulate simulated Hamiltonian dynamics, resulting in trajectories that track narrow, curved ridges of the target distribution manifold. The associated symplectic integrator operates with a semi-explicit update retaining the symplecticity needed for high acceptance rates. Such geometry-aware sampling yields dramatic improvements in effective sample size and decorrelation, specifically when target distributions are highly anisotropic or have strong correlations.

Lagrangian approaches on Monge-patch-induced metrics, e.g., $$G_M(x) = I + \alpha^2 \nabla \ell(x) \nabla \ell(x)^\top$$ for $\ell(x) = \log\pi_X(x)$, retain the computational efficiency (using Sherman–Morrison formula for inversion and determinant) by using only first-order information, while still encoding local density-driven stretching of space (Hartmann et al., 2022). This makes explicit on-manifold integration feasible in higher dimensions.

2. Manifold Learning and On-the-Fly Manifold Construction

Nonlinear dimensionality reduction constitutes a foundational mechanism for on-manifold exploration in high-dimensional spaces. Techniques such as Diffusion Maps (DMAPs) (Holiday et al., 2018), conformally invariant diffusion maps (CIDM) (Mahler et al., 2023), and spectral geometry analysis based on the Laplace–Beltrami operator (Elhag et al., 2023) are employed to uncover low-dimensional intrinsic coordinates. In parameter reduction and model optimization, DMAPs utilize output-driven similarity measures:

$$K(p,p') = \exp\left(-\frac{\|f(p) - f(p')\|^2}{\epsilon^2}\right)$$

to reveal effective parameter combinations dictating system behavior, collapsing the input–output mapping onto an intrinsically meaningful submanifold (Holiday et al., 2018).

In molecular dynamics (Chiavazzo et al., 2016), iMapD overlays these techniques with geometric extensions—local PCA provides tangents at the boundary of explored regions, and "lifting" via local linear maps projects out-of-sample extensions back to the physical configuration space. The critical innovation is data-driven extrapolation and reinitialization on the intrinsic manifold, vastly accelerating discovery of metastable states versus brute-force simulation.

Fast, on-the-fly, coarse-grained optimizers (Pozharskiy et al., 2020) further use short-burst local exploration plus DMAPs to define the slow manifold of significant directions, so that stochastic optimization can be projected, and then lifted, leading to rapid convergence in complex landscapes.

3. Manifold-Constrained Optimization and Experimental Design

Optimization subject to manifold constraints recasts complex search spaces as smooth Riemannian manifolds. Classic examples include the sphere, Stiefel, and Grassmann manifolds, as well as more flexible relaxations such as the Relaxed Indicator Matrix (RIM) manifold (Yuan et al., 26 Mar 2025). The toolbox for on-manifold exploration in optimization includes:

• Riemannian gradient and Hessian computation via tangent space projection,
• Various retraction operators (exponential map, Dykstra’s projection, Sinkhorn-based) for stepping along the manifold while respecting curvature and constraints,
• Fast iterative solvers operating at $O(n)$ complexity for large matrix manifolds with millions of variables.

These strategies are validated in contexts such as image denoising and large-scale clustering, where switching from double stochastic to RIM manifold optimization yields both superior objective values and two orders of magnitude speedup in convergence (Yuan et al., 26 Mar 2025).

In experimental design, manifold regularization is integrated into D-optimal and G-optimal design frameworks, whereby the information matrix is augmented with Laplacian penalties to account for the manifold structure of the covariate space (Li et al., 2019):

$$M_{\mathrm{Lap}}(\varepsilon) = \int_{x} \xi(x) g(x)g(x)^\top\,dx + C,\quad C = \lambda_A I_p + \lambda_I \int_{\mathcal{M}} g(x) \Delta_{\mathcal{M}} g(x)^\top\,d\mu$$

The ODOEM algorithm guarantees convergence and outperforms Euclidean design strategies, especially when data lie on complex manifolds.

4. On-Manifold Exploration in Machine Learning and Robotics

Modern machine learning exploits manifold structures for more robust representation and exploration. MAADA (Satou et al., 21 May 2025) decomposes adversarial data augmentation into on-manifold (tangent space, semantic) and off-manifold (normal, nonsemantic) perturbations, enforcing invariance and smoothness via respective loss terms:

• On-manifold consistency: $||f(x) - f(x^{on})||^2 \leq \varepsilon_c$, with $$x^{on} = x + \alpha \cdot \mathrm{Proj}_{T_x(\mathcal{M})}(\nabla_x\ell(f(x),y))$$.
• Off-manifold regularization: applied to $$x^{off} = x + \beta \cdot \delta_{off}/||\delta_{off}||$$, with $\delta_{off}$ orthogonal to the manifold.

A geometry-aware alignment loss based on geodesic discrepancy and curvature mismatch minimizes the domain shift in transfer learning under differing source and target manifolds.

In robotic policy improvement, SOE (Jin et al., 23 Sep 2025) builds a low-dimensional latent action manifold via a variational information bottleneck (VIB):

$\max_\theta I(Z;A) - \beta I(Z;O)$

with the exploration constrained by structured perturbation of the bottleneck. This yields safe, sample-efficient improvement without sacrificing baseline policy performance. Signal-to-noise ratio analysis of latent dimensions supports human-in-the-loop steering along semantically interpretable directions.

5. On-Manifold Generative and Exploratory Modeling

Generative modeling has moved towards explicit on-manifold exploration in several domains:

• Manifold Diffusion Fields (MDF) (Elhag et al., 2023) use spectral geometry (Laplace–Beltrami eigenfunctions) for intrinsic encodings, diffusing only the signal while retaining fixed, geometry-aware positional embeddings. This enables the generation of continuous functions on arbitrary manifolds, and superior fidelity/diversity in both synthetic and scientific applications (e.g., climate field generation, molecular conformations).
• Diffusion models for maximum-entropy manifold exploration (Santi et al., 18 Jun 2025) formalize exploration as entropy maximization over the approximate data manifold defined by a pretrained diffusion model. The entropy variation,

$\delta H(\mu)(x) = -\log \mu(x)$

is related directly to the model's score function, enabling fine-tuning via mirror descent to push sample density towards low-density, novel regions of the manifold without explicit density estimation.

Such approaches bypass the limitations of uncertainty quantification or explicit coverage-based objectives in generative exploration, scaling to high-dimensional data with provable convergence guarantees.

6. Statistical and Representation-Theoretic Perspectives

The statistical foundation for on-manifold exploration is provided by the Latent Metric Model (LMM) (Whiteley et al., 2022). High-dimensional data, $Y_i$, are described as noisy projections of latent functions $X_j(Z_i)$ where $Z_i$ are sampled from a latent low-dimensional compact domain. PCA and spherical projections recover the latent geometry, while graph-analytic algorithms such as k-NN graphs and persistent homology measure true geodesic distances and topological invariants directly from data.

In affordance labeling (Özçil et al., 22 Jul 2024), subspace clustering and local curvature-based manifold estimation from pre-trained feature vectors yield interpretable, label-efficient affordance recognition that can even uncover latent categories not annotated in the ground truth—demonstrating the descriptive power of geometric on-manifold methods.

In materials science, mapping microstructural variability to a low-dimensional stochastic material manifold (Mason et al., 18 Sep 2025) enables invertible bijections between processing conditions and microstructure, underpinning model-based optimization and exploratory design of novel materials.

7. Future Directions and Open Challenges

Outstanding challenges in on-manifold exploration include:

• Scalable, scalable estimation and projection operators in high-dimensional, nonlinear, or unknown manifold geometries—especially when the manifold is only accessible via samples.
• Intrinsic versus extrinsic Gaussian process modeling for Bayesian optimization (Fang et al., 2022, Liu et al., 2023): heat-kernel-based surrogates and Brownian motion sampling respect intrinsic distances, but necessitate large-scale SDE simulation and computational efficiency innovations.
• Unified frameworks for aligning, transferring, and regularizing across manifolds with differing or evolving geometry, as necessitated by domain adaptation or lifelong learning.
• Robust methodologies for handling singularities, boundaries, and non-smooth structures in real-world manifolds, especially in geometric deep learning and the physical sciences.

On-manifold exploration is now a central paradigm in both the analysis and modeling of data, providing a geometric backbone for principled, efficient, and interpretable discovery across scientific and engineering domains.