Intrinsic Green's Learning: Supervised Learning on Manifolds via Inverse PDE

Abstract: We introduce Intrinsic Green's Learning (IGL), a framework that models a target function on a manifold as the solution to a linear PDE whose source term is learned from data. Rather than approximating the target directly, IGL learns a source and integrates it against a Green's kernel. An encoder discovers a low-dimensional coordinate chart on the manifold where both the source and the kernel decompose as low-rank tensors, collapsing a high-dimensional integral into independent one-dimensional integrals with cost linear in the intrinsic dimension. A two-stage algorithm separates coordinate discovery from source fitting, a near-convex linear solve, preventing the dimensional collapse of joint training. Learnable gates on each coordinate automatically discover the intrinsic dimension of the manifold. We validate IGL on synthetic manifolds and on MNIST, where it simultaneously achieves near-optimal classification and automatic recovery of the intrinsic dimension.

• The paper's main contribution is a two-stage algorithm that decouples coordinate discovery from source fitting, preventing dimensional collapse and boosting sample efficiency.
• It leverages tensor factorization and operator superposition to scale linearly with intrinsic dimensions while maintaining the manifold's topology.
• Empirical results demonstrate IGL's superior accuracy and robust geometric regularization across synthetic, high-dimensional, and MNIST datasets, with significant gains in R² performance.

Intrinsic Green's Learning: Inverse PDE-Based Supervised Learning on Manifolds

Framework Overview and Motivation

Intrinsic Green's Learning (IGL) introduces a new paradigm for supervised learning on data manifolds by positing targets as solutions to linear PDEs with data-driven sources. Rather than directly regressing on the desired function $u$, IGL first learns the source term $f$ for the PDE $Lu = f$, then constructs $u$ via integration against the corresponding Green's function $G$. The framework exploits the hypothesis that observed data lie on a low-dimensional manifold $\mathcal{M} \subset \mathbb{R}^D$ with intrinsic dimension $d \ll D$. By leveraging differentiable coordinate charts and tensor factorization, IGL collapses the associated $d$-dimensional convolution integrals into efficient combinations of 1D operations, producing models whose complexity is linear in the intrinsic dimension and not ambient dimension.

The algorithm exhibits a two-stage structure: encoder-driven coordinate discovery followed by near-convex source fitting. Stage 1 computes a coordinate chart where both the source term and kernel admit low-rank tensor decompositions; Stage 2 performs linear regression to fit source and kernel parameters, decoupling geometric complexity from fitting. Learnable gating mechanisms drive automatic dimension discovery, allowing the effective latent dimensionality to emerge from task requirements.

Two-Stage Algorithm: Topology Preservation and Sample Efficiency

The separation of coordinate discovery from source fitting is crucial to maintaining manifold topology and preventing dimensional collapse. Empirical results on synthetic manifolds demonstrate that joint optimization leads to degenerate embeddings or collapse, while the two-stage approach reliably reconstructs the intrinsic geometry. The convergence curves in the aligned and rotated Swiss Roll experiments evidence over 100-fold reduction in MSE due to the two-stage method, with recovered latent spaces retaining 2D topology irrespective of initialization.

Figure 1: Two-stage training prevents dimensional collapse. Top: aligned manifold, bottom: rotated manifold; two-stage recovers intrinsic topology, joint training collapses.

The sample efficiency experiment further elucidates a phase transition: two-stage training achieves $R^2 > 0.9$ with only a few hundred samples, while joint optimization saturates at $R^2 \approx 0.55$ even with thousands. Notably, the traditional MLP and kernel ridge baselines fail to approach the accuracy achieved by IGL.

Figure 2: Two-stage (IGL) achieves dramatically higher sample efficiency on a rotated plane in $f$0, with a sharp phase transition in $f$1.

Computational Scalability

IGL's computational cost is dominated by the encoder, which is agnostic to the backbone architecture and can utilize MLPs or CNNs. The source fitting stage adds only $f$2 cost, independent of ambient dimension and negligible relative to the encoder. Experiments varying ambient dimension $f$3, intrinsic dimension $f$4, and rank $f$5 corroborate linear scaling in each, with source fitting overhead remaining a small fraction.

Figure 3: Encoder cost scales linearly with ambient dimension and is independent of source rank or intrinsic dimension; source fitting adds negligible overhead.

Expressivity: Operator Superposition and Gabor Bases

IGL's architectural choices circumvent the typical smoothness bias of Green's function-based models. By superposing operators (e.g., Laplacian, Helmholtz, harmonic oscillator) and utilizing Gabor wavelets as basis functions, IGL can capture sharp, non-axis-aligned decision boundaries. The empirical results for rotated geometric patterns show that Stage 1 discovers rotation-invariant coordinates, after which Stage 2's Gabor basis achieves high fidelity for discontinuous and textured targets.

Figure 4: Gabor wavelets in IGL capture sharp transitions on rotated datasets, overcoming smoothness bias.

Hybrid Architectures: Geometric Regularization

IGL can be integrated as a parallel branch to conventional MLP architectures (ResIGL). In modeling targets with discontinuities, the IGL branch maintains the geometric integrity and stratification of iso-curves, while the MLP branch captures non-smooth features. This hybrid yields outputs with preserved topology, demonstrating IGL's efficacy as a geometric regularizer.

Figure 5: ResIGL preserves manifold stratification for discontinuous targets, whereas pure MLP collapses geometry.

Coordinate Discovery and Regression in High Dimensions

Experiments on randomly rotated 2D manifolds embedded in $f$6 validate the full pipeline: the encoder recovers a 2D chart despite arbitrary rotation, and source fitting achieves $f$7 for challenging multi-frequency targets. The learned latent space aligns with the intrinsic manifold, confirming the theoretical scaling properties.

Figure 6: IGL discovers intrinsic coordinates and achieves precise regression even in high-dimensional ambient spaces.

Latent Space Regularization on MNIST

IGL serves as an effective topological regularizer for autoencoder latent spaces in classification tasks. On MNIST, the two-stage IGL variant produces geometrically smooth clusters in latent space and discovers the effective dimensionality (12 out of 64 possible), while contrastive methods yield tighter clusters but lack dimension discovery. The IGL latent representations enable nearly optimal linear probe accuracy, high silhouette scores, and reduced label smoothness, matching prior estimates of MNIST intrinsic dimension.

Figure 7: IGL yields smooth, dimension-reduced clusters in AE latent space on MNIST; contrastive learning achieves cluster tightness but lacks dimension discovery.

Theoretical Implications and Algorithmic Guarantees

IGL's expressivity is governed by tensor decomposition theory and operator choice. The Green's function admits efficient exponential sum approximations ($f$8 rank), independent of intrinsic dimension. Bounded mixed derivatives, rather than Sobolev regularity, guarantee tensor efficiency, which justifies the encoder-driven coordinate search. Compensation mechanisms absorb non-separabilities induced by learned charts, and the envelope theorem ensures encoder gradients faithfully update topology discovery parameters.

Hybrid architectures and operator superposition imbue IGL with flexibility to serve as a regularizer alongside classic deep models, integrating geometric priors without hindering expressivity.

Practical Implications and Future Directions

The practical utility of IGL as a geometric regularizer is apparent in vision and manifold learning domains, offering tractable, scalable modeling for autoencoders, classifiers, and regression tasks on low-dimensional structures. The two-stage architecture is amenable to meta-learning, as Stage 1 can be amortized across tasks, adapting the lightweight Stage 2 for rapid problem-specific fitting. Extension to higher-dimensional benchmarks (e.g., CIFAR-10, ImageNet) and more complex manifold structures is a logical progression.

Conclusion

IGL reframes supervised learning as an inverse problem for linear PDEs, separating geometry and fitting to achieve efficient, topology-preserving models. The two-stage structure prevents dimensional collapse, ensures automatic dimension discovery, and achieves strong empirical results on synthetic and real datasets. By leveraging operator superposition, tensor factorization, and latent gating, IGL enables scalable, interpretable, and geometrically faithful learning on manifolds, with broad implications for manifold-based representation learning and geometric deep learning.