Geometry-Aware Optimization

• Geometry-aware optimization is a framework that aligns algorithmic updates with intrinsic geometric structures such as manifolds and norms to improve convergence and robustness.
• It employs non-Euclidean techniques like the proximal point method and mirror descent, adapting update directions and constraints to local and global curvature for sharper convergence.
• The approach is applied in deep learning, domain adaptation, robotics, and 3D reconstruction, where careful geometric modeling enhances performance and generalization.

Geometry-aware optimization encompasses methodologies that exploit the intrinsic geometric structure—manifold, norm, or parameter space—of problems to improve convergence, accuracy, robustness, and efficiency in machine learning, signal processing, computer vision, and robotics. Rather than solving tasks in an ambient Euclidean space, geometry-aware frameworks match algorithmic updates and constraints to the curvature, topology, or anisotropy of the solution set, enabling principled handling of non-Euclidean domains, normed parameter blocks, or implicit representations.

1. Foundational Principles of Geometry-Aware Optimization

Geometry-aware optimization generalizes classical methods by selecting update directions and constraints that reflect the local and global geometric properties of the domain. Formally, given a convex objective $f:S\to \mathbb{R}\cup\{+\infty\}$ on a vector space $S$ equipped with norm $\|\cdot\|$, the non-Euclidean Broximal Point Method (BPM) iteratively minimizes $f$ over norm balls $B_{\|\cdot\|}(x, t)$, where the norm may arise from $\ell_p$, operator, Mahalanobis, or bespoke geometric constructions. This approach yields strong global convergence guarantees—including function-value contraction and nonincreasing gradient norms for differentiable $f$—but distance contraction may only be preserved when the norm is inner-product induced (e.g., Mahalanobis) (Gruntkowska et al., 1 Oct 2025).

Mirror descent, a related geometry-aware algorithm, utilizes Bregman divergences induced by distance-generating functions such as the negative entropy for simplex-constrained optimization, matching the geometry of implicit probability distributions, simplex relaxations in NAS, or other manifold-oriented spaces (Li et al., 2020).

2. Geometry-Aware Methods in Deep Learning Optimization

Geometry-aware optimizers for neural network training, including Muon, Scion, and D-Muon, separate parameters into norm groups (e.g., spectral norms for hidden matrices, $\ell_1 \to \ell_\infty$ for embedding and output layers, RMS-norms for normalization parameters). Updates are performed via norm-constrained Linear Minimization Oracles (LMOs), enabling steepest-descent steps in structured directions (e.g., sign-gradient, low-rank factorization). Recent advances address layerwise stochastic noise heterogeneity by dynamically assigning adaptive learning rates, estimated via dual-norm gradient variance, resulting in sharper convergence rates and faster sample efficiency on large transformer models (Hao et al., 15 Oct 2025). These optimizers, grounded in non-Euclidean BPM theory, rigorously exploit the geometry of the loss landscape, often outperforming Euclidean methods such as AdamW, LAMB, or block-wise learning rate schemes.

3. Geometry-Aware Optimization in Domain Alignment and Computer Vision

Explicit modeling of manifolds and tangent spaces is central to domain adaptation, 3D synthesis, and robust feature correspondence.

In domain adaptation, the GAMA framework achieves explicit manifold alignment via adversarial perturbation and tangent space exploration. Separate loss terms regularize semantic consistency along the data manifold (by projecting gradients to local tangents), robustness to off-manifold noise (orthogonal perturbations), and bidirectional geodesic manifold discrepancy between source and target domains. The result is tightened generalization bounds, reduced geodesic divergence, and superior robustness under adversarial attacks (Satou et al., 21 May 2025).

For 3D-aware image synthesis, discriminators are augmented to extract per-pixel geometry (depth, normals, albedo), supervising generators to encode accurate, consistent implicit geometry and multi-view coherence. Adding geometric heads and consistency branches stabilizes GAN training and improves mesh realism and reprojection error, without detracting from 2D image quality (Shi et al., 2022).

Feature matching for Structure from Motion (SfM) leverages geometric consistency constraints, enforcing matches that minimize the Sampson distance under estimated fundamental matrices, anchored by sparse, reliable detector-based correspondences. This hybrid optimization iteratively refines dense matches, increasing multi-view stability, correspondence density, camera pose accuracy, and point cloud density across challenging scenarios (Chen et al., 3 Sep 2024).

4. Manifold-Constrained Optimization in Signal Processing and Robotics

Several domains require optimization of functions over non-Euclidean spaces such as spheres, rotation groups ($SO(3)$), or the manifold of symmetric positive definite matrices ($SPD(n)$). Geometry-aware Bayesian optimization imposes Gaussian processes priors built from Riemannian Matérn kernels, constructed via spectral theory of the Laplace–Beltrami operator or stochastic PDEs. These kernels are adapted to manifold topology and curvature, resulting in superior convergence and regret over Euclidean-RBF or naïve geodesic kernels, particularly in high-dimensional or curved spaces (Jaquier et al., 2021).

Case studies in robotics demonstrate domain-specific benefits: orientation control exploits $SO(3)$ kernels for efficient convergence, manipulability optimization uses $SPD(2)$ kernels for trajectory smoothness and control, and hyperbolic path planning benefits from exponential volume growth. Practical guidelines recommend geometry-aware kernels in nonzero curvature or high-dimensional domains, leveraging spectral truncation for computational feasibility.

Meta-learning frameworks for RIS precoder and phase-shift optimization operate on products of complex circles ($S^1$) and real spheres, utilizing Riemannian gradients, tangent space projections, and retraction mappings. Explicit Euler-inspired updates and complex-valued neural networks ensure all iterates remain on the underlying manifold, drastically accelerating convergence and enhancing weighted sum rate and power efficiency (Devapriya et al., 17 Sep 2024).

5. Geometry-Aware Algorithms in Neural Architecture Search

NAS problems are fundamentally discrete and typically defined over graph-based architectures. By relaxing this setup to continuous optimization over products of simplices, geometry-aware mirror descent with entropic Bregman divergences yields sparse architectural parameters that collapse toward optimal subgraph selections more quickly than Euclidean gradient schemes. Convergence guarantees are provided under relative weak convexity, and empirical gains include reduced test errors and improved transferability on CIFAR and ImageNet search spaces (Li et al., 2020).

Sparse representation and rapid discretization are induced by the exponentiated-gradient updates, strongly suppressing suboptimal operation weights and facilitating efficient architecture selection.

6. Multimodal 3D Reconstruction with Geometry-Aware Splatting

In multimodal rendering, unified frameworks such as UniGS utilize 3D ellipsoidal Gaussian splats as primitives, encoding position, scale, rotation, opacity, color coefficients, semantics, and learnable gradient factors. Differentiable rasterization computes depth via analytic ray-ellipsoid intersection and propagates gradients with respect to structural attributes. Surface normals are inferred from rendered depth via finite differences and fused by orientation-aware normalization. A unified loss jointly optimizes RGB, depth, normal, semantic, and pruning outputs, linking geometric attributes with learnable parameters through exact backpropagation (Xie et al., 14 Oct 2025).

This approach ensures analytic consistency among geometry, color, and segmentation modalities, with tile-based CUDA acceleration and learnable pruning for computational and memory efficiency.

7. Summary Table: Geometry-Aware Optimization Paradigms

Optimization Domain	Geometry Model	Algorithmic Approach
Deep net training	Normed blocks/groups	Non-Euclidean BPM, LMO, Muon, LANTON
Domain adaptation	Data manifold	Tangent-space projection, geodesic alignment (GAMA)
NAS	Product of simplices	Mirror descent, exponentiated-gradient (GAEA)
Bayesian opt., robotics	Riemannian manifolds	Spectral Matérn kernels, trust-region
3D reconstruction	Ellipsoidal splats	Differentiable rasterization, analytic gradients (UniGS)

Geometry-aware optimization frameworks are increasingly central to state-of-the-art algorithms in high-dimensional modeling, classification, rendering, and control. By aligning algorithmic steps and constraints to the underlying geometric structure—whether through non-Euclidean norms, manifold-constrained gradients, or spectral kernels—these methods address key challenges in expressivity, convergence, and generalization across diverse scientific and engineering domains.