Nonsmooth Riemannian optimization with inexact manifold primitives via bundle methods

Abstract: Optimization on Hadamard manifolds -- the natural Riemannian setting for globally geodesically convex problems -- relies on exponential maps to retract tangent vectors and parallel transport to connect tangent spaces across the manifold. These primitives are often computationally expensive, leading software packages to rely on approximations: first-order retractions and vector transports. However, existing results for optimization on Hadamard manifolds either require exact primitives or lack non-asymptotic rates. We bridge this gap by introducing a proximal bundle method for nonsmooth geodesically convex optimization and establishing the first oracle-complexity bounds that rely only on subgradients and inexact primitives. We obtain sublinear rates for general objectives and optimal linear convergence under sharp function growth.

• The paper presents a proximal bundle method that achieves provable convergence for nonsmooth geodesically convex optimization on Hadamard manifolds using only subgradients and inexact primitives.
• The method attains sublinear complexity rates and matches optimal Euclidean rates in flat settings, with complexity bounds explicitly dependent on manifold curvature and approximation errors.
• Empirical results on symmetric positive definite matrices and hyperbolic total variation denoising validate the technique’s efficiency and robustness compared to traditional bundle approaches.

Nonsmooth Optimization on Hadamard Manifolds Under Inexact Manifold Primitives: A Bundle Method Framework

Introduction and Motivation

Optimization on manifolds—particularly Hadamard manifolds, which are complete, simply connected Riemannian manifolds of nonpositive sectional curvature—has gained importance for problems where global geodesic convexity is present, such as covariance matrix estimation, representation learning, and diffusion tensor imaging. However, the practical deployment of Riemannian optimization methods is often hindered by the high computational cost of exact manifold primitives: the exponential map and parallel transport. In computational packages, these are commonly replaced with first-order retractions and vector transports, but existing theoretical developments either require exact primitives or can only provide qualitative convergence guarantees without rates.

This work addresses the critical question: Can we design provably convergent algorithms for nonsmooth geodesically convex (g-convex) optimization on Hadamard manifolds that require only subgradients, first-order retractions, and vector transports? The authors propose a proximal bundle method for such problems, establishing the first oracle-complexity bounds in this setting—importantly, only requiring access to subgradients and inexact manifold primitives.

Algorithmic Construction and Theoretical Framework

The Riemannian proximal bundle method is formulated for nonsmooth optimization on a Hadamard manifold $\mathcal{M}$. The objective is to minimize a $g$-convex, possibly nonsmooth function $f:\mathcal{M}\to\mathbb{R}$. Instead of computing exact geodesics, the method employs retractions $R_x$ and (possibly non-linear) transporters $T^y_x$ that approximate parallel transport.

The core of the algorithm parallels the classical proximal bundle method, adapted to the manifold setting:

• At each iteration, subgradients are gathered at previous iterates to build local, tangent-space convex lower models of $f$.
• A proximal point is computed by minimizing a composite of this model and a quadratic regularization in the tangent space, followed by mapping with a retraction.
• Progress is evaluated by comparing actual and predicted decreases; if sufficient, a descent step is taken, otherwise a null step.

Distinctive features and challenges in the manifold setting—especially under nonzero curvature and inexact primitives—necessitate:

• Local, rather than global, minorant models due to curvature-induced transport errors and retraction approximation errors.
• Explicit bounds on the geometric inaccuracies, which propagate into the convergence rates.
• Careful control of the proximal parameter $\rho$ to maintain iterates within a neighborhood where local models are valid.

Main Results and Complexity Bounds

Convergence Rates With General $g$-Convex Objectives

The analysis establishes oracle complexity rates for finding an $\varepsilon$-minimizer. Given only subgradient oracles and inexact retraction/transport approximations (with explicit quantitative bounds), the method achieves:

• Sublinear complexity: $\mathcal{O}(\rho\cdot\varepsilon^{-3})$ for general $g$0-convex, Lipschitz objectives.
• Here, the required size of the proximal parameter $g$1 depends adaptively on the curvature and allowed inexactness of the primitives.
  • Flat settings: For zero curvature, the method recovers the optimal Euclidean rate $g$2 attained by the classical bundle method.
  • Nonzero/Negative curvature: The parameter $g$3 must increase with curvature, leading in the worst case to $g$4 and overall $g$5 complexity.

Acceleration Under H\"older Growth

When the objective exhibits $g$6-th order H\"older growth ($g$7), accelerated rates are achievable:

• Sharp growth ($g$8): Linear convergence rate is achieved.
• Intermediate regimes ($g$9): The method matches the minimax Euclidean complexity $f:\mathcal{M}\to\mathbb{R}$0.
• $f:\mathcal{M}\to\mathbb{R}$1: Suboptimal yet improved complexity $f:\mathcal{M}\to\mathbb{R}$2.

These rates are proved under a schedule for the proximal parameter that depends on both the current function gap and growth constant—realizable in idealized settings with access to such structural information.

Memory Efficiency

A further algorithmic improvement is obtained by demonstrating that the method achieves the above rates using at most three affine cuts (subgradients) per iteration. This is a substantial reduction in memory relative to previous Riemannian bundle approaches, which often require full-memory or full-history models for provable convergence.

Empirical Validation

The practical effectiveness of the method is demonstrated through two manifold optimization problems:

Riemannian Median on Symmetric Positive Definite Matrices

The problem involves finding the geometric median in $f:\mathcal{M}\to\mathbb{R}$3 endowed with the affine-invariant metric, a Hadamard manifold. The proposed method is compared with other proximal bundle and subgradient methods, both with access to the exponential map and with first-order retractions (a practical but only first-order accurate substitute).

Figure 1: Minimizing Riemannian median objective for $f:\mathcal{M}\to\mathbb{R}$4 demonstrates that the Riemannian proximal bundle method is competitive under both exact and inexact primitives. Performance is similar with first-order retraction and exponential map; using inexact primitives reduces computation time significantly.

The experiments reveal that employing inexact primitives has negligible adverse effect on oracle call complexity while substantially improving computational efficiency. All bundle methods exhibit linear convergence.

Total Variation Denoising in Hyperbolic Space

A non-Euclidean total variation denoising problem in hyperbolic space $f:\mathcal{M}\to\mathbb{R}$5 is considered, where both data fidelity and regularization terms are geodesically convex. Here, exponential maps and parallel transport are efficiently available, but illustrate broader class applicability.

Figure 2: Minimizing total variation denoising objective over signals in hyperbolic space. The method stably converges and is empirically robust to the choice of proximal parameter.

Bundle-based and subgradient methods all converge sublinearly as predicted by the nonsmooth theory. The empirical growth of the proximal parameter remains moderate compared with the theoretical worst case.

Implications and Impact

This work provides the first nonasymptotic complexity rates for nonsmooth, geodesically convex Riemannian optimization under inexact manifold primitives, with explicit dependencies on curvature and approximation error. Practically, it justifies the widespread empirical practice of using inexpensive first-order approximations in Riemannian solvers—showing that with carefully controlled algorithms, theoretical guarantees are retained.

Theoretically, it bridges a key gap in the literature: previous works necessitated either exact (often unavailable) manifold operations or could only establish asymptotic or qualitative convergence. Here, a tight link is made between geometric and algorithmic errors, curvature, and algorithmic progress, illuminating the fundamental trade-offs and their impact on performance.

Future Directions

Several promising avenues are identified for further research:

• Adaptive and parallel schedule design: Removing the need for a priori knowledge of growth constants without sacrificing acceleration, possibly via adaptive estimation schemes.
• Extension to general Riemannian and metric spaces: Generalizations beyond Hadamard manifolds, where geodesic convexity is lost and only stationarity can be certified.
• Stochastic and incremental settings: Adapting the method to settings with stochastic gradients or incremental updates for large-scale data.
• Smoothing and acceleration: Integration with smoothing or acceleration techniques to further improve rates, building on Euclidean developments.

Conclusion

The proximal bundle framework introduced in this paper delivers the first nonsmooth Riemannian optimization method with nonasymptotic complexity guarantees that relies only on subgradients and inexact, computationally tractable manifold primitives. The approach offers both strong theoretical guarantees and practical efficiency, validated empirically in significant signal processing and manifold learning problems. The insights and techniques here are poised to shape both the theoretical and algorithmic landscape of optimization on manifolds.

---

References:

Diaz, M., Grimmer, B., & McPherson, I. "Nonsmooth Riemannian optimization with inexact manifold primitives via bundle methods" (2604.27078)