Adversarial Robustness Datasets

• Adversarial Robustness Datasets are curated collections designed to evaluate model sensitivity to crafted adversarial perturbations.
• They provide practical benchmarks for comparing the efficacy of defense mechanisms and adversarial training techniques.
• Researchers leverage these datasets to identify vulnerabilities and improve robustness, supporting real-world security and deployment challenges.

A Riemannian trust-region method is a second-order optimization framework for minimizing a smooth or semismooth objective function defined on a Riemannian manifold. It generalizes classical Euclidean trust-region strategies to constraint sets with manifold structure, combining local quadratic modeling, curvature exploitation, exact or inexact subproblem solvers, and global convergence rules. This approach provides a robust mechanism for optimization in structured non-Euclidean spaces, including low-rank matrix/tensor completion, sparse representation, nonlinear eigenproblems, and geometric statistics.

1. Mathematical Foundation and Algorithmic Structure

Riemannian trust-region methods address the problem

$\min_{x\in\M} f(x)$

where $\M$ is a Riemannian manifold, $f\colon\M\to\R$ is (typically) twice differentiable, and $\grad f(x)$ and $\Hess f(x)$ denote the Riemannian gradient and Hessian.

At each iterate $x_k$, one builds a local quadratic model

$m_k(\eta) = f(x_k) + \langle \grad f(x_k), \eta\rangle + \tfrac12\langle\eta, \Hess f(x_k)[\eta]\rangle$

in the tangent space $T_{x_k}\M$, constraining the step size to a trust-region ball $\|\eta\|_{x_k} \le \Delta_k$. The trial step $\eta_k$ is (approximately) the minimizer of $\M$0 in this region. The next iterate is obtained via a chosen retraction $\M$1.

The predicted/actual reduction ratio

$\M$2

is used to accept or reject the candidate step and to update $\M$3. This core mechanism is present in all variants, with differences in subproblem solution, accept/reject thresholds, and regularization techniques (Zhang et al., 2023, Zhang et al., 2023, Hu et al., 2017, Sun et al., 2015).

2. Regularity, Curvature, and Convergence Guarantees

Riemannian trust-region methods require only geometric regularity properties: the manifold is complete, the retraction is of at least second order, and the gradient/Hessian are Lipschitz– or Hölder–continuous, depending on the desired complexity results (Zhang et al., 2023). If the objective is only SC$\M$4 (semismooth gradient in the Clarke sense), convergence theorems still apply with suitably generalized subproblems (Zhang et al., 2023). Fundamentally, these conditions guarantee that the local quadratic (or semismooth-quadratic) model provides a sufficiently accurate approximation to the real objective in a small neighborhood.

Essential results include:

• Global convergence: Every accumulation point is a critical point, under compactness of level sets and basic curvature conditions.
• Local rates: Quadratic convergence if the Hessian is Lipschitz and positive-definite at a minimizer; superlinear rates if subproblem solvers are inexact but controlled (Zhang et al., 2023, Zhang et al., 2023, Hu et al., 2017).
• Worst-case bounds: Number of iterations to reach $\M$5 is $\M$6 under standard regularity, or $\M$7 with only Hölder–smoothness, where $\M$8 quantifies the weakest among function, retraction, and subproblem regularity (Zhang et al., 2023).

For strict saddle objectives (negative curvature at non-minimizer stationary points and strong convexity near minimizers), iteration complexity improves to $\M$9 for local criticality, relative to $f\colon\M\to\R$0 in the general nonconvex case (Goyens et al., 2024).

3. Subproblem Solvers and Practical Implementation

Solving the trust-region subproblem

$f\colon\M\to\R$1

is a core step. Classical choices include:

• Exact solution: Feasible for small to moderate problem sizes.
• Truncated Conjugate Gradient (tCG): Efficiently detects negative curvature and reaches the trust-region boundary, often used in large-scale settings.
• Dogleg and Gauss–Newton variants: For least-squares or approximate Hessian settings, these can accelerate convergence when second derivatives are costly or unnecessary (Breiding et al., 2017, Adachi et al., 2022).
• Negative curvature exploitation: Detection of indefinite Hessians facilitates escape from saddle points; tCG is modeled to take such steps (Hu et al., 2017, Sun et al., 2015).

Cubic regularization and subsampled Hessians further extend practical tractability for large-scale problems, maintaining complexity guarantees by controlling model accuracy (Deng et al., 2023).

4. Geometric Modeling and Retractions

The choice of retraction is crucial for global and local analysis. Standard options include exponential and projection maps, or problem-specific retractions such as ST-HOSVD for tensors (Breiding et al., 2017), Cayley transforms for symplectic constraints (Jensen et al., 2024), or matrix exponentials for the Stiefel/orthogonal groups (Sepehri et al., 2023).

Trust-region approaches flexibly accommodate constraints. For inequality- or equality-constrained settings, trust-region subproblems are embedded in augmented Lagrangian or primal-dual interior point iterations on the manifold, preserving the same convergence framework (Obara et al., 26 Jan 2025, Zhang et al., 2023).

5. Extensions to Nonsmooth, Nonconvex, and Structured Problems

Riemannian trust-region techniques have been generalized to minimize nonsmooth, nonconvex, and semismooth objectives (SC$f\colon\M\to\R$2) (Zhang et al., 2023), handle strict saddle landscapes (Goyens et al., 2024), and specialized manifold structures (e.g., fixed-rank symmetric positive-definite matrices (Mishra et al., 2013), symplectic Stiefel manifolds (Jensen et al., 2024), low-rank tensors (Heidel et al., 2017)).

In nonconvex landscapes with ridable saddle points, the second-order model's curvature information enables systematic escape mechanisms—guaranteed by theory—to bypass strict saddle attractors and converge to minimizers (Sun et al., 2015, Goyens et al., 2024).

The Riemannian Levenberg–Marquardt (RLM) scheme can be seen as a trust-region-inspired Gauss–Newton method, obtaining comparable global complexity and local rates under error-bound conditions, but with unconstrained substeps and dynamic damping (Adachi et al., 2022).

Augmented Lagrangian and primal-dual interior point trust-region schemes extend the applicability to composite and constrained settings, maintaining second-order stationarity and KKT convergence (Obara et al., 26 Jan 2025, Zhang et al., 2023).

6. Complexity, Performance, and Comparative Insights

Iteration complexity depends crucially on function and retraction regularity; optimal results ($f\colon\M\to\R$3) are achievable with Lipschitz second derivatives and smooth retractions, but only suboptimal rates with weaker assumptions (Zhang et al., 2023). Inexact subproblem solvers introduce minor degradation unless their inexactness is the dominant limitation.

Empirically, Riemannian trust-region methods outperform first-order methods and adaptive regularized Newton/cubic schemes, especially in attaining high-accuracy or when the objective landscape is challenging due to curvature. They show improved convergence and robustness to ill-conditioning for tensor/matrix factorization, dictionary learning, structured PCA, and molecular orbital problems (Sepehri et al., 2023, Heidel et al., 2017, Breiding et al., 2017, Sun et al., 2015, Adachi et al., 2022, Sembach et al., 2021).

A distinguishing feature is their ability to guarantee global convergence in highly nonconvex, high-dimensional settings, while enabling strong local rates—often quadratic—close to the minimizer whenever curvature information is exploited at sufficient accuracy (Zhang et al., 2023, Zhang et al., 2023, Hu et al., 2017, Sun et al., 2015).

---

Table: Riemannian Trust-Region Framework—Core Components

Component	Mathematical Form	Variants/Notes
Quadratic model	$f\colon\M\to\R$4	Exact Hessian, Gauss–Newton, semismooth, regularized
Trust-region subproblem	$f\colon\M\to\R$5	tCG, dogleg, eigen-step, hot-restart
Acceptance ratio	$f\colon\M\to\R$6	Actual/predicted improvement, standard thresholds
Retraction	$f\colon\M\to\R$7	Exponential, projection, Cayley, ST-HOSVD, exp map
Regularity assumptions	Lipschitz/Hölder	Determines complexity and rate guarantees
Extension to constraints	ALM, primal-dual	Inequality/equality by barrier or augmented Lagrangian

7. Applications and Emerging Directions

Riemannian trust-region methods are now foundational in high-accuracy solution of low-rank matrix/tensor completion, dictionary recovery, canonical tensor rank approximation, quantum chemistry, geometry-aware statistics, and structure-exploiting machine learning. They support robust optimization on a diverse range of manifolds, including Stiefel, Grassmann, symmetric positive-definite, low-rank, fixed-dimension, and symplectic matrix groups (Heidel et al., 2017, Breiding et al., 2017, Sun et al., 2015, Sepehri et al., 2023, Jensen et al., 2024).

Continued research is extending these methods' reach to non-smooth, nonconvex, and high-dimensional settings, refining adaptive and cubic-regularized solvers, and integrating with automatic differentiation and large-scale numerical libraries. Enhanced complexity results for strict saddle and semismooth functions, as well as advancements in constraint-handling via manifold ALM and barrier approaches, further broaden their impact (Zhang et al., 2023, Goyens et al., 2024, Obara et al., 26 Jan 2025).