--- title: Adversarial Robustness Datasets url: https://www.emergentmind.com/topics/adversarial-robustness-datasets type: topic --- # Adversarial Robustness Datasets A Riemannian trust-region method is a second-order optimization framework for minimizing a smooth or semismooth objective function defined on a Riemannian manifold. It generalizes classical Euclidean trust-region strategies to constraint sets with manifold structure, combining local quadratic modeling, curvature exploitation, exact or inexact subproblem solvers, and global convergence rules. This approach provides a robust mechanism for optimization in structured non-Euclidean spaces, including low-rank matrix/tensor completion, sparse representation, nonlinear eigenproblems, and geometric statistics. ## 1. Mathematical Foundation and Algorithmic Structure Riemannian trust-region methods address the problem \[ \min_{x\in\M} f(x) \] where \(\M\) is a Riemannian manifold, \(f\colon\M\to\R\) is (typically) twice differentiable, and \(\grad f(x)\) and \(\Hess f(x)\) denote the Riemannian gradient and Hessian. At each iterate \(x_k\), one builds a local quadratic model \[ m_k(\eta) = f(x_k) + \langle \grad f(x_k), \eta\rangle + \tfrac12\langle\eta, \Hess f(x_k)[\eta]\rangle \] in the tangent space \(T_{x_k}\M\), constraining the step size to a trust-region ball \(\|\eta\|_{x_k} \le \Delta_k\). The trial step \(\eta_k\) is (approximately) the minimizer of \(m_k\) in this region. The next iterate is obtained via a chosen retraction \(x_{k+1} = R_{x_k}(\eta_k)\). The predicted/actual reduction ratio \[ \rho_k = \frac{f(x_k) - f(R_{x_k}(\eta_k))}{m_k(0) - m_k(\eta_k)} \] is used to accept or reject the candidate step and to update \(\Delta_k\). This core mechanism is present in all variants, with differences in subproblem solution, accept/reject thresholds, and regularization techniques [2309.04052, 2307.00490, 1708.02016, 1511.04777]. ## 2. Regularity, Curvature, and Convergence Guarantees Riemannian trust-region methods require only geometric regularity properties: the manifold is complete, the retraction is of at least second order, and the gradient/Hessian are Lipschitz– or Hölder–continuous, depending on the desired complexity results [2309.04052]. If the objective is only SC\(^1\) (semismooth gradient in the Clarke sense), convergence theorems still apply with suitably generalized subproblems [2307.00490]. Fundamentally, these conditions guarantee that the local quadratic (or semismooth-quadratic) model provides a sufficiently accurate approximation to the real objective in a small neighborhood. Essential results include: - **Global convergence**: Every accumulation point is a critical point, under compactness of level sets and basic curvature conditions. - **Local rates**: Quadratic convergence if the Hessian is Lipschitz and positive-definite at a minimizer; superlinear rates if subproblem solvers are inexact but controlled [2309.04052, 2307.00490, 1708.02016]. - **Worst-case bounds**: Number of iterations to reach \(\|\grad f\|\le\epsilon\) is \(O(\epsilon^{-3/2})\) under standard regularity, or \(O(\epsilon^{-(2+\alpha)/(1+\alpha)})\) with only Hölder–smoothness, where \(\alpha\) quantifies the weakest among function, retraction, and subproblem regularity [2309.04052]. For strict saddle objectives (negative curvature at non-minimizer stationary points and strong convexity near minimizers), iteration complexity improves to \(O(\log\log(1/\epsilon))\) for local criticality, relative to \(O(1/\epsilon^2)\) in the general nonconvex case [2402.07614]. ## 3. Subproblem Solvers and Practical Implementation Solving the trust-region subproblem \[ \min_{\eta\in T_{x_k}\M,\,\|\eta\|\le\Delta_k} m_k(\eta) \] is a core step. Classical choices include: - **Exact solution**: Feasible for small to moderate problem sizes. - **Truncated Conjugate Gradient (tCG)**: Efficiently detects negative curvature and reaches the trust-region boundary, often used in large-scale settings. - **Dogleg and Gauss–Newton variants**: For least-squares or approximate Hessian settings, these can accelerate convergence when second derivatives are costly or unnecessary [1709.00033, 2210.00253]. - **Negative curvature exploitation**: Detection of indefinite Hessians facilitates escape from saddle points; tCG is modeled to take such steps [1708.02016, 1511.04777]. Cubic regularization and subsampled Hessians further extend practical tractability for large-scale problems, maintaining complexity guarantees by controlling model accuracy [2302.11076]. ## 4. Geometric Modeling and Retractions The choice of retraction is crucial for global and local analysis. Standard options include exponential and projection maps, or problem-specific retractions such as ST-HOSVD for tensors [1709.00033], Cayley transforms for symplectic constraints [2404.08463], or matrix exponentials for the Stiefel/orthogonal groups [2305.09791]. Trust-region approaches flexibly accommodate constraints. For inequality- or equality-constrained settings, trust-region subproblems are embedded in augmented Lagrangian or primal-dual interior point iterations on the manifold, preserving the same convergence framework [2501.15419, 2307.00490]. ## 5. Extensions to Nonsmooth, Nonconvex, and Structured Problems Riemannian trust-region techniques have been generalized to minimize nonsmooth, nonconvex, and semismooth objectives (SC\(^1\)) [2307.00490], handle strict saddle landscapes [2402.07614], and specialized manifold structures (e.g., fixed-rank symmetric positive-definite matrices [1312.4883], symplectic Stiefel manifolds [2404.08463], low-rank tensors [1703.10019]). In nonconvex landscapes with ridable saddle points, the second-order model's curvature information enables systematic escape mechanisms—guaranteed by theory—to bypass strict saddle attractors and converge to minimizers [1511.04777, 2402.07614]. The Riemannian Levenberg–Marquardt (RLM) scheme can be seen as a trust-region-inspired Gauss–Newton method, obtaining comparable global complexity and local rates under error-bound conditions, but with unconstrained substeps and dynamic damping [2210.00253]. Augmented Lagrangian and primal-dual interior point trust-region schemes extend the applicability to composite and constrained settings, maintaining second-order stationarity and KKT convergence [2501.15419, 2307.00490]. ## 6. Complexity, Performance, and Comparative Insights Iteration complexity depends crucially on function and retraction regularity; optimal results (\(O(\epsilon^{-3/2})\)) are achievable with Lipschitz second derivatives and smooth retractions, but only suboptimal rates with weaker assumptions [2309.04052]. Inexact subproblem solvers introduce minor degradation unless their inexactness is the dominant limitation. Empirically, Riemannian trust-region methods outperform first-order methods and adaptive regularized Newton/cubic schemes, especially in attaining high-accuracy or when the objective landscape is challenging due to curvature. They show improved convergence and robustness to ill-conditioning for tensor/matrix factorization, dictionary learning, structured PCA, and molecular orbital problems [2305.09791, 1703.10019, 1709.00033, 1511.04777, 2210.00253, 2104.14957]. A distinguishing feature is their ability to guarantee global convergence in highly nonconvex, high-dimensional settings, while enabling strong local rates—often quadratic—close to the minimizer whenever curvature information is exploited at sufficient accuracy [2307.00490, 2309.04052, 1708.02016, 1511.04777]. --- ### Table: Riemannian Trust-Region Framework—Core Components | Component | Mathematical Form | Variants/Notes | |-------------------------|--------------------|--------------------------------------------------------| | Quadratic model | \(m_k(\eta)\) | Exact Hessian, Gauss–Newton, semismooth, regularized | | Trust-region subproblem | \(\|\eta\|\le\Delta\) | tCG, dogleg, eigen-step, hot-restart | | Acceptance ratio | \(\rho_k\) | Actual/predicted improvement, standard thresholds | | Retraction | \(R_{x_k}(\eta)\) | Exponential, projection, Cayley, ST-HOSVD, exp map | | Regularity assumptions | Lipschitz/Hölder | Determines complexity and rate guarantees | | Extension to constraints| ALM, primal-dual | Inequality/equality by barrier or augmented Lagrangian | ## 7. Applications and Emerging Directions Riemannian trust-region methods are now foundational in high-accuracy solution of low-rank matrix/tensor completion, dictionary recovery, canonical tensor rank approximation, quantum chemistry, geometry-aware statistics, and structure-exploiting machine learning. They support robust optimization on a diverse range of manifolds, including Stiefel, Grassmann, symmetric positive-definite, low-rank, fixed-dimension, and symplectic matrix groups [1703.10019, 1709.00033, 1511.04777, 2305.09791, 2404.08463]. Continued research is extending these methods' reach to non-smooth, nonconvex, and high-dimensional settings, refining adaptive and cubic-regularized solvers, and integrating with automatic differentiation and large-scale numerical libraries. Enhanced complexity results for strict saddle and semismooth functions, as well as advancements in constraint-handling via manifold ALM and barrier approaches, further broaden their impact [2307.00490, 2402.07614, 2501.15419]. Source: https://www.emergentmind.com/topics/adversarial-robustness-datasets