Invariance Proximity

• Invariance Proximity is a metric that quantifies how close a function space, model, or graph is to being invariant under a specified transformation.
• It provides precise error bounds and interpretable guarantees through methods like principal angle analysis and proximal optimization.
• Its applications span Koopman operator theory, computer vision, graph theory, and sparse recovery, driving robust model design and performance.

Invariance proximity quantifies how nearly a mathematical object (e.g., a function space, dictionary, or model family) is invariant under a specified transformation, operator, or group action. This concept pervades multiple fields—Koopman operator theory, control and dynamical systems, proximal optimization, computer vision invariance, and combinatorial graph theory—enabling rigorous assessment of subspace or model suitability, error bounding, and invariance testing. The term admits formal, typically subspace-based, definitions yielding computable certificates of “almost-invariance” and often leads to interpretable, tight error guarantees.

1. Formal Definitions Across Domains

The core idea of invariance proximity is to provide a scalar or metric certificate of how close a subspace, feature space, or model family is to true invariance under a map or set of transformations. Key formalizations include:

• Koopman Operator Setting:

Given an inner-product space of observables $F$ and a Koopman operator $K: F \to F$, for a finite-dimensional subspace $S \subset F$, the invariance proximity is

$$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$

where $P_S$ is the orthogonal projector onto $S$ (Haseli et al., 2023, Haseli et al., 2023). In the presence of control/input variables, the definition generalizes with respect to the lifted operator $\tilde{\mathcal K}$ and the augmented state-input space.

• Proximal Optimization of Invariant Functions:

For $f: \mathbb{R}^n \to \mathbb{R}\cup\{+\infty\}$ scale- and signed-permutation invariant, the associated “proximity operator” evaluates

$$\operatorname{prox}_{\lambda f}(x) = \arg\min_{u} \frac{1}{2\lambda}\|u - x\|_2^2 + f(u),$$

with the invariance reducing the non-smooth minimization to a low-dimensional, symmetrized, and often closed-form procedure (Jia et al., 2024).

• Semantic Proximity in Invariance Testing:

In background-invariance testing for ML models, semantic proximity between objects is defined as the minimal distance between keyword sets (extracted from images) in a directed, weighted ontology graph, with semantic distance

$$d_{\text{sem}}(x, b) = \min_{u \in K_x,\, v \in K_b} d_G(u \rightarrow v)$$

(Liao et al., 2022).

• Distance Invariants in Graphs:

For a simple, connected graph $K: F \to F$0, the proximity $K: F \to F$1 arises as a minimum normalized transmission, measuring the least average geodesic distance from any node to all others, and relates dually to “remoteness” (Sedlar, 2012).

2. Theoretical Properties and Error Control

Invariance proximity as formulated in Koopman-related settings controls worst-case prediction error for finite-dimensional model approximations:

• For $K: F \to F$2 (or $K: F \to F$3) and $K: F \to F$4, the approximation error for any $K: F \to F$5 after projection satisfies

$K: F \to F$6

This makes $K: F \to F$7 a tight, a priori upper bound on the relative one-step prediction error, independent of basis choice or coordinate scaling (Haseli et al., 2023, Haseli et al., 2023).

• In applications to graph invariants, inequalities such as

$K: F \to F$8

position proximity as the dual of remoteness and connect it to notions of eccentricity, diameter, and radius, with corresponding extremal results and characterizations (Sedlar, 2012).

• In background invariance testing, semantic proximity underlies quantitative orderings used for systematic robustness assessment (Liao et al., 2022).

3. Computational Techniques and Closed-Form Formulas

Recent advancements provide efficient computational strategies and closed-form results for invariance proximity:

• Koopman Subspaces:

$K: F \to F$9 admits a closed-form in terms of the maximal principal angle between $S \subset F$0 and $S \subset F$1:

$S \subset F$2

where $S \subset F$3 is the largest principal angle between $S \subset F$4 and $S \subset F$5. When $S \subset F$6 are the orthonormal bases for these subspaces, $S \subset F$7 is given by the maximal $S \subset F$8 over singular values $S \subset F$9 of $$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$0. All computations reduce to SVD and QR on finite-dimensional matrices—no function-space integrals required (Haseli et al., 2023).

• Proximal Algorithms for Invariant Functions:

Invariance enables the reduction of proximity operator computation to three-step WRD (w-step, r-step, d-step) procedures, sometimes yielding closed-form global minimizers (as in $$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$1 penalties) and highly efficient algorithms for use in sparsity-promoting contexts (Jia et al., 2024).

• Semantic Proximity for ML Testing:

The construction of semantic proximity employs association mining and graph traversals on ontologies derived from large-scale object recognition, enabling tractable ordering and sampling for model evaluation (Liao et al., 2022).

4. Applications in Modeling, Control, and Robust ML

• Koopman-Based Model Reduction:

In dynamical system identification and control, invariance proximity is used as an objective for dictionary or subspace selection. Minimizing $$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$2 on data (via EDMD-type formulas) drives down the worst-case model error, resulting in robust, basis-free guarantees for nonlinear and control-augmented lifting strategies (Haseli et al., 2023, Haseli et al., 2023).

• Computer Vision and Invariant Similarity:

Latent-optimization-based proximity measures—such as the “transforming distance” with invariance regularization—not only improve instance-based recognition accuracy over conventional metrics, but also yield strong empirical robustness to sparsity and attention shifts, closely paralleling human “mental rotation” mechanisms (Ding et al., 2014).

• Robustness Testing for ML Classifiers:

Ordered background sampling via semantic proximity supports the construction and visualization of variance matrices, enabling systematic audit and automated assessment (via e.g. random forest–based ML4ML assessors) of invariance to semantic scene changes (Liao et al., 2022).

• Sparse Recovery and Optimization:

Proximity operators for scale- and signed-permutation invariant penalties underlie efficient algorithms for nonconvex, nonsmooth optimization central in sparse signal recovery, with invariance properties enabling tractable routines (Jia et al., 2024).

5. Comparative Significance and Theoretical Foundations

• Contrast with Standard Notions:

Whereas naive measures of invariance simply evaluate function or subspace closure, invariance proximity provides a continuous, quantitative, and computationally accessible measure with a geometric or statistical interpretation (principal angle, semantic graph distance, etc.) and basis independence (Haseli et al., 2023, Haseli et al., 2023).

• Extremal/Invariant Structures:

In combinatorial settings, proximity gives rise to sharp extremal theorems (e.g., three-armed trees maximizing $$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$3) and tightly characterizes the spatial layout of optimal invariants in graphs (Sedlar, 2012).

• Functorial and Categorical Generality:

At the large scale, the “proximity at infinity” construction encapsulates invariance of proximity structure under coarse equivalence (quasi-isometry), yielding an isomorphism invariant of the underlying metric space, with direct analogs in geometric group theory and topology (Grzegrzolka et al., 2018).

6. Illustrative Examples

Domain	Invariance Proximity Definition	Computation/Impact
Koopman operator subspaces	$$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$4	Closed-form via principal angles, SVD
Sparse recovery/operators	$$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$5 with $$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$6 invariant	Reduced to low-dim optimization, explicit WRD
Semantic ML testing	$$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$7 as minimal path in semantic ontology graph	Enables ordered sampling and automated testing
Graph invariants (proximity)	$$I_K(S) = \sup_{f \in S,\, \|Kf\| \neq 0} \frac{\|Kf - P_S Kf\|}{\|Kf\|},$$8	Extremal characterization, centroid theory

In all settings, the metric of invariance proximity enables principled subspace or model design, permits explicit error control, and reveals intrinsic geometric or combinatorial features not available through ad hoc or purely qualitative invariance measures.