Self-Supervised Physical Invariants

• Self-supervised physical invariant extraction is a representation learning approach that computes features robust to transformations such as rotation, shift, and lighting changes.
• Techniques leverage architectural designs, tailored loss functions, and data augmentation strategies to enforce invariance and enhance downstream tasks in vision, robotics, and multimodal sensing.
• These methods enable reliable feature learning from unlabeled data, facilitating robust performance in object recognition, sensor fusion, and transfer learning while addressing scalability and task-specific challenges.

Self-supervised physical invariant extraction comprises a family of representation learning techniques designed to yield features or latent codes that are insensitive—to a prescribed degree—to physically meaningful transformations of the input. Examples include spatial shifts, rotations, lighting changes, multimodal correspondence, and sensor configuration variability. By exploiting architectural constraints, loss objectives, and augmentation strategies, these methods bypass the need for labeled data, producing representations robust to nuisance factors but faithful to physical invariants such as object identity, pose, or state. This paradigm underpins current advances in visual, sensory, and multimodal reasoning systems across computer vision, robotics, wireless sensing, and scientific domains.

1. Formal Definitions and Taxonomy of Invariants

Physical invariance in learned representations is formalized as the requirement that certain group actions or transformations on the input $x\in X$ leave the output of an encoder $f$ unchanged (invariant), or transform it in a predictable manner (equivariant). Let $G$ be a group of transformations $\rho_X(g)\cdot x$. Invariant and equivariant mappings are defined as:

• Invariant representation: $$f_{\mathrm{inv}}(\rho_X(g)\cdot x) = f_{\mathrm{inv}}(x),\quad \forall g\in G,\,x\in X$$.
• Equivariant representation: $$f_{\mathrm{equi}}(\rho_X(g)\cdot x) = \rho_Y(g)\left[ f_{\mathrm{equi}}(x) \right]$$ for some linear action $\rho_Y(g)$ on the latent space $V_{\mathrm{equi}}$ (Garrido et al., 2023).

Typical transformation groups $G$ include spatial shifts ($\mathbb{Z}^2$), rotations ($SO(3)$), scaling ($\mathbb{R}^+$), lighting/color change ($\mathbb{R}$), multimodal alignment, or channel-space deformations in wireless sensing.

In complex domains, "split" techniques separate latent spaces into invariant and equivariant components, supporting richer downstream reasoning (Garrido et al., 2023).

2. Architectures and Learning Objectives Enforcing Invariance

A diverse set of architectures implement self-supervised invariant extraction across sensory domains:

• Transform Invariant Auto-encoder (TIAE): Encoders produce a canonical descriptor $z$ for all transforms $T_\theta(I)$ within the group, enforced by a loss penalizing decoder output differences $L_{\mathrm{inv}}$ across transforms, and optionally inferring $T^{-1}_\theta$ via a secondary regressor (Matsuo et al., 2017).
• RI-MAE (Rotation-Invariant Masked AutoEncoder): PCA-based local patch decomposition yields content tokens and rotation/position embeddings. Self-attention with rotation-invariant orientation/position bias constructs a latent space stable under arbitrary $SO(3)$ rotations (Su et al., 2024).
• RIPT + SDMM (Rotation-Invariant Point-set Token Transformer with Self-Distillation): Global-scale tokenization aligns local frames and aggregates them in a fully SO(3)-invariant manner, optimized via multi-crop and cut-mix augmentations plus self-distillation (Furuya et al., 2023).
• Split Invariant-Equivariant (SIE): A joint-embedding backbone splits representations into invariant and equivariant branches, regularized via contrastive losses and a hypernetwork predictor parameterized by explicit group elements (Garrido et al., 2023).
• Wireless Transformer (SWiT): A BYOL-style two-branch transformer, invariant to gain, fading, flipping, sign, and subcarrier permutations, optimized via macro (global) and micro (local) cross-entropy objectives on augmented channel estimates (Salihu et al., 2023).
• Predictive Coding (PreludeNet): Hierarchical ConvLSTM encoders minimize future-frame prediction error (self-supervised), while a supervised decoder extracts depth (physical invariant) robust to lighting via parallel skip-connections (Ziskind et al., 2022).
• Self-Organizing Maps with Hebbian Cross-links: Unsupervised SOM arrays with pairwise Hebbian links learn nonlinear physical relations (e.g., intensity-gradient-flow) in multimodal sensory streams, without explicit global supervision (Xiaorui et al., 2020).

Loss functions typically combine invariance-promoting terms (e.g., $$L_{\mathrm{inv}} = \sum \| D(E(I)) - D(E(T_{\theta}(I)))\|^2$$) with reconstructive, contrastive, and regularization components.

3. Data Augmentation Strategies and Group Actions

Augmentation is central to driven acquisition of invariances in self-supervised learning:

Domain	Augmentations / Group Actions	Papers
Visual	Shift, rotation, scale, lighting, viewpoint	(Matsuo et al., 2017, Biscione et al., 2021, Ziskind et al., 2022)
3D Point Clouds	Rotation ($SO(3)$), crop/mix/scale	(Su et al., 2024, Furuya et al., 2023)
Sensor/Channel	Gain, fading, subcarrier permutation, sign flip	(Salihu et al., 2023)
Multimodal	Time, sensor dropout, amplitude variations	(Xiaorui et al., 2020)

Each method samples or enumerates a family of transformations $T_{\theta}$ per mini-batch to enforce invariance, either by matching embeddings (contrastive/instance matching) or reconstructing canonical representations (masked/decoder-based objectives).

For SIE (Garrido et al., 2023), explicit group labels are leveraged to inform the equivariant split; for others, group elements are either inferred or sampled.

4. Quantitative Evaluation and Benchmarks

State-of-the-art invariance-centric SSL models are validated by:

• Object-centric retrieval/classification: MacroMAP, SVM accuracy, NMI under transformation (Furuya et al., 2023).
• Transfer robustness: Rotation accuracy, few-shot learning (ModelNet40, ScanObjectNN), segmentation (ShapeNetPart, S3DIS). RI-MAE improves classification under arbitrary rotations to >90%, exceeding prior masked modeling methods (Su et al., 2024).
• Physical variable inference: Depth estimation metrics—Abs Rel, RMSE, $\delta<1.25^k$—remains stable under extreme lighting/shadowing in predictive coding (Ziskind et al., 2022).
• Wireless localization: RMSE on coarse and fine subregion spot classification, transfer across environments (Salihu et al., 2023).
• Multimodal relation regression: Reconstruction and inference errors (RMSE) for physically constrained quantities (e.g., optical flow, intensity-gradient mapping) typically fall below 5–10% even under noise and missing input (Xiaorui et al., 2020).
• Representation invariance metrics: Adjusted true invariance $\tilde I_t(\theta)$, 5AFC accuracy, cosine similarity on held-out transformation grids (Biscione et al., 2021).

Ablations confirm that architecturally enforced invariance (PCA-aligned tokens, LRF normalization) and targeted augmentations (multi-crop, cut-mix, neighborhood sampling) yield substantial performance gains under transformations. Notably, self-supervised models often exceed supervised counterparts when labels are scarce or test-time nuisances are substantial (Salihu et al., 2023).

5. Application Domains and Extensions

Physical invariant extraction via SSL is foundational across several application domains:

• 3D vision and robotics: Canonical shape/pose representations support SLAM, object detection, segmentation, robot imitation, and grasping (Matsuo et al., 2017, Su et al., 2024, Furuya et al., 2023, Garrido et al., 2023).
• Low-shot or transfer learning: Invariant features enable robust adaptation to new environments or tasks with minimal supervision (Salihu et al., 2023).
• Sensor fusion / multimodal reasoning: Hebbian-linked SOMs extract invariant relations for multisensory integration, e.g. visual flow, audio, inertial measurement (Xiaorui et al., 2020).
• Vision under environmental variation: Predictive coding encoders decouple geometry from appearance factors, crucial for autonomous navigation in dynamic lighting (Ziskind et al., 2022).
• Wireless channel mapping: Invariant descriptors cluster positions and system states even under severe equipment/signal variability (Salihu et al., 2023).
• Cognitive modeling: Same/different SSL over synthetic visual objects demonstrates emergence and transfer of invariances analogous to human perceptual continuity (Biscione et al., 2021).

Methods such as SIE (Garrido et al., 2023) suggest direct extensions to learning arbitrary physical/hard transformations, including articulations and nonlinear deformations, and can be adapted to optical, radar, tactile, and temporal domains.

6. Limitations, Open Challenges, and Future Directions

While self-supervised invariant extraction outperforms alternative approaches in many regimes, key limitations remain:

• Explicit group element requirement: Some frameworks (e.g., SIE) need ground-truth transformation labels $g$ at training time (Garrido et al., 2023).
• Scalability to higher-order and composite transformations: Most implementations focus on pairwise or group-manifold actions; complex physics may require tensor-Hebbian extensions or hierarchical stacking (Xiaorui et al., 2020).
• Architecture sensitivity: Selection of patch-token scale, number of splits, layer depth, and attention type substantially affects invariance properties and downstream generalization (Furuya et al., 2023, Su et al., 2024).
• Task-specific down-weighting/optimization: For some applications, over-enforcing invariance may suppress task-relevant variation (e.g., pose estimation vs. identity classification) (Li et al., 2022).

Possible directions include inference of group parameters from raw data, multi-modal scaling, extension to nonlinear or temporally compositional invariances, adaptive optimization schedules balancing invariance/equivariance, and use in semantic, physical, and generative modeling across wide scientific applications.

7. Comparative Summary of Methods

Approach	Domain	Invariance Enforced	Objective	Representative Results
TIAE (Matsuo et al., 2017)	2D vision	Shift/rotation/scale	Reconstruction + inv	Canonical templates, high clustering
RI-MAE (Su et al., 2024)	3D point clouds	SO(3) rotation	Masked SSL in latent	91.6% accuracy ScanObjectNN, SOTA mIoU
RIPT+SDMM (Furuya et al., 2023)	3D point clouds	SO(3) rotation	Tokenization, distill	2× gain macroMAP, 83% MN40 accuracy
SIE (Garrido et al., 2023)	Vision/3D	Split inv/equi (pose/color)	Split contrast + hyper	0.73 R² pose prediction, .07 color R²
SWiT (Salihu et al., 2023)	Wireless	Fading, gain, flipping	Global+local SSL	459 mm RMSE with 10k labels, 99.9% Top-1
PreludeNet (Ziskind et al., 2022)	Vision	Lighting (illumination/shadow)	Hybrid pred. coding	Depth RMSE stable under lighting change
SOM-Hebbian (Xiaorui et al., 2020)	Multi-sensory	Nonlinear physical (e.g. gradient/flow)	Pairwise unsupervised	<5% RMSE mapping, robust missing input

These methods operationalize self-supervised physical invariant extraction as a principled pipeline combining data-driven augmentations, explicit architectural biases, and carefully constructed loss functions, yielding representations with provable invariance to broad classes of physical transforms and enabling robust transferable reasoning in high-dimensional domains.