Trust-SSL: Additive-Residual Selective Invariance for Robust Aerial Self-Supervised Learning

Abstract: Self-supervised learning (SSL) is a standard approach for representation learning in aerial imagery. Existing methods enforce invariance between augmented views, which works well when augmentations preserve semantic content. However, aerial images are frequently degraded by haze, motion blur, rain, and occlusion that remove critical evidence. Enforcing alignment between a clean and a severely degraded view can introduce spurious structure into the latent space. This study proposes a training strategy and architectural modification to enhance SSL robustness to such corruptions. It introduces a per-sample, per-factor trust weight into the alignment objective, combined with the base contrastive loss as an additive residual. A stop-gradient is applied to the trust weight instead of a multiplicative gate. While a multiplicative gate is a natural choice, experiments show it impairs the backbone, whereas our additive-residual approach improves it. Using a 200-epoch protocol on a 210,000-image corpus, the method achieves the highest mean linear-probe accuracy among six backbones on EuroSAT, AID, and NWPU-RESISC45 (90.20% compared to 88.46% for SimCLR and 89.82% for VICReg). It yields the largest improvements under severe information-erasing corruptions on EuroSAT (+19.9 points on haze at s=5 over SimCLR). The method also demonstrates consistent gains of +1 to +3 points in Mahalanobis AUROC on a zero-shot cross-domain stress test using BDD100K weather splits. Two ablations (scalar uncertainty and cosine gate) indicate the additive-residual formulation is the primary source of these improvements. An evidential variant using Dempster-Shafer fusion introduces interpretable signals of conflict and ignorance. These findings offer a concrete design principle for uncertainty-aware SSL. Code is publicly available at https://github.com/WadiiBoulila/trust-ssl.

• The paper introduces an additive-residual selective invariance mechanism that fuses evidential uncertainty signals to enhance aerial SSL robustness.
• It demonstrates significant improvements in mean linear-probe accuracy and corruption resilience across benchmarks like EuroSAT, AID, and NWPU-RESISC45.
• The method preserves backbone gradients via stop-gradient operations, enabling effective cross-domain transfer and improved out-of-distribution detection.

Trust-SSL: Additive-Residual Selective Invariance for Robust Aerial SSL

Motivation and Problem Formalization

Self-supervised learning (SSL) methods for visual representations—specifically, contrastive methods like SimCLR, BYOL, and VICReg—rely on enforcing invariance between augmented views of the same image. This works well when augmentations preserve semantic content. However, aerial and satellite imagery routinely suffers from severe degradations (e.g., haze, blur, occlusion), which may remove critical evidence. Standard SSL objectives that treat a clean and a highly corrupted view as equivalent can imprint spurious equivalence in the representation, ultimately reducing the robustness and accuracy of downstream classifiers.

The "Trust-SSL: Additive-Residual Selective Invariance for Robust Aerial Self-Supervised Learning" (2604.21349) approach targets this limitation by introducing a mechanism to embed fine-grained, per-sample, per-factor trust signals into the SSL objective—crucially, in a way that preserves the backbone's learning signal and is robust against miscalibrated uncertainty estimates early in training.

Architectural Innovations

The Trust-SSL model introduces two central architectural components:

1. Factorized Evidential Representation: The backbone output is partitioned into several augmentation-anchored subspaces (factors), each targeted by a separate projection head.
2. Evidential Uncertainty and Additive-Residual Alignment: Each factor's embedding is processed through evidential heads to produce conflict ($K$) and ignorance ($I$) scores per factor, using Dempster-Shafer theory as the uncertainty quantification framework. These views are fused to obtain a trust weight $w^t$ for each factor.

The training objective integrates the trust-weighted alignment via an additive-residual with stop-gradient, instead of the naive multiplicative formulation.

Figure 1: Overview of the Trust-SSL architecture with the essential components for evidential trust computation and the additive-residual loss composition.

This design preserves the full SimCLR gradient through the backbone while adding a trust-aware corrective signal, decoupled from the evidential head's backpropagation path, thereby avoiding the undesirable gradient attenuation and instability observed with multiplicative gates when uncertainty heads are not yet calibrated.

Objective Function and Training Dynamics

The Trust-SSL loss is:

$$\mathcal{L} = \mathcal{L}_{\text{SimCLR}} + \lambda_{\text{sel}} \frac{1}{T} \sum_{t=1}^T \mathrm{sg}(w^t) \cdot (1-\cos(z_1^t, z_2^t)) + \text{auxiliary/regularizer terms}$$

with $\mathrm{sg}(\cdot)$ denoting stop-gradient. The selective residual ramps in only after the base contrastive loss has shaped the representation.

Training is staged: the additive-residual term is annealed in after initial backbone pretraining, as visualized in the training dynamics.

Figure 3: Training dynamics of Trust-SSL show clear phases of loss convergence, evidential metric evolution, and activation of the selective-invariance regime.

Empirical ablation reveals that switching from the multiplicative to the additive-residual trust mechanism is the main source of the strong performance gains: the multiplicative form severely degrades backbone representations, while the additive-residual implementation yields consistent improvements.

Clean-Condition and Corruption-Robustness Evaluation

Trust-SSL achieves superior mean linear-probe accuracy ($90.20\%$) over three aerial scene benchmarks (EuroSAT, AID, NWPU-RESISC45), outperforming SimCLR and VICReg under identical protocols, and especially excelling on AID and NWPU. Notably, the gains are robust across ablation variants—scalar and cosine trust gates achieve similar accuracy, underlining that the additive-residual construction is the primary driver of performance.

Corruption robustness is a central focus. Trust-SSL outperforms all baselines on severe information-erasing corruptions (e.g., haze, Gaussian/motion blur, occlusion) especially on EuroSAT, with gains as large as $+19.9$ points over SimCLR under severe haze (severity 5). For contradiction-type corruptions, covariance-regularized methods such as VICReg remain highly competitive, especially on larger, more diverse datasets.

Figure 4: Robustness heatmap across methods and datasets: Trust-SSL's distinctive strength is in resisting erasure corruptions on EuroSAT.

Evidential K–I Mechanism Analysis

The paper provides a thorough analysis of the evidential K (conflict) and I (ignorance) trajectories under controlled corruption sweeps. Contradiction-family corruptions yield monotonic increases in conflict with severity, confirming the theoretical expectation. For erasure corruptions, a nuanced behavior is observed: instead of higher ignorance, the auxiliary classifier encourages the model to confidently infer a "corrupted" prototype, slightly suppressing the classical ignorance rise. This interaction suggests future decoupling/ablations to disentangle the effect.

Figure 5: Trajectories for conflict ($\bar K$) and ignorance ($\bar I$) across increasing corruption: predictable monotonic response for contradiction, nuanced effects for erasure.

Cross-Domain Stress Testing and OOD Sensitivity

Zero-shot cross-domain transfer is validated on the BDD100K driving-scene dataset: Mahalanobis-based AUROC for OOD detection is consistently improved by all additive-residual variants over non-selective baselines, with the largest performance increases on the hardest shift (snow). Additionally, the Dempster-Shafer-based $K+I$ metric yields a native, interpretable OOD score, though with lower absolute AUROC than Mahalanobis.

Figure 6: Zero-shot AUROC on BDD100K weather splits: all additive-residual selective-invariance models occupy the top tier.

Ablation Findings and Theoretical Implications

Extensive ablations demonstrate:

• The additive-residual stop-gradient mechanism is the principal factor for accuracy and corruption robustness improvements; the precise functional form of the trust gate is less critical.
• Factorization and full evidential theory introduce interpretability (via K and I) that is unavailable in non-evidential or simple cosine-gate variants, providing gated explanations and OOD scoring.
• The auxiliary corruption-family classifier is a double-edged sword—improving backbone robustness but partially interfering with classical subjective logic behavior in the evidential heads.

Broader Implications and Future Directions

The study establishes a methodological design principle: to incorporate learned trust or uncertainty signals into SSL, use additive-residual corrections with stop-gradient to preserve base learning dynamics. This principle extends to settings beyond aerial imagery, including multimodal, masked modeling, and general SSL regimes where alignment reliability is uncertain.

The results delineate a clear regime of practical scope for Trust-SSL:

• Most effective under information-erasure corruptions and moderate dataset scale.
• Outperformed by covariance-regularization methods on major contradiction-type corruptions or larger dataset complexity.

Potential research directions include: integrating masked image modeling under identical protocols, decoupling evidential heads from auxiliary tasks to recover classical ignorance dynamics, and generalizing the design to video and multimodal SSL.

Conclusion

Trust-SSL advances uncertainty-aware self-supervised representation learning for robust aerial imagery, demonstrating that the structural form in which uncertainty signals are embedded in the loss is as consequential as the quality of the signals themselves. The additive-residual, stop-gradient selective-invariance approach provides strong performance on aerial benchmarks, excelling under evidence-erasing corruptions and delivering improved cross-domain transfer. The evidential variant further enhances interpretability and OOD transparency. These design insights are likely transferable to a wide suite of SSL applications in machine perception.