Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning

Abstract: Most self-supervised learning (SSL) methods encourage invariance across augmentations, but strict flip invariance can suppress informative left--right correspondences in approximately bilateral data such as medical images and human faces. We propose Mirror-Fusion-Augmented Self-Supervised Learning (MFASSL), a Vision Transformer framework that injects a soft reflection prior into standard SSL without redesigning the backbone. MFASSL constructs mirror-paired views aligned to an estimated symmetry axis and introduces a lightweight Mirror-Fusion Attention (MFA) module for adaptive token-level interaction between mirrored regions while preserving asymmetric cues. The base SSL objective is further coupled with reflection-consistency and mid-layer token-alignment losses. Across CheXpert, BraTS, CelebA-HQ, and WFLW, MFASSL improves downstream performance, calibration, and reflection robustness over MoCo-v3, DINO, and MAE baselines under matched ViT-B/16 settings. It also achieves stronger and more consistent gains than recent equivariant SSL approaches with only approximately 2.7\% additional parameters. These results show that lightweight geometry-aware priors can effectively complement invariance-based SSL.

• The paper presents MFASSL, which integrates mirror-paired views and mirror-fusion attention to balance reflection invariance with asymmetric cues.
• It enhances SSL by inserting a lightweight cross-attention module in mid-transformer layers to smartly fuse symmetric and asymmetric features.
• Experimental results demonstrate improved AUROC, Dice scores, and flip-consistency across diverse datasets with only a minimal parameter increase.

Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning

Motivation and Background

Recent advances in self-supervised learning (SSL) for vision have largely relied on invariance-based objectives over augmented views, with the intent of suppressing nuisance factors and enforcing robust feature learning. However, strict reflection invariance, commonly induced by horizontal flipping, is often detrimental for structured domains like medical imaging and faces, which possess approximate but not perfect bilateral symmetry. In such cases, local asymmetries (e.g., unilateral pathologies) are diagnostic, and full invariance discards signal. By contrast, equivariant and symmetry-aware approaches encode geometric priors but typically enforce hard architectural constraints or group actions that may not flexibly accommodate natural, approximate symmetries.

To address this limitation, "Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning" (2607.00850) introduces Mirror-Fusion-Augmented Self-Supervised Learning (MFASSL), a plug-and-play mechanism for Vision Transformers (ViTs) that infuses a soft reflection prior into standard SSL pipelines. MFASSL enables controlled information flow between bilaterally-paired regions during pretraining, enhancing reflection consistency while preserving informative asymmetric cues without modifying the base encoder architecture.

Methodology

MFASSL introduces a comprehensive approach with three main components:

1. Mirror-Paired View Construction: For each input, mirror-paired crops are extracted about an estimated vertical symmetry axis (with random jitter to improve robustness). The right crop is horizontally flipped to align spatial coordinates, enabling tokenwise correspondence between left and right patches throughout the transformer pipeline.
2. Mirror-Fusion Attention (MFA): At a user-selected intermediate transformer layer (layer~8 for ViT-B/16 in most experiments), the model injects a lightweight cross-attention-based module. MFA performs bidirectional information exchange between mirror-aligned token sequences via a learnable, per-token, distance-based gate:
   • For strongly aligned tokens, MFA amplifies symmetric interaction via cross-attention.
   • For tokens with large local feature discrepancies, the fusion is downweighted, and an explicit asymmetry-preserving (residual) term is used.
   • The cross-view attention and discrepancy residual are both scaled by learned coefficients initialized conservatively to stabilize early training.

     Figure 1: The MFASSL architecture combines standard augmentations, mirror-paired crops, and Mirror-Fusion Attention to inject reflection-awareness into SSL pretraining.

3. Symmetry-Aware Objective: MFASSL augments the base SSL objective (contrastive, distillation, or reconstruction loss) with two symmetry-specific regularization terms:
   • A global negative cosine similarity loss between mean features of aligned mirror views ($\mathcal{L}_{\text{eq}}$).
   • A token-wise mean squared error loss on $\ell_2$-normalized pre-fusion tokens, enforcing local correspondence ($\mathcal{L}_{\text{mid}}$). Both are ramped up gradually during initial epochs before MFA is activated, allowing stable feature alignment formation.

MFASSL adapts the pretraining regime but removes all mirror-specific operations at finetuning and deployment, leaving behind a standard ViT encoder free of architectural burden.

Experimental Results

MFASSL is evaluated on both medical (CheXpert, BraTS, OASIS-3) and natural (CelebA-HQ, WFLW) datasets under three major SSL paradigms (MoCo-v3, DINO, MAE), using ViT-B/16 and ViT-S/16 backbones. All baselines are trained from random initialization, without ImageNet pretraining or test-time augmentations, to isolate method-level effects.

Key empirical findings include:

• CheXpert (chest radiograph classification): MFASSL consistently improves AUROC, AUPRC, F1, calibration (ECE/NLL), and Flip-Consistency across all three SSL backbones in fine-tuning, with the largest gain of +1.24 AUROC points over the DINO baseline (84.72 to 85.96).
• BraTS (brain tumor segmentation): MFASSL delivers superior mean Dice (0.836 vs. 0.827 for DINO) and reduced HD95, along with better calibration.
• Natural Image Domains: On CelebA-HQ (attribute classification) and WFLW (landmark localization), accuracy/NME and Flip-Consistency are consistently improved, e.g. a 2.1–2.8% increase in flip-consistent predictions.
• Symmetry-aware Competitors: Compared with recent equivariant SSL approaches (E-SSL [dangovski2021equivariant], OcticViT [nordstrom2025stronger]), MFASSL yields larger and more consistent improvement despite requiring only 2.7% extra parameters.
• Ablation: Mirror views alone or adding only symmetry-aware losses yields marginal benefit; the combination with MFA and token-level regularization is necessary for maximal gain.
• Layer Placement: Fusion at mid-to-late transformer layers (e.g., layer 8 of 12) achieves optimal stability and performance; early-layer insertion causes optimization instability.

Theoretical Perspective

MFA does not impose hard equivariance but instead enacts a data-adaptive, soft reflection bias. The learnable tokenizer gate modulates cross-mirror exchange based on feature similarity, and the residual channel preserves local asymmetric evidence, yielding flexible reflection-aware pretraining. This complements the standard invariance-driven protocols and does not require architecture redesign or explicit group actions.

Practical and Theoretical Implications

Practically, MFASSL provides a generic, low-cost, plug-in for reflection-aware SSL pretraining on ViTs, directly benefiting medical imaging, face analysis, and other domains with strong bilateral regularity. Unlike prior equivariant methods, it does not compromise the transferability or efficiency of the base architecture and is compatible with most SSL pipelines.

Theoretically, this work demonstrates that lightweight, token-level geometric priors can yield measurable gains by partially relaxing strict invariance/equivariance dichotomies, instead allowing learned, data-adaptive signal mixing. This points to a spectrum of future techniques—learned equivariant/flexible priors—for other structured domains, including three-dimensional medical imaging, biological data, or more complex group actions.

Future Developments

Potential directions for extension include:

• Generalizing mirror-fusion to richer symmetry groups or learning symmetry axes end-to-end.
• Extending fusion-based priors to other domains with partial or approximate geometric regularity.
• Combining MFASSL with large-scale pretraining (e.g., medical foundation models) and exploring transfer to more diverse data modalities.

Conclusion

MFASSL establishes a practical, theoretically sound approach for leveraging approximate bilateral structure in SSL, outperforming both naive invariance and hard equivariant baselines across diverse tasks and domains. The method’s design enables seamless integration with existing ViT-based pipelines, and it provides strong evidence that mid-layer, data-driven, geometry-aware priors can nontrivially enhance self-supervised visual representations.