Mirror-Fusion Attention (MFA)

• The paper introduces MFA, a lightweight adapter that integrates mirror-paired cross-attention into self-supervised learning without altering the test-time ViT architecture.
• MFA aligns mirror-paired token sequences and employs a gated cross-attention mechanism to fuse complementary information while retaining asymmetric cues.
• Empirical results demonstrate improved performance and calibration on diverse medical and natural-image benchmarks with minimal parameter overhead (~2.7%) and extra FLOPs.

Mirror-Fusion Attention (MFA) is a lightweight, pretraining-only adapter introduced within Mirror-Fusion-Augmented Self-Supervised Learning (MFASSL), a Vision Transformer framework that injects a soft reflection prior into standard self-supervised learning (SSL) without redesigning the backbone. It is motivated by the observation that most SSL methods encourage invariance across augmentations, but strict flip invariance can suppress informative left–right correspondences in approximately bilateral data such as medical images and human faces. MFA operates on mirror-paired views aligned to an estimated symmetry axis, enabling adaptive token-level interaction between mirrored regions while preserving asymmetric cues; in the reported formulation, it is removed at test time, so the model retains a standard single-image Vision Transformer inference path (Li et al., 1 Jul 2026).

1. Conceptual setting and scope

MFA is designed for approximately bilateral visual domains in which horizontal reflection is neither purely nuisance variation nor a symmetry that should be enforced exactly. The underlying claim is that strict flip invariance can discard structure that is diagnostically or semantically informative, while hard equivariant treatments may be overly rigid for real data that only approximately obey bilateral symmetry (Li et al., 1 Jul 2026).

Within MFASSL, the role of MFA is narrowly defined. Standard SSL views from MoCo-v3, DINO, or MAE follow the original ViT path end-to-end. In parallel, mirror-paired views are created from left and right halves of the same image, aligned to an estimated vertical symmetry axis, and processed up to a selected transformer block. MFA then fuses the corresponding token sequences at that intermediate layer. The fused tokens continue through the remaining transformer blocks and contribute to the same base SSL objective (Li et al., 1 Jul 2026).

This design distinguishes MFA from a backbone redesign. No changes are made to positional encodings or to the single-image inference path, and at test time MFA is removed and the model is a standard ViT. A plausible implication is that the method is intended to alter representation formation during pretraining rather than to impose a persistent architectural constraint during deployment (Li et al., 1 Jul 2026).

2. Mathematical formulation

At a chosen transformer block $\ell$, MFA operates on two aligned token sequences

$X_L, X_R \in \mathbb{R}^{N \times D},$

corresponding to the left crop and the horizontally flipped right crop of the same image. Cross-attention is formed from $R \rightarrow L$ by projecting $X_L$ to queries and $X_R$ to keys and values:

$$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$

where

$W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$

and $D_h$ is the head dimension. The cross-attention output is

$$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$

Fusion strength is adapted per token through a learnable distance-based gate

$g \in \mathbb{R}^{N \times 1},$

with

$X_L, X_R \in \mathbb{R}^{N \times D},$0

where $X_L, X_R \in \mathbb{R}^{N \times D},$1 is sigmoid, $X_L, X_R \in \mathbb{R}^{N \times D},$2, and $X_L, X_R \in \mathbb{R}^{N \times D},$3.

The final fused token representations combine identity, gated cross-attention, and an asymmetry-preserving residual:

$X_L, X_R \in \mathbb{R}^{N \times D},$4

$X_L, X_R \in \mathbb{R}^{N \times D},$5

where $X_L, X_R \in \mathbb{R}^{N \times D},$6 are initialized to $X_L, X_R \in \mathbb{R}^{N \times D},$7. During backpropagation, MFA receives gradients both from the base SSL loss and from the symmetry-aware regularizers (Li et al., 1 Jul 2026).

The formulation encodes two complementary commitments. The gated cross-attention term promotes interaction between mirrored regions when the two token streams are sufficiently compatible, while the residual difference term explicitly preserves asymmetry. This is consistent with the paper’s characterization of MFA as a “data-dependent, locally adaptive relaxation between strict flip-invariance and hard group-equivariance” (Li et al., 1 Jul 2026).

3. Symmetry axis estimation and token correspondence

The pretraining procedure assumes approximate bilateral symmetry about a vertical midline. Each image $X_L, X_R \in \mathbb{R}^{N \times D},$8 is resized to $X_L, X_R \in \mathbb{R}^{N \times D},$9, and a “jittered” axis is defined as

$R \rightarrow L$0

Left and right halves are then cropped around that axis. The right patch is horizontally flipped so that both views share the same orientation, and each view is tokenized into $R \rightarrow L$1 patches. Because both views now use the same coordinate frame, patch $R \rightarrow L$2 in $R \rightarrow L$3 directly corresponds to patch $R \rightarrow L$4 in $R \rightarrow L$5 (Li et al., 1 Jul 2026).

This alignment procedure is central to MFA’s token-level semantics. The module does not infer correspondence through a learned matching stage; rather, correspondence is supplied by the mirror-pair construction itself. A plausible implication is that the quality of the soft reflection prior depends on how well approximate bilateral structure is captured by the estimated axis and crop geometry (Li et al., 1 Jul 2026).

The method also imposes a specific operational distinction between standard augmentations and mirror-paired inputs. Standard SSL views remain unchanged, while mirror-paired views are introduced as an auxiliary geometric pathway during pretraining. In that sense, MFA is not a replacement for the underlying SSL mechanism but an added reflection-aware pathway coupled to it (Li et al., 1 Jul 2026).

4. Integration into ViT and training objective

MFA is inserted only during pretraining at a mid-to-late layer $R \rightarrow L$6, with $R \rightarrow L$7 in a 12-layer ViT-B/16. Mirror-paired views are run up to block $R \rightarrow L$8, yielding $R \rightarrow L$9 and $X_L$0. These pre-fusion tokens are used in the symmetry losses and then fused by MFA. The fused tokens $X_L$1 continue through blocks $X_L$2 and contribute to the same base SSL objective (Li et al., 1 Jul 2026).

MFASSL augments the underlying SSL loss $X_L$3 with two reflection-aware terms at layer $X_L$4.

For the reflection-consistency loss, the normalized mean-pooled representation is defined as

$X_L$5

and

$X_L$6

For the mid-layer token alignment loss, each patch token is normalized to unit norm:

$X_L$7

and per-token agreement is enforced through

$X_L$8

The total pretraining objective at epoch $X_L$9 is

$X_R$0

and

$X_R$1

with $X_R$2 and $X_R$3.

After $X_R$4 with $X_R$5, MFA is activated and its gate is ramped by

$X_R$6

These choices make clear that symmetry regularization precedes full fusion strength. This suggests a staged optimization strategy in which the model first learns correspondence-aware intermediate representations and only subsequently activates token fusion at increasing strength (Li et al., 1 Jul 2026).

5. Parameters, compute, and implementation profile

The additional parameters introduced by MFA are limited to two global scalars for the gate $X_R$7, two global scalars for fusion $X_R$8, and cross-attention projection weights $X_R$9 with biases. These additions amount to approximately $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$0 parameters over the base ViT. The computational overhead is one extra cross-attention block at layer $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$1 on mirror-paired views only during pretraining, corresponding to approximately $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$2–$$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$3 extra FLOPs (Li et al., 1 Jul 2026).

The key hyperparameters are fixed across SSL backbones and ViT scales unless noted: fusion layer $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$4, gate $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$5, jitter $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$6 image width, $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$7, $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$8, $$Q_L = X_L W_q,\qquad K_R = X_R W_k,\qquad V_R = X_R W_v,$$9, $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$0, and $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$1 (Li et al., 1 Jul 2026).

All reported pretraining uses ViT-B/16 from scratch for 300 epochs with AdamW. Baselines are MoCo-v3, DINO, MAE, plus E-SSL and OcticViT for comparison (Li et al., 1 Jul 2026).

A common misunderstanding would be to treat MFA as an inference-time architectural modification. The reported method explicitly states the opposite: MFA is pretraining-only, and at test time the model is a standard ViT. Another possible misunderstanding is that mirrored views alone supply the gains; the ablation results state that mirrored views alone have no effect (Li et al., 1 Jul 2026).

6. Empirical behavior and ablations

MFASSL with MFA is evaluated on two medical benchmarks—CheXpert chest X-ray classification and BraTS brain MRI segmentation—and two natural-image tasks—CelebA-HQ face attribute classification and WFLW face landmark localization. Across these settings, the reported effect is improved downstream performance, calibration, and reflection robustness relative to MoCo-v3, DINO, and MAE baselines under matched ViT-B/16 settings, along with stronger and more consistent gains than recent equivariant SSL approaches (Li et al., 1 Jul 2026).

Task	Baseline $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$2 MFASSL with MFA	Additional reported effects
CheXpert multi-label classification (14 labels)	DINO: $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$3; MoCo-v3: $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$4; MAE: $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$5	Flip-Consistency $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$6–$W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$7 pp; ECE reduced by up to $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$8
BraTS slice-wise segmentation (ET/TC/WT)	DINO: $W_q, W_k, W_v \in \mathbb{R}^{D \times D_h}$9; MoCo-v3: $D_h$0; MAE: $D_h$1	Mean HD95 reduced by $D_h$2–$D_h$3 mm; NLL and ECE likewise improved or comparable
CelebA-HQ attribute classification	DINO $D_h$4; MoCo-v3 $D_h$5; MAE $D_h$6	Flip-Consistency $D_h$7–$D_h$8 pp, NLL $D_h$9–$$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$0, ECE $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$1–$$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$2
WFLW landmark localization	DINO $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$3; MoCo-v3 $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$4; MAE $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$5	[email protected] and Flip-Consistency also improved

The comparison to equivariant SSL baselines is specific on CheXpert: E-SSL and OcticViT yield less than $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$6 pp gains under identical settings (Li et al., 1 Jul 2026).

The ablations sharpen the interpretation of MFA’s contribution. Mirrored views alone have no effect. Adding only $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$7 gives approximately $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$8 pp AUROC on CheXpert and $$A_{L \leftarrow R} = \mathrm{Softmax}\left(\frac{Q_L K_R^\top}{\sqrt{D_h}}\right)V_R.$$9 pp Dice on BraTS. MFA without symmetry losses yields smaller gains than full MFASSL. The best fusion depth is $g \in \mathbb{R}^{N \times 1},$0; $g \in \mathbb{R}^{N \times 1},$1 diverges, while $g \in \mathbb{R}^{N \times 1},$2 is weaker (Li et al., 1 Jul 2026).

These ablations constrain the mechanism claims that can be made. They indicate that the reported gains do not arise from view construction alone and that the full method depends on the interaction between mid-layer symmetry regularization and adaptive token fusion. They also indicate that fusion depth is not a negligible detail, since earlier insertion can destabilize training and later insertion weakens the effect (Li et al., 1 Jul 2026).

7. Relation to invariance, equivariance, and asymmetric structure

MFA is positioned between two familiar SSL tendencies: strict invariance to augmentation and hard group-equivariance. The motivating argument is that strict flip invariance can suppress informative left–right correspondences in approximately bilateral data, whereas the reported MFA mechanism preserves asymmetric cues through the residual term $g \in \mathbb{R}^{N \times 1},$3 and its counterpart on the right branch (Li et al., 1 Jul 2026).

The method therefore does not enforce exact reflection symmetry. Instead, it introduces a soft reflection prior by combining three elements: aligned mirror-paired views, symmetry-aware losses at an intermediate layer, and gated cross-attention whose strength depends on token-level discrepancy. This suggests that reflection is treated as a structured inductive bias rather than as an exact transformation law (Li et al., 1 Jul 2026).

The reported conclusion is correspondingly narrow and technical: lightweight geometry-aware priors can effectively complement invariance-based SSL, and MFA provides a data-dependent, locally adaptive relaxation between strict flip-invariance and hard group-equivariance. In the reported experiments, this yields consistent downstream improvements and enhanced robustness to reflection with only approximately $g \in \mathbb{R}^{N \times 1},$4 extra parameters and minimal changes to the ViT backbone (Li et al., 1 Jul 2026).