Adaptive Focused Regularization in SC-Net

• AFR is a module within SC-Net that uses position-aware attention and soft filtering to refine motion fields for improved pose estimation.
• It leverages multi-head graph attention with spatial priors and confidence-based filtering to selectively enhance correspondence classification and regression.
• Empirical results demonstrate that AFR adds over 17 mAP improvement in high-disparity, noisy settings, proving its efficacy in robust geometric reasoning.

SC-Net is a neural architecture for two-view correspondence learning, designed to integrate spatial and cross-channel contextual information for improved robustness in motion field estimation and correspondence classification, particularly in challenging visual scenes with high disparity or spurious matches. The core contributions reside in its multi-stage pipeline that combines Convolutional Neural Network (CNN) backbones with specialized modules for adaptive regularization, bilateral context modeling, and position-sensitive recovery. SC-Net achieves state-of-the-art results on large-scale pose estimation and outlier rejection benchmarks, notably YFCC100M and SUN3D, outperforming previous CNN and multilayer perceptron-based methods by a significant margin (Lin et al., 29 Dec 2025).

1. Network Architecture and Workflow

SC-Net comprises three principal modules applied in sequence within each of its $L$ rectifying layers:

1. Adaptive Focused Regularization (AFR): Receives unordered motion feature matches and grid embeddings, transforms them into motion field estimates via multi-head graph attention augmented with spatial priors and confidence-based filtering.
2. Bilateral Field Adjustment (BFA): Refines motion fields by capturing long-range dependencies across both spatial positions and feature channels.
3. Position-Aware Recovery (PAR): Recovers dense motion vectors with guaranteed spatial precision and consistency.

Each rectifying layer processes the outputs of the previous one, iteratively enhancing the structure and sharpness of the sparse motion fields before producing the final correspondence and pose predictions (Lin et al., 29 Dec 2025). The AFR module initiates every rectifying layer, feeding its output to BFA and subsequently PAR.

2. Adaptive Focused Regularization (AFR) Module

AFR is central to SC-Net's ability to generate position-aware, reliable motion fields. Its main characteristics:

• Inputs: Motion features $M^{l-1} \in \mathbb{R}^{N \times C}$, grid embeddings $G \in \mathbb{R}^{K^2 \times C}$, match coordinates $X \in \mathbb{R}^{N \times 2}$, grid center coordinates $Y \in \mathbb{R}^{K^2 \times 2}$, and previous inlier logits $\hat z_{cls}^{l-1} \in \mathbb{R}^{N}$.
• Multi-Head Graph Attention: Each grid cell attends to all putative matches, computing attention with added position-aware bias and value soft-filtering.
• Mathematical Formulation: For each attention head $i$, the computation is:

  • Query/key/value projections:

    $$Q_i = G W_i^Q,\quad K_i = M^{l-1} W_i^K,\quad V_i = M^{l-1} W_i^V$$

  • Spatial correlation via shared MLP $F_3$:

    $$\Phi_X = F_3(X),\quad \Phi_Y = F_3(Y),\quad S = \Phi_Y \Phi_X^T$$

  • Position-aware bias:

    $$B_i = \psi\Big(\alpha_i \cdot \frac{S}{\sqrt{C}} + \beta_i\Big)$$

    with $\psi$: LeakyReLU, and $\alpha_i$, $\beta_i$ learned scalars.

  • Attention weights:

    $$A_i = \operatorname{Softmax}( Q_i K_i^T / \sqrt{d_{qk}} + B_i )$$

  • Soft filtering (using inlier probability $p$):

    $$Ẑ = \operatorname{diag}(\sigma(\hat z_{cls}^{l-1}))$$

    $O_i = A_i\, (Ẑ\, V_i)$

  • Output:

    $F^l = \Big[\text{Concat}_{i=1}^H O_i\Big]\, W^O$

This design enhances the motion field's robustness by aligning attention with both geometric proximity and dynamic confidence, mitigating oversmoothing and suppressing outliers (Lin et al., 29 Dec 2025).

3. Loss Functions and Training

SC-Net is optimized end-to-end with a hybrid loss aggregated across its $L$ rectifying layers:

$$\mathcal{L} = \sum_{l=0}^{L-1} \Big[ \mathcal{L}_{cls}(\hat z_{cls}^l, z_{cls}) + \lambda \mathcal{L}_{reg}(\hat E^l, E) \Big]$$

• $\mathcal{L}_{cls}$: Binary cross-entropy loss for correspondence inlier-outlier classification.
• $\mathcal{L}_{reg}$: Regression loss aligning the predicted essential matrix $\hat E^l$ with the ground-truth essential matrix $E$.
• $\lambda$: Weight ramps from $0$ to $0.5$ over training (after 20K steps).

AFR does not introduce any auxiliary loss but directly improves classification and regression by the refined features it produces (Lin et al., 29 Dec 2025).

4. Hyperparameter Settings and Implementation Details

Key architectural parameters used in reported experiments:

• Grid size $K=16$ (i.e., a $16 \times 16$ spatial grid, $K^2=256$).
• Number of rectifying layers $L=6$.
• Number of multi-head attention heads $H=4$; per-head dimension is $C/H$.
• Position MLP $F_3$: two fully-connected layers with ReLU, output dimension $C$.
• Attention activation $\psi$: LeakyReLU (negative slope 0.2).
• Adam optimizer (initial learning rate $1\mathrm{e}{–4}$).
• $\lambda$ schedules as above.

Ablation studies confirm $K=16$ and $L=6$ as optimal; increasing these broadens context but can trade off efficiency (Lin et al., 29 Dec 2025).

5. Empirical Performance and Ablation

SC-Net delivers substantial performance gains on established benchmarks:

• On YFCC100M (known scenes), the mean average precision at 5° (mAP@5°) is:
  • Baseline (no AFR, no BFA): 46.09
  • +HED (encoder-decoder in BFA): 57.96
  • +MFM: 59.60
  • +SF (soft filtering in AFR): 61.96
  • +SF + PA (full AFR): 64.35

Isolating AFR's contributions:

• Soft filtering alone yields a +2.36 mAP point increase.
• Position-aware attention adds another +2.39 points.
• Total improvement over unregularized baseline: >17 mAP.

On both correspondence and pose estimation tasks, SC-Net demonstrates greater robustness in high-disparity, outlier-rich scenarios compared to prior CNN backbones (Lin et al., 29 Dec 2025).

6. Design Rationale and Theoretical Properties

The two principal enhancements in AFR—position-aware attention and soft filtering—address known limitations of global-attention CNNs:

• By incorporating learned spatial biases, AFR preserves motion field discontinuities and spatial structure that standard graph attention tends to oversmooth.
• Soft filtering, enabled by the inlier confidence, suppresses spurious or inconsistent matches, concentrating model capacity on reliable correspondences.
• These mechanisms yield motion representations that are both globally informed and spatially localized, facilitating both more accurate pose estimation and reliable outlier rejection (Lin et al., 29 Dec 2025).

A plausible implication is the architectural pattern of using position- and confidence-aware attention can be generalized to other dense correspondence and geometric reasoning tasks suffering from spatial or semantic ambiguity.

7. Applications and Broader Impact

SC-Net is tailored for two-view correspondence, relative pose estimation, and outlier removal in large-scale visual datasets. It is especially effective where precise spatial-awareness and resilience to noisy matches are needed, such as structure from motion, SLAM, and challenging robotics perception contexts. Its modular form, especially with AFR as a front-end regularizing module, permits seamless integration with downstream or alternative refinement stages. Performance advances demonstrated on YFCC100M and SUN3D highlight SC-Net's capacity to push the state of the art in vision-based geometric reasoning (Lin et al., 29 Dec 2025).