Inlier Attention (IA) Block Overview

• Inlier Attention Block is a robust architectural component that uses a soft attention mechanism to compute inlier confidence, enhancing feature-based correspondence.
• It replaces standard normalization with a weighted mean computation using a 1×1 attention head and softmax-derived weights to suppress outlier effects.
• Empirical studies on the COLMAP dataset show significant F1-score improvements, validating its effectiveness over traditional approaches.

The Inlier Attention (IA) Block is an architectural component introduced for robust mismatch removal in feature-based correspondence tasks. It addresses the challenge of corrupted global context statistics in neural networks caused by a high proportion of outlier correspondences. The IA Block implements a learned attention mechanism, termed Inlier-Attention Normalization (IAN), which computes a soft inlier-confidence for each correspondence and uses these weights to modulate feature normalization. By downweighting outlier contributions in the global mean calculation while preserving individual feature integrity, the IA Block enables more reliable context modeling and separation of inliers from outliers (Chen et al., 2019).

1. Motivation and Problem Setting

In traditional mismatch removal pipelines, each correspondence influences the global statistics used for normalization, such as mean and variance. Standard Context Normalization (CN) treats all correspondences equally, leading to problematic context estimation when outliers are abundant. This often manifests as skewed global statistics, which degrade the separability of inliers and outliers. The IA Block replaces undifferentiated CN with a feature-wise soft attention mechanism that learns inlier confidence scores, introducing a data-driven approach to downweight the influence of potential outliers during normalization. The rationale is to alleviate the issue of outlier interference and enhance global context extraction (Chen et al., 2019).

2. Architectural Structure

At each network layer $l$, the IA Block operates over a collection of $N$ feature vectors $\{o_i^l\}_{i=1}^N$ with dimensionality $C$. The core structure involves:

• 1×1 Attention Head: For each feature $o_i^l$, the attention logit is computed as $r_i^l = w_{att}^\top o_i^l + b_{att}$, generating a vector $\mathbf{r}^l \in \mathbb{R}^N$.
• Softmax-Based Confidence: Inlier-confidence weights are derived via $w_i^l = \exp(r_i^l)/\sum_{j=1}^N \exp(r_j^l)$, resulting in attention weights $w^l \in \mathbb{R}^N$ summing to 1.
• Weighted Mean Computation: The global context vector is calculated as $\mu^l = \frac{1}{N}\sum_{i=1}^N (w_i^l N) o_i^l$, with attention modulating each feature’s contribution.
• Unweighted Variance Estimation: The standard deviation is retained from standard CN, $$\sigma^l = \sqrt{\frac{1}{N}\sum_{i=1}^N (o_i^l - \mu^l)^2}$$.
• Normalization: Each feature is normalized as $\hat o_i^l = (o_i^l - \mu^l)/\sigma^l$ before passing to subsequent processing layers, such as multi-layer perceptrons or residual blocks.

3. Mathematical Description

Let $O^l \in \mathbb{R}^{N \times C}$ denote the feature matrix. The IAN operation is specified by: $$\begin{aligned} r^l & = O^l w_{att} + b_{att} \in \mathbb{R}^N, \ w_i^l & = \frac{\exp(r^l_i)}{\sum_{j=1}^N \exp(r^l_j)}, \ \mu^l & = \frac{1}{N}\sum_{i=1}^N (w_i^l N) o_i^l, \ \sigma^l & = \sqrt{\frac{1}{N}\sum_{i=1}^N (o_i^l - \mu^l)^2}, \ \hat o_i^l & = \frac{o_i^l - \mu^l}{\sigma^l}. \end{aligned}$$ The principal differentiation from standard CN is the use of attention weights $w_i^l$ in the weighted mean $\mu^l$, which biases the context toward presumed inliers. No direct scaling is applied to the features themselves based on attention, preserving individual feature characteristics.

4. Algorithmic Workflow

The following pseudocode formalizes the IA Block procedure (Chen et al., 2019):

// Input: features O ∈ ℝ^{N×C} // Output: normalized features Ō ∈ ℝ^{N×C} 1. // Compute attention logits for i=1…N: r[i] ← Linear_att(O[i]) // 1×1 conv or FC to scalar 2. // Normalize to get inlier‐confidence weights w ← Softmax(r) // w ∈ ℝ^N, sum(w)=1 3. // Compute weighted mean μ ∈ ℝ^C μ ← (1/N) * sum_{i=1..N} (w[i]*N) * O[i] 4. // Compute unweighted std σ ∈ ℝ^C σ ← sqrt{ (1/N) * sum_{i=1..N} (O[i] − μ)² } 5. // Normalize each feature for i=1…N: Ō[i] ← (O[i] − μ) ./ σ // element‐wise return Ō

5. Integration in GLA-Net

The IA Block is deployed within two processing stages of the end-to-end GLA-Net architecture:

• Crude Context Subnet: Each stage utilizes an IA Block to generate a robust global context while suppressing outlier influence. The resulting features are input to an MLP, yielding a preliminary inlier probability map supervised by an auxiliary loss.
• Fine Optimization Subnet: Preliminary inlier probabilities supplied by the crude subnet directly replace the attention head in subsequent IA Blocks, refining the normalization process in a coarse-to-fine paradigm. The 1×1 convolution is bypassed, and the crude subnet’s output is used as attention weights.
• Guided Loss Supervision: All IA Blocks and the downstream classifier are optimized jointly through a Guided Loss function designed to maximize the $F_n$-score metric.

A key aspect is the mutual adaptation between IA Block normalization and classification supervised by the targeted metric.

6. Empirical Validation

The effectiveness of the IA Block is substantiated through extensive ablation and comparison studies:

Method	F1 Score (%)	Setting
Baseline (LFGC‐Net w/o IA)	34.83	COLMAP
+ IA Block only	38.10	COLMAP
+ Guided Loss only	38.10	COLMAP
Full GLA‐Net (IA + Guided Loss)	44.12	COLMAP
LFGC‐Net (Original CN)	38.10	COLMAP
LFGC‐Net (SE‐Block, spatial attn)	35.98	COLMAP
LFGC‐Net (NM‐Net‐sp, local graph)	38.25	COLMAP
LFGC‐Net (IA Block)	43.20	COLMAP

Evaluation on the COLMAP dataset shows that augmenting the baseline with the IA Block yields a significant F1 gain (+3.3%), and combining IA Block with Guided Loss in GLA-Net gives an overall F1 improvement of 9.3%. Replacement experiments indicate that the IA Block outperforms common alternatives such as SE-Block and NM-Net-sp.

Quantitative analysis of attention weights reveals that the average inlier-weight to outlier-weight ratio exceeds 1.0 across all IA layers, increasing with network depth, validating that the IA Block learns to accentuate inlier feature contributions.

7. Context and Impact

The IA Block offers a targeted solution to outlier-induced degradation in global context estimation for correspondence-based tasks, notably mismatch removal. Empirical evidence demonstrates that the learned soft inlier weighting in IAN produces more robust feature normalization, facilitating state-of-the-art F1-score performance in GLA-Net compared to previous approaches (Chen et al., 2019). This approach highlights a broader direction in context-sensitive normalization methods tailored via task-driven attention mechanisms.