Reflection Contrast Mining Module (RCMM)

• RCMM is a neural module that explicitly mines contrasting reflective cues from paired illumination images to improve glass detection and depth estimation.
• It integrates multi-rate dilated convolutions with paired UNet branches to extract local and context-dependent reflection features for robust segmentation.
• Empirical results show RCMM increases glass IoU by up to 4.21% and reduces depth estimation errors, confirming its practical effectiveness in complex scenes.

The Reflection Contrast Mining Module (RCMM) is a neural architectural component designed to identify, extract, and leverage reflective cues for visual tasks in challenging environments, specifically targeting glass surface detection in paired flash/no-flash imagery (Yan et al., 21 Nov 2025) and monocular depth estimation in reflection-containing scenes (&&&1&&&). RCMM operates by explicitly mining and contrasting features that characterize reflection, in order to overcome the inherent limitations of purely appearance- or boundary-based methods. It enables networks to robustly localize problematic reflective regions and mitigate their ambiguous effects on supervision or segmentation. RCMM is central to models such as NFGlassNet for glass surface detection in paired illumination settings and has also influenced loss-oriented training modules in self-supervised depth learning.

1. Motivation and Problem Framing

Conventional computer vision methods for glass or mirror detection, as well as self-supervised monocular depth estimation, rely heavily on appearance, color, and consistency cues. However, glass surfaces are intrinsically feature-poor, colorless, and possibly frameless, rendering appearance-based localization unreliable. Reflections, although potentially strong cues, are highly context-dependent and easily confused with scene content or background textures. In no-flash/flash scenarios, the reflective component’s intensity and shape change dramatically depending on illumination and scene geometry. RCMM is designed to mine these reflection contrasts, detecting cues that are otherwise inaccessible to single-image or unstructured methods (Yan et al., 21 Nov 2025).

Similarly, in self-supervised monocular depth estimation, photometric losses are corrupted at reflective regions due to the violation of Lambertian surface assumptions. This causes training instabilities and significant prediction errors. RCMM addresses this by automatically identifying such problematic regions and applying targeted contrastive treatments that neutralize misleading gradients (Choi et al., 20 Feb 2025).

2. Architectural Integration and Position in Networks

In flash/no-flash glass detection (NFGlassNet (Yan et al., 21 Nov 2025)), RCMM is situated after dual Swin-Transformer V2 encoders that extract hierarchical feature representations from both flash and no-flash input images at four spatial scales. At each scale, RCMM operates on the corresponding feature maps from both modalities and outputs reflection features and intermediate reflection maps. Downstream, a Reflection Guided Attention Module (RGAM) fuses reflection features with encoded representations before final upsampling and segmentation decoding.

In self-supervised monocular depth estimation (Choi et al., 20 Feb 2025), RCMM is a special training-time module. It wraps around the backbone depth and pose estimation modules (e.g., ResNet18 encoder in Monodepth2, HRDepth, or MonoViT) and introduces additional loss branches involving triplet mining, reflective mask computation, and (optionally) teacher-student knowledge distillation.

3. Internal Structure and Feature Mining

The RCMM in glass detection is comprised of three coordinated branches:

• Dilated-Convolution Contrast Branch: For a set of dilation rates $r\in\{1,2,4,8\}$, 3×3 dilated convolutions are applied to both flash and no-flash feature maps. Each unordered pair of dilation rates produces summed features, and the flash/no-flash difference is computed to yield six "contrast" tensors indicating localized reflection changes. All six are concatenated along the channel dimension.
• Paired UNet Appearance Branches: Two parallel four-stage UNets, one for each input modality, extract more holistic appearance features from flash and no-flash maps.
• Fusion and Projection: The contrast branch output is elementwise-multiplied by each UNet branch’s features, and their sum forms the final reflection feature. Two distinct $1\times1$ convolution heads predict the intermediate reflection maps for each modality.

The following summarizes the computation at scale $i$:

Branch	Input Features	Operation	Output Feature
Contrast	$F_\text{flash}^{(i)}$, $F_\text{no-flash}^{(i)}$	Multi-rate dilated conv, pairwise sum/diff, concat	$F_\text{refl}^{CB}$
UNet-flash	$F_\text{flash}^{(i)}$	4-stage UNet	$F_\text{refl}^{UB{\text -flash}}$
UNet-no-flash	$F_\text{no-flash}^{(i)}$	4-stage UNet	$F_\text{refl}^{UB{\text -no-flash}}$
Fusion	Above three features	Elementwise product + sum	$F_\text{refl}$

This precise stacking enables RCMM to capture both local and context-dependent reflection cues and effectively isolate glass boundaries under varying illumination.

4. Mathematical Formulation and Training Objectives

The internal steps of RCMM are mathematically defined as follows (Yan et al., 21 Nov 2025):

1. Contrast Branch:
   • Dilated feature extraction:

   $$DF_r^f = DConv_r(F_f),\quad DF_r^n = DConv_r(F_n),\quad r \in \{1,2,4,8\}$$

• Pairwise addition:

  $RF_{r_i,r_j}^f = DF_{r_i}^f + DF_{r_j}^f$

• Pairwise contrast (six unordered pairs):

  $$\Delta RF_{r_i, r_j} = RF_{r_i, r_j}^f - RF_{r_i, r_j}^n$$

• Concatenation:

  $$F_{\text{refl}}^{CB} = \text{Concat}_{(r_i<r_j)} \left( \Delta RF_{r_i, r_j} \right)$$

1. UNet Branches:

   $$F_{\text{refl}}^{UB-f} = U^f(F_f),\qquad F_{\text{refl}}^{UB-n} = U^n(F_n)$$

2. Fusion:

   $$F_{\text{refl}} = F_{\text{refl}}^{CB} \odot F_{\text{refl}}^{UB-f} + F_{\text{refl}}^{CB} \odot F_{\text{refl}}^{UB-n}$$

3. Reflection Maps:

   $$R_{\text{flash}} = \text{Conv}_{1\times1}(F_{\text{refl}}),\qquad R_{\text{no-flash}} = \text{Conv}_{1\times1}(F_{\text{refl}})$$

Training loss is a weighted sum of a binary cross-entropy and IoU-based glass mask supervision and an auxiliary multi-scale reflection map supervision:

$$L = \sum_{i=1}^4 L_{\text{glass}}(G_i, \hat{G}_i) + \lambda \sum_{i=1}^4 \sum_{j \in \{f, n\}} L_{\text{refl}}(R_i^j, \hat{R}_i^j)$$

where

$$L_{\text{glass}}(G_i, \hat{G}_i) = 1 - \text{IoU}(G_i, \hat{G}_i) + \text{BCE}(G_i, \hat{G}_i)$$

$$L_{\text{refl}}(R_i^j, \hat{R}_i^j) = [1 - \text{SSIM}(R_i^j, \hat{R}_i^j)] + \|R_i^j - \hat{R}_i^j\|_1$$

Pseudo-ground-truth $\hat{R}_i^j$ is computed by masking the input with ground-truth glass regions and a pre-trained reflection detector.

In monocular depth, RCMM is expressed as a per-pixel mask mining and loss modulation mechanism. A hinge-based triplet loss penalizes misleading gradients at reflective pixels:

$$L_{\text{tri}}(I_{\text{ref}}, I_{s \to r}, I_{r \to s}) = M_r \odot [E^+ - E^- + \delta]_+ + (1-M_r) \odot E^+$$

Key variables:

• $E^+ = P(I_{s\to r}, I_{\text{ref}})$: standard photometric loss
• $E^- = P(I_{s\to r}, I_{r\to s})$: cross-view photometric loss
• $M_r$: predicted reflective region mask (via thresholding)
• $[x]_+ = \max(0, x)$: hinge
• $\delta$: adaptive or fixed margin (Choi et al., 20 Feb 2025)

A knowledge distillation extension enables a student network to combine the best properties of "reflective-aware" and standard depth estimates using a log-$L_1$ pixelwise loss.

5. Implementation Details and Hyperparameterization

NFGlassNet (Yan et al., 21 Nov 2025):

• Backbones: Swin-Transformer V2, 4 scales
• UNet branch: 4-stage, standard conv/activation/pooling
• Contrast branch: 3×3 conv, dilation $\in\{1,2,4,8\}$
• Fusion: elementwise product, channelwise sum
• Reflection map channels: single channel per modality
• Typical feature sizes: input resolution $(384/2^{i-1}) \times (384/2^{i-1})$, channels $48 \cdot 2^{i-1}$
• Loss coefficients: $\lambda = 0.8$
• Training target for reflection maps: LANet reflection detector, masked by glass GT

Monocular Depth (Choi et al., 20 Feb 2025):

• Input resolution: $384 \times 288$
• Backbones: Monodepth2 (ResNet18), HRDepth, MonoViT
• Optimizer: Adam, learning rate $1 \times 10^{-4}$, multistep decay
• Batch sizes: 12 (Monodepth2, HRDepth), 8 (MonoViT)
• Loss weights: $\alpha_1=0.85$, $\alpha_2=0.15$ (photometric), $\delta=0.05$–$0.1$ or adaptive
• Additional: Monodepth2’s auto-masking for occlusion removal

6. Quantitative Impact and Ablation Analysis

Experiments corroborate that RCMM yields significant and robust improvements over backbone baselines in both glass detection and reflection-robust depth estimation tasks.

NFGlassNet (IoU, Table 5 (Yan et al., 21 Nov 2025)):

• Baseline (no RCMM/RGAM): 81.96%
• +RCMM: 84.85% (+2.89)
• +RGAM: 82.24%
• +RCMM+RGAM: 86.17%

Ablations show removal of the contrast branch or key permutations each reduce IoU by 1–2 points, confirming the necessity of the explicit contrast design.

Reflection-robust Monocular Depth (Choi et al., 20 Feb 2025):

Dataset	Backbone	Baseline AbsRel	AbsRel with L_tri	AbsRel with L_rkd (distill)
ScanNet-Reflection	Monodepth2	0.181	0.157	0.150
NYU-v2	Monodepth2	0.171	0.166	0.155
Booster (zero-shot)	Monodepth2	0.520	0.430	0.419

No degradation is observed on splits with no reflection (<0.002 AbsRel change), indicating precise localization and selective gradient suppression.

7. Limitations and Broader Implications

RCMM can still be confused by specular highlights on non-glass surfaces (e.g., polished tiles (Yan et al., 21 Nov 2025)). In extreme cases, additional scene cues or sensors (e.g., depth or IR) may further disambiguate reflection sources. Importantly, the module's lightweight structure, mathematical transparency, and generality make it adaptable to a wide array of reflection-afflicted vision problems.

A plausible implication is that reflection contrast mining will remain relevant as a building block for multi-modal, context-aware segmentation and 3D scene understanding in complex real-world settings. Its core principle—contrastive structural mining of ambiguous visual cues—may generalize to other domains where signal ambiguity is illumination- or modality-dependent.