Early-Fusion Sun-Glint Mask in Airborne Imaging

• The method introduces early fusion of a continuous sun-glint mask with RGB channels to enhance artifact suppression in bathymetric imaging.
• It employs HSV conversion and a dual-threshold approach to isolate glint-affected regions for targeted denoising and structural reconstruction.
• Evaluations on the synthetic Sea-Undistort dataset show improved perceptual quality, DSM reconstruction, and depth estimation compared to conventional pipelines.

An early-fusion sun-glint mask is an algorithmic strategy for airborne bathymetric image restoration, designed to suppress the adverse effects of sun glint—specular reflected sunlight—which complicates the remote extraction of seabed structure. In airborne imaging, dynamic water surfaces and solar illumination induce complex optical distortions, with sun glint posing a particular challenge because it obscures or saturates parts of the scene, especially in high-resolution mapping of shallow water environments. The early-fusion sun-glint mask methodology supplements traditional restoration pipelines by explicitly encoding glint-affected regions as an additional input tensor, enabling more targeted artifact suppression and improved preservation of information critical to bathymetric workflows.

1. Construction of the Early-Fusion Sun-Glint Mask

The sun-glint mask is generated directly from the input RGB imagery. The RGB image is first converted to the HSV (Hue, Saturation, Value) color space to isolate characteristics indicative of glint: high brightness and saturation. Mask values are assigned based on the V (value) channel with a dual-threshold approach:

• Pixels exceeding an upper threshold ($T_{high}$) are fully marked as glint ($M(x, y) = 1$).
• Pixels below a lower threshold ($T_{low}$) are classified as glint-free ($M(x, y) = 0$).
• For pixels within these bounds, mask values are linearly interpolated:

$$M(x, y) = \begin{cases} 1, & V(x, y) > T_{high} \ \frac{V(x, y) - T_{low}}{T_{high} - T_{low}}, & T_{low} \leq V(x, y) \leq T_{high} \ 0, & V(x, y) < T_{low} \ \end{cases}$$

The resulting mask is continuous-valued and one-channel. It is concatenated with the three RGB channels, forming a four-channel input tensor. This integration is termed “early fusion,” as the mask is incorporated from the very first processing stage, allowing downstream modules—specifically, the diffusion-based restoration network—to leverage explicit glint localization for targeted image correction (Kromer et al., 11 Aug 2025).

2. Architectural Integration in Diffusion-Based Restoration

The early-fusion sun-glint mask is operationalized in a lightweight diffusion-based framework, specifically a variant of the ResShift model. In this setup, the concatenated four-channel input (three RGB + mask) feeds directly into the image restoration pipeline. The model is thereby equipped to focus its denoising, deblurring, and structural reconstruction processes preferentially on those regions identified as glint-corrupted, without globally degrading unaffected areas.

Standard restoration models typically treat optical distortions homogeneously, risking either under-correction or structural oversmoothing. The early-fusion approach instead localizes correction efforts, preserving fine seabed detail, and suppressing artifacts (such as glint and scattering) that impede both visual inspection and quantitative bathymetric analysis. A plausible implication is that similar fusion strategies can extend to other scene-specific distortions beyond glint.

3. Dataset Design for Sun-Glint Mask Training and Benchmarking

The methodological development and benchmarking of the early-fusion sun-glint mask strategy are grounded in the Sea-Undistort synthetic dataset. This dataset comprises 1200 paired 512x512 through-water scenes, each rendered in Blender with paired “distorted” (including glint, waves, scattering) and “non-distorted” (minimal artifacts) representations.

Crucial for the nuanced design and effective training of the sun-glint mask, Sea-Undistort includes extensive per-image metadata:

• Camera parameters: focal length, sensor dimensions, platform altitude
• Sun position: elevation angles
• Average scene depth

Controlled sun elevation parameters directly inform aggregation of the mask, allowing the training process to adapt to varying illumination and surface states. Supervised training is thus feasible, with the diffusion model learning to exploit the fused glint mask for spatially-informed correction unattainable in unpaired real aerial data.

4. Performance Evaluation and Quantitative Metrics

The enhanced diffusion model with early-fusion sun-glint mask (“ResShift+EF”) demonstrates significant improvements over baseline approaches when tested on both synthetic Sea-Undistort imagery and real aerial datasets:

• Perceptual restoration quality: Achieves the lowest LPIPS (improved perceptual similarity) and the highest CLIPIQA, MUSIQ scores.
• Fidelity: Maintains SSIM and PSNR values close to vanilla ResShift, indicating that artifact suppression does not introduce loss of overall structure.
• DSM reconstruction: Under the Structure-from-Motion and Multi-View Stereo (SfM–MVS) pipeline, ResShift+EF yields denser pixel reconstructions, particularly in areas with stronger distortions at greater depth.
• Bathymetric mapping: Produces DSMs with bathymetric errors (RMSE, MAE, STD) comparable to non-restored imagery, but enhances coverage in challenging regimes.
• Learning-based depth prediction: Lowers RMSE and MAE compared to both original and other restoration baselines, evidencing enhanced depth estimation.

This performance underscores the efficacy of mask-guided restoration for practical mapping scenarios.

5. Contextual Applications and Implications

The methodology is directly applicable to high-resolution mapping of shallow seabeds, facilitating accurate, complete DSMs in regions historically plagued by glint-induced data loss. Further potential applications include:

• Remote sensing domains where surface reflections compromise structure identification (e.g., coastal monitoring, inland water quality inspection)
• Extension to aerial photographic surveys in variable atmospheric environments

The paradigm of early fusion—integrating auxiliary spatial information (here, the glint mask) at base-level input—suggests broader utility for distortion-aware restoration, possibly with masks addressing other context-specific artifacts.

Potential limitations include the environmental specificity of threshold parameters and interpolation rules; these may require dataset or sensor-adaptive tuning. The synthetic nature of the Sea-Undistort dataset, while invaluable for controlled supervision, may leave some residual gaps for real-world deployment, suggesting continued refinement and possible transfer learning strategies.

6. Limitations and Future Directions

Deployment of the early-fusion sun-glint mask outside controlled synthetic conditions raises several considerations. Glint mask parameters (thresholds, interpolation) might need calibration for heterogeneous water surface states and sensor characteristics. Real-world scenes could present composite distortions not fully replicable in synthetic data, potentially necessitating hybrid or unsupervised adaptation techniques.

This suggests exploration of adaptive thresholding and fusion mechanisms, as well as domain adaptation methods, for maximizing real-environment applicability. The underlying concept of early-fusion integration remains broadly extensible to other masking needs in restoration pipelines, offering a framework for spatially-aware, context-driven image correction across remote sensing modalities.