Osmosis: RGBD Diffusion Prior for Underwater Image Restoration (2403.14837v2)

Abstract: Underwater image restoration is a challenging task because of water effects that increase dramatically with distance. This is worsened by lack of ground truth data of clean scenes without water. Diffusion priors have emerged as strong image restoration priors. However, they are often trained with a dataset of the desired restored output, which is not available in our case. We also observe that using only color data is insufficient, and therefore augment the prior with a depth channel. We train an unconditional diffusion model prior on the joint space of color and depth, using standard RGBD datasets of natural outdoor scenes in air. Using this prior together with a novel guidance method based on the underwater image formation model, we generate posterior samples of clean images, removing the water effects. Even though our prior did not see any underwater images during training, our method outperforms state-of-the-art baselines for image restoration on very challenging scenes. Our code, models and data are available on the project website.

• The paper introduces an RGBD diffusion prior that leverages in-air RGBD data to improve underwater image restoration.
• It integrates a revised physical image formation model that accounts for wavelength-dependent underwater distortions.
• The approach simultaneously optimizes unknown parameters during sampling, outperforming existing methods on real-world benchmarks.

RGBD Diffusion Prior for Underwater Image Restoration

The paper, titled "Osmosis: RGBD Diffusion Prior for Underwater Image Restoration," introduces an advanced methodology for tackling the challenging issue of underwater image restoration. The authors propose a novel approach leveraging diffusion models trained on RGBD data from in-air scenes, addressing the intricate challenges posed by underwater image degradation.

Key Contributions and Methodology

The authors identify two main issues obstructing effective underwater image restoration: the lack of ground truth data in underwater environments and the inadequacy of color data alone for robust restorations. To circumvent these issues, they present the following methodological advancements:

1. Training an RGBD Prior: They employ RGBD datasets from natural outdoor scenes to train an unconditional diffusion model prior. This joint consideration of both color and depth data allows the proposed method to capture the intricate correlations that naturally exist between these two modalities, enhancing the restoration capability.
2. Guidance via Physical Image Formation Model: The method incorporates the underwater image formation model, a revised variant that accounts for wavelength-dependent effects. By embedding this model within the diffusion prior framework, they enable posterior sampling that effectively reconstructs and removes the visual distortions caused by underwater environments.
3. Parameter Optimization: Unknown parameters such as the attenuation and backscatter coefficients are simultaneously optimized during sampling. This involves iterative adjustment based on a likelihood term derived from the physical model.
4. Evaluations and Comparisons: The paper conducts thorough evaluations against a set of baseline methods, demonstrating superior performance both qualitatively and quantitatively on real-world datasets and synthetic benchmarks.

Implications and Future Research Directions

The work implies significant advancements in the field of underwater image restoration. By creating an effective strategy without relying on underwater ground truth datasets, the methodology indicates promising scalable solutions for real-world applications such as marine research, underwater surveillance, and oceanic exploration.

Additionally, the use of a jointly trained RGBD diffusion model encourages further exploration of multi-modal priors in image restoration tasks beyond the underwater domain, such as in scenarios involving intricate light and depth interactions.

The paper opens possibilities for future research, particularly in enhancing model efficiency for higher resolution restorations and minimizing computational costs. Extending this approach to additional environmental models and exploring unsupervised refinement methods can bolster its applicability to a broader range of underwater settings.

In summary, the authors adeptly navigate the complex challenges of underwater image processing, offering a methodologically robust and theoretically impactful contribution to the domain, potentially guiding future explorations within image restoration and beyond.