Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Semi-supervised Underwater Image Enhancement Using A Physics-Aware Triple-Stream Network (2307.11470v3)

Published 21 Jul 2023 in cs.CV

Abstract: Underwater images normally suffer from degradation due to the transmission medium of water bodies. Both traditional prior-based approaches and deep learning-based methods have been used to address this problem. However, the inflexible assumption of the former often impairs their effectiveness in handling diverse underwater scenes, while the generalization of the latter to unseen images is usually weakened by insufficient data. In this study, we leverage both the physics-based Image Formation Model (IFM) and deep learning techniques for Underwater Image Enhancement (UIE). To this end, we propose a novel Physics-Aware Triple-Stream Underwater Image Enhancement Network, i.e., PATS-UIENet, which comprises a Direct Signal Transmission Estimation Steam (D-Stream), a Backscatter Signal Transmission Estimation Steam (B-Stream) and an Ambient Light Estimation Stream (A-Stream). This network fulfills the UIE task by explicitly estimating the degradation parameters of a revised IFM. We also adopt an IFM-inspired semi-supervised learning framework, which exploits both the labeled and unlabeled images, to address the issue of insufficient data. To our knowledge, such a physics-aware deep network and the IFM-inspired semi-supervised learning framework have not been used for the UIE task before. Our method performs better than, or at least comparably to, sixteen baselines across six testing sets in the degradation estimation and UIE tasks. These promising results should be due to the fact that the proposed method can not only model the degradation but also learn the characteristics of diverse underwater scenes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Recovery of underwater visibility and structure by polarization analysis. IEEE Journal of Oceanic Engineering, 30(3):570–587, 2005.
  2. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12):2341–2353, 2010.
  3. Transmission estimation in underwater single images. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 825–830. 2013.
  4. Generalization of the dark channel prior for single image restoration. IEEE Transactions on Image Processing, 27(6):2856–2868, 2018.
  5. Underwater image enhancement by wavelength compensation and dehazing. IEEE Transactions on Image Processing, 21(4):1756–1769, 2011.
  6. Initial results in underwater single image dehazing. In Oceans 2010 Mts/IEEE Seattle, pages 1–8. IEEE, 2010.
  7. Non-local image dehazing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1674–1682. 2016.
  8. Diving into haze-lines: Color restoration of underwater images. In Proc. British Machine Vision Conference (BMVC), vol. 1. 2017.
  9. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition. In MultiMedia Modeling: 24th International Conference, MMM 2018, Bangkok, Thailand, February 5-7, 2018, Proceedings, Part I 24, pages 453–465. Springer, 2018.
  10. Underwater image enhancement using an integrated colour model. IAENG International Journal of Computer Science, 34(2), 2007.
  11. Land, E. H. The retinex theory of color vision. Scientific American, page 108–128, 2010.
  12. Underwater image enhancement via extended multi-scale retinex. Neurocomputing, 245:1–9, 2017.
  13. Enhancing underwater images and videos by fusion. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 81–88. IEEE, 2012.
  14. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognition, 98:107038, 2020.
  15. Emerging from water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal processing letters, 25(3):323–327, 2018.
  16. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29:4376–4389, 2019.
  17. Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Transactions on Image Processing, 30:4985–5000, 2021.
  18. Sguie-net: Semantic attention guided underwater image enhancement with multi-scale perception. IEEE Transactions on Image Processing, 31:6816–6830, 2022.
  19. Estimation of ambient light and transmission map with common convolutional architecture. In OCEANS 2016 MTS/IEEE Monterey, pages 1–7. IEEE, 2016.
  20. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  21. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 2223–2232. 2017.
  22. Enhancing underwater imagery using generative adversarial networks. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7159–7165. IEEE, 2018.
  23. Zero-shot single image restoration through controlled perturbation of koschmieder’s model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16205–16215. 2021.
  24. Unsupervised underwater image restoration: From a homology perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pages 643–651. 2022.
  25. Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8):2822–2837, 2020.
  26. Zuiderveld, K. Contrast Limited Adaptive Histogram Equalization, page 474–485. Academic Press Professional, Inc., USA, 1994.
  27. Color balance and fusion for underwater image enhancement. IEEE Transactions on Image Processing, 27(1):379–393, 2017.
  28. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. Ieee, 2009.
  29. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017.
  30. Girshick, R. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440–1448. 2015.
  31. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, pages 2961–2969. 2017.
  32. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition Workshops, pages 136–144. 2017.
  33. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1833–1844. 2021.
  34. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5728–5739. 2022.
  35. An imaging information estimation network for underwater image color restoration. IEEE Journal of Oceanic Engineering, 46(4):1228–1239, 2021.
  36. Underwater image enhancement using a multiscale dense generative adversarial network. IEEE Journal of Oceanic Engineering, 45(3):862–870, 2019.
  37. Contrastive semi-supervised learning for underwater image restoration via reliable bank. arXiv preprint arXiv:2303.09101, 2023.
  38. Mobley, C. D. Light and water: radiative transfer in natural waters. Academic press, 1994.
  39. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7132–7141. 2018.
  40. An image is worth 16x16 words: Transformers for image recognition at scale. ArXiv, abs/2010.11929, 2021.
  41. Conformer: Local features coupling global representations for visual recognition. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 357–366, 2021.
  42. Buchsbaum, G. A spatial processor model for object colour perception. Journal of The Franklin Institute-engineering and Applied Mathematics, 310:1–26, 1980.
  43. Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light. IEEE Transactions on Circuits and Systems for Video Technology, 30(12):4861–4875, 2020.
  44. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  45. Human-visual-system-inspired underwater image quality measures. IEEE Journal of Oceanic Engineering, 41(3):541–551, 2015.
  46. An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 24(12):6062–6071, 2015.
  47. Fixing weight decay regularization in adam. ArXiv, abs/1711.05101, 2017.
Citations (2)

Summary

We haven't generated a summary for this paper yet.