Papers
Topics
Authors
Recent
Search
2000 character limit reached

FieldNet: Efficient Real-Time Shadow Removal for Enhanced Vision in Field Robotics

Published 13 Mar 2024 in cs.CV | (2403.08142v2)

Abstract: Shadows significantly hinder computer vision tasks in outdoor environments, particularly in field robotics, where varying lighting conditions complicate object detection and localisation. We present FieldNet, a novel deep learning framework for real-time shadow removal, optimised for resource-constrained hardware. FieldNet introduces a probabilistic enhancement module and a novel loss function to address challenges of inconsistent shadow boundary supervision and artefact generation, achieving enhanced accuracy and simplicity without requiring shadow masks during inference. Trained on a dataset of 10,000 natural images augmented with synthetic shadows, FieldNet outperforms state-of-the-art methods on benchmark datasets (ISTD, ISTD+, SRD), with up to $9$x speed improvements (66 FPS on Nvidia 2080Ti) and superior shadow removal quality (PSNR: 38.67, SSIM: 0.991). Real-world case studies in precision agriculture robotics demonstrate the practical impact of FieldNet in enhancing weed detection accuracy. These advancements establish FieldNet as a robust, efficient solution for real-time vision tasks in field robotics and beyond.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (27)
  1. Towards Ghost-Free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN., in: Proc.~AAAI Conf. on Artificial Intelligence, pp. 10680–10687.
  2. Argan: Attentive recurrent generative adversarial network for shadow detection and removal, in: Proc.~IEEE Int’l Conf.~Computer Vision, pp. 10213–10222.
  3. Auto-exposure fusion for single-image shadow removal, in: Proc.~IEEE Int’l Conf.~Computer Vision and Pattern Recognition, pp. 10571–10580.
  4. ShadowFormer: Global Context Helps Shadow Removal, in: Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023. doi:10.1609/aaai.v37i1.25148.
  5. Paired regions for shadow detection and removal. IEEE Trans. on Pattern Analysis and Machine Intelligence 35, 2956–2967.
  6. Direction-Aware Spatial Context Features for Shadow Detection and Removal. IEEE Trans. on Pattern Analysis and Machine Intelligence 42, 2795–2808.
  7. Mask-ShadowGAN: Learning to remove shadows from unpaired data, in: Proc.~IEEE Int’l Conf.~Computer Vision, pp. 2472–2481.
  8. Learning from Synthetic Shadows for Shadow Detection and Removal. IEEE Transactions on Circuits and Systems for Video Technology 31, 4187–4197. doi:10.1109/TCSVT.2020.3047977.
  9. DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network, in: Proceedings of the IEEE International Conference on Computer Vision. doi:10.1109/ICCV48922.2021.00498.
  10. Perceptual losses for real-time style transfer and super-resolution, in: European conference on computer vision, pp. 694–711.
  11. C2ShadowGAN: cycle-in-cycle generative adversarial network for shadow removal using unpaired data. Applied Intelligence 53, 15067–15079. URL: https://link.springer.com/article/10.1007/s10489-022-04269-7, doi:10.1007/S10489-022-04269-7/FIGURES/5.
  12. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 .
  13. Quantifying the Carbon Emissions of Machine Learning URL: /green-ai/publications/2019-11-lacoste-quantifying.html.
  14. Shadow removal via shadow image decomposition, in: Proc.~IEEE Int’l Conf.~Computer Vision, pp. 8578–8587.
  15. From Shadow Segmentation to Shadow Removal, in: Proc.~IEEE European Conf.~Computer Vision.
  16. Physics-Based Shadow Image Decomposition for Shadow Removal. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 9088–9101. doi:10.1109/TPAMI.2021.3124934.
  17. AP Shadow Net: A Remote Sensing Shadow Removal Network Based on Atmospheric Transport and Poisson’s Equation. Entropy 2022, Vol. 24, Page 1301 24, 1301. URL: https://www.mdpi.com/1099-4300/24/9/1301/htmhttps://www.mdpi.com/1099-4300/24/9/1301, doi:10.3390/E24091301.
  18. From Shadow Generation to Shadow Removal, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. doi:10.1109/CVPR46437.2021.00489.
  19. A Boundary-Aware Network for Shadow Removal. IEEE Transactions on Multimedia 25. doi:10.1109/TMM.2022.3214422.
  20. Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classification, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. doi:10.1109/CVPR46437.2021.00936.
  21. Deshadownet: A multi-context embedding deep network for shadow removal, in: Proc.~IEEE Int’l Conf.~Computer Vision and Pattern Recognition, pp. 4067–4075.
  22. Style-Guided Shadow Removal, in: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). doi:10.1007/978-3-031-19800-7_21.
  23. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal, in: Proc.~IEEE Int’l Conf.~Computer Vision and Pattern Recognition, pp. 1788–1797.
  24. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 600–612.
  25. RIS-GAN: Explore residual and illumination with generative adversarial networks for shadow removal, in: AAAI 2020 - 34th AAAI Conference on Artificial Intelligence. doi:10.1609/aaai.v34i07.6979.
  26. Bijective Mapping Network for Shadow Removal, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5627–5636.
  27. Efficient Model-Driven Network for Shadow Removal .

Summary

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.