Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How to Augment for Atmospheric Turbulence Effects on Thermal Adapted Object Detection Models? (2405.06383v1)

Published 10 May 2024 in cs.CV

Abstract: Atmospheric turbulence poses a significant challenge to the performance of object detection models. Turbulence causes distortions, blurring, and noise in images by bending and scattering light rays due to variations in the refractive index of air. This results in non-rigid geometric distortions and temporal fluctuations in the electromagnetic radiation received by optical systems. This paper explores the effectiveness of turbulence image augmentation techniques in improving the accuracy and robustness of thermal-adapted and deep learning-based object detection models under atmospheric turbulence. Three distinct approximation-based turbulence simulators (geometric, Zernike-based, and P2S) are employed to generate turbulent training and test datasets. The performance of three state-of-the-art deep learning-based object detection models: RTMDet-x, DINO-4scale, and YOLOv8-x, is employed on these turbulent datasets with and without turbulence augmentation during training. The results demonstrate that utilizing turbulence-specific augmentations during model training can significantly improve detection accuracy and robustness against distorted turbulent images. Turbulence augmentation enhances performance even for a non-turbulent test set.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (93)
  1. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  2. Contrastive learning rivals masked image modeling in fine-tuning via feature distillation. arXiv preprint arXiv:2205.14141, 2022.
  3. Image as a foreign language: Beit pretraining for all vision and vision-language tasks. arXiv preprint arXiv:2208.10442, 2022.
  4. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022.
  5. Exploring plain vision transformer backbones for object detection. arXiv preprint arXiv:2203.16527, 2022.
  6. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
  7. Photorealistic text-to-image diffusion models with deep language understanding. arXiv: 2205.11487, 2022.
  8. A survey on contrastive self-supervised learning. Technologies, 9(1):2, 2020.
  9. S4l: Self-supervised semi-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1476–1485, 2019.
  10. A survey on deep semi-supervised learning. arXiv preprint arXiv:2103.00550, 2021.
  11. A comprehensive survey of image augmentation techniques for deep learning. arXiv preprint arXiv:2205.01491, 2022.
  12. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1–48, 2019.
  13. Data augmentation for brain-tumor segmentation: a review. Frontiers in computational neuroscience, 13:83, 2019.
  14. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781, 2019.
  15. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  16. Scaled-yolov4: Scaling cross stage partial network. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 13029–13038, 2021.
  17. Mdfn: Multi-scale deep feature learning network for object detection. Pattern Recognition, 100:107149, 2020.
  18. You only learn one representation: Unified network for multiple tasks. arXiv preprint arXiv:2105.04206, 2021.
  19. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
  20. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 10347–10357. PMLR, 2021.
  21. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8697–8710, 2018.
  22. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.
  23. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023–6032, 2019.
  24. Attentive cutmix: An enhanced data augmentation approach for deep learning based image classification. arXiv preprint arXiv:2003.13048, 2020.
  25. Puzzle mix: Exploiting saliency and local statistics for optimal mixup. In International Conference on Machine Learning, pages 5275–5285. PMLR, 2020.
  26. Manifold mixup: Better representations by interpolating hidden states. In International conference on machine learning, pages 6438–6447. PMLR, 2019.
  27. Saliencymix: A saliency guided data augmentation strategy for better regularization. arXiv preprint arXiv:2006.01791, 2020.
  28. Snapmix: Semantically proportional mixing for augmenting fine-grained data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1628–1636, 2021.
  29. Ricap: Random image cropping and patching data augmentation for deep cnns. In Asian conference on machine learning, pages 786–798. PMLR, 2018.
  30. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020.
  31. Synthetic data augmentation using gan for improved liver lesion classification. In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pages 289–293. IEEE, 2018.
  32. Chest x-ray generation and data augmentation for cardiovascular abnormality classification. In Medical imaging 2018: Image processing, volume 10574, pages 415–420. SPIE, 2018.
  33. Ida-gan: a novel imbalanced data augmentation gan. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 8299–8305. IEEE, 2021.
  34. Mfc-gan: class-imbalanced dataset classification using multiple fake class generative adversarial network. Neurocomputing, 361:212–221, 2019.
  35. Bagan: Data augmentation with balancing gan. arXiv preprint arXiv:1803.09655, 2018.
  36. Effective data generation for imbalanced learning using conditional generative adversarial networks. Expert Systems with applications, 91:464–471, 2018.
  37. Auggan: Cross domain adaptation with gan-based data augmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 718–731, 2018.
  38. Data augmentation using conditional generative adversarial networks for leaf counting in arabidopsis plants. In BMVC, page 324, 2018.
  39. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231, 2018.
  40. Simplified unsupervised image translation for semantic segmentation adaptation. Pattern Recognition, 105:107343, 2020.
  41. Emotion classification with data augmentation using generative adversarial networks. In Pacific-Asia conference on knowledge discovery and data mining, pages 349–360. Springer, 2018.
  42. Generative adversarial network with multi-branch discriminator for imbalanced cross-species image-to-image translation. Neural Networks, 141:355–371, 2021.
  43. Delta-encoder: an effective sample synthesis method for few-shot object recognition. Advances in neural information processing systems, 31, 2018.
  44. Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04340, 2017.
  45. Stylemix: Separating content and style for enhanced data augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14862–14870, 2021.
  46. Technique for simulating anisoplanatic image formation over long horizontal paths. Optical Engineering, 51(10):101704–101704, 2012.
  47. Turbulence-induced 2d correlated image distortion. In 2017 IEEE International Conference on Computational Photography (ICCP), pages 1–13. IEEE, 2017.
  48. Simulating anisoplanatic turbulence by sampling intermodal and spatially correlated zernike coefficients. Optical Engineering, 59(8):083101–083101, 2020.
  49. Real-time stabilization of long range observation system turbulent video. Journal of Real-Time Image Processing, 2:11–22, 2007.
  50. Removing atmospheric turbulence via space-invariant deconvolution. IEEE transactions on pattern analysis and machine intelligence, 35(1):157–170, 2012.
  51. Accelerating atmospheric turbulence simulation via learned phase-to-space transform. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14759–14768, 2021.
  52. Augmentation of atmospheric turbulence effects on thermal adapted object detection models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 241–248, 2022.
  53. Colorful cutout: Enhancing image data augmentation with curriculum learning. arXiv preprint arXiv:2403.20012, 2024.
  54. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 113–123, 2019.
  55. Active object localization with deep reinforcement learning. In Proceedings of the IEEE international conference on computer vision, pages 2488–2496, 2015.
  56. Adaptive data augmentation for image classification. In 2016 IEEE international conference on image processing (ICIP), pages 3688–3692. Ieee, 2016.
  57. Learning to compose domain-specific transformations for data augmentation. Advances in neural information processing systems, 30, 2017.
  58. Adversarial autoaugment. arXiv preprint arXiv:1912.11188, 2019.
  59. Jointly optimize data augmentation and network training: Adversarial data augmentation in human pose estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2226–2234, 2018.
  60. Imaging through turbulence. CRC press, 1996.
  61. Learning to restore images degraded by atmospheric turbulence using uncertainty. In 2021 IEEE International Conference on Image Processing (ICIP), pages 1694–1698. IEEE, 2021.
  62. Video stabilization of atmospheric turbulence distortion. Inverse Probl. Imaging, 7(3):839–861, 2013.
  63. Single frame atmospheric turbulence mitigation: A benchmark study and a new physics-inspired transformer model. In European Conference on Computer Vision, pages 430–446. Springer, 2022.
  64. Simultaneous video stabilization and moving object detection in turbulence. IEEE transactions on pattern analysis and machine intelligence, 35(2):450–462, 2012.
  65. Detecting and tracking moving objects in long-distance imaging through turbulent medium. Applied optics, 53(6):1181–1190, 2014.
  66. Uiu-net: U-net in u-net for infrared small object detection. IEEE Transactions on Image Processing, 32:364–376, 2022.
  67. Yolo-firi: Improved yolov5 for infrared image object detection. IEEE access, 9:141861–141875, 2021.
  68. Tirnet: Object detection in thermal infrared images for autonomous driving. Applied Intelligence, 51:1244–1261, 2021.
  69. Object detection from uav thermal infrared images and videos using yolo models. International Journal of Applied Earth Observation and Geoinformation, 112:102912, 2022.
  70. A multi-task framework for infrared small target detection and segmentation. IEEE Transactions on Geoscience and Remote Sensing, 60:1–9, 2022.
  71. Thermal infrared single image dehazing and blind image quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 459–469, 2023.
  72. Detection-friendly dehazing: Object detection in real-world hazy scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  73. Robust moving objects detection in long-distance imaging through turbulent medium. Infrared Physics & Technology, 100:87–98, 2019.
  74. The comprehensive art of atmospheric turbulence mitigation methodologies for visible and infrared sequences. In Advances in Information Communication Technology and Computing: Proceedings of AICTC 2021, pages 145–153. Springer, 2022.
  75. Dynamic turbulence mitigation for long-range imaging in the presence of large moving objects. EURASIP journal on image and video processing, 2019:1–22, 2019.
  76. At-ddpm: Restoring faces degraded by atmospheric turbulence using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3434–3443, 2023.
  77. Evaluation of neural network algorithms for atmospheric turbulence mitigation. In Signal Processing, Sensor/Information Fusion, and Target Recognition XXXI, volume 12122, pages 223–236. SPIE, 2022.
  78. Subsampled turbulence removal network. arXiv preprint arXiv:1807.04418, 2018.
  79. Rtmdet: An empirical study of designing real-time object detectors. arXiv preprint arXiv:2212.07784, 2022.
  80. Ultralytics yolov8. 2023.
  81. Andreas Quirrenbach. The effects of atmospheric turbulence on astronomical observations. A. Extrasolar planets. Saas-Fee Advanced Course, 31(137):137, 2006.
  82. Surveillance in long-distance turbulence-degraded videos. In Electro-Optical Remote Sensing, Photonic Technologies, and Applications VII; and Military Applications in Hyperspectral Imaging and High Spatial Resolution Sensing, volume 8897, pages 26–31. SPIE, 2013.
  83. Analysis regarding the effects of atmospheric turbulence on aircraft dynamics. INCAS Bulletin, 8(2):123, 2016.
  84. Imaging through turbulence. CRC press, 2018.
  85. Robert J Noll. Zernike polynomials and atmospheric turbulence. JOsA, 66(3):207–211, 1976.
  86. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155, 2019.
  87. MMYOLO Contributors. MMYOLO: OpenMMLab YOLO series toolbox and benchmark. https://github.com/open-mmlab/mmyolo, 2022.
  88. FLIR. Free flir thermal dataset for algorithm training. https://www.flir.com/oem/adas/adas-dataset-form/, 2020.
  89. Sstn: Self-supervised domain adaptation thermal object detection for autonomous driving. In 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 206–213. IEEE, 2021.
  90. Object detection in thermal spectrum for advanced driver-assistance systems (adas). IEEE Access, 9:156465–156481, 2021.
  91. A comprehensive survey on object detection yolo. Proceedings http://ceur-ws. org ISSN, 1613:0073, 2023.
  92. Object detection in 20 years: A survey. Proceedings of the IEEE, 2023.
  93. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now