A fast horizon detector and a new annotated dataset for maritime video processing (2110.13694v4)
Abstract: Accurate and fast sea horizon detection is vital for tasks in autonomous navigation and maritime security, such as video stabilization, target region reduction, precise tracking, and obstacle avoidance. This paper introduces a novel sea horizon detector from RGB videos, focusing on rapid and effective sea noise suppression while preserving weak horizon edges. Line fitting methods are subsequently employed on filtered edges for horizon detection. We address the filtering problem by extracting line segments with a very low edge threshold, ensuring the detection of line segments even in low-contrast horizon conditions. We show that horizon line segments have simple and relevant properties in RGB images, which we exploit to suppress noisy segments. Then we use the surviving segments to construct a filtered edge map and infer the horizon from the filtered edges. We propose a careful incorporation of temporal information for horizon inference and experimentally show its effectiveness. We address the computational constraint by providing a vectorized implementation for efficient CPU execution, and leveraging image downsizing with minimal loss of accuracy on the original size. Moreover, we contribute a public horizon line dataset to enrich existing data resources. Our algorithm's performance is rigorously evaluated against state-of-the-art methods, and its components are validated through ablation experiments. Source code and dataset files are available at:
- “Video Processing From Electro-Optical Sensors for Object Detection and Tracking in a Maritime Environment: A Survey” In IEEE Transactions on Intelligent Transportation Systems 18.8, 2017, pp. 1993–2016 DOI: 10.1109/TITS.2016.2634580
- Miro Petković, Igor Vujović and Ivica Kuzmanić “An Overview on Horizon Detection Methods in Maritime Video Surveillance” In Transactions on Maritime Science 9.1, 2020, pp. 106–112 DOI: 10.7225/toms.v09.n01.010
- Yassir Zardoua, Abdelali Astito and Mohammed Boulaala “A survey on horizon detection algorithms for maritime video surveillance: advances and future techniques” In The Visual Computer 39.1, 2023, pp. 197–217 DOI: 10.1007/s00371-021-02321-0
- “Saliency-aware convolution neural network for ship detection in surveillance video” In IEEE Transactions on Circuits and Systems for Video Technology 30.3 IEEE, 2019, pp. 781–794
- “Fast infrared horizon detection algorithm based on gradient directional filtration” In J. Opt. Soc. Am. A 37.11 Optica Publishing Group, 2020, pp. 1795–1805 DOI: 10.1364/JOSAA.402620
- “Sea–sky line detection using gray variation differences in the time domain for unmanned surface vehicles” In Signal, Image and Video Processing 15.1, 2021, pp. 139–146 DOI: 10.1007/s11760-020-01733-0
- “Horizon Detection From Electro-Optical Sensors Under Maritime Environment” In IEEE Transactions on Instrumentation and Measurement 69.1, 2020, pp. 45–53 DOI: 10.1109/TIM.2019.2893008
- “A review of intelligent ship marine object detection based on RGB camera” In IET Image Processing Wiley Online Library, 2023
- “A robust dynamic programming algorithm to extract skyline in images for navigation” In Pattern Recognition Letters 26.2, 2005, pp. 221–230 DOI: https://doi.org/10.1016/j.patrec.2004.08.021
- “Resource Efficient Mountainous Skyline Extraction using Shallow Learning” In 2021 International Joint Conference on Neural Networks (IJCNN), 2021, pp. 1–9 DOI: 10.1109/IJCNN52387.2021.9533859
- “A Short Overview of Horizon Detection Methods Applied to Maritime Video Stabilization” In Advanced Intelligent Systems for Sustainable Development (AI2SD’2020) Cham: Springer International Publishing, 2022, pp. 857–864
- “A Horizon-Tracking Method for Shipboard Video Stabilization and Rectification” In Journal of Atmospheric and Oceanic Technology 32.1 Boston MA, USA: American Meteorological Society, 2015, pp. 164 –176 DOI: 10.1175/JTECH-D-14-00047.1
- Chengtao Cai, Xiangyu Weng and Qidan Zhu “Sea-skyline-based image stabilization of a buoy-mounted catadioptric omnidirectional vision system” In EURASIP Journal on Image and Video Processing 2018.1, 2018, pp. 1 DOI: 10.1186/s13640-017-0240-z
- “Detection and tracking of ships in open sea with rapidly moving buoy-mounted camera system” In Ocean Engineering 54, 2012, pp. 1–12 DOI: https://doi.org/10.1016/j.oceaneng.2012.06.028
- “Real-Time Monocular Obstacle Detection Based on Horizon Line and Saliency Estimation for Unmanned Surface Vehicles” In Mobile Networks and Applications 26.3, 2021, pp. 1372–1385 DOI: 10.1007/s11036-021-01752-2
- “A deep neural network method for detection and tracking ship for unmanned surface vehicle” In 2019 IEEE 8th Data Driven Control and Learning Systems Conference (DDCLS), 2019, pp. 1279–1283 IEEE
- “Infrared Small Dim Target Detection Under Maritime Near Sea-sky Line Based on Regional-Division Local Contrast Measure” In IEEE Geoscience and Remote Sensing Letters IEEE, 2023
- “Distance estimation for marine vehicles using a monocular video camera” In 2016 24th European Signal Processing Conference (EUSIPCO), 2016, pp. 2405–2409 IEEE
- Baozhi Jia, Rui Liu and Ming Zhu “Real-time obstacle detection with motion features using monocular vision” In The Visual Computer 31 Springer, 2015, pp. 281–293
- “Obstacle avoidance approaches for autonomous navigation of unmanned surface vehicles” In The Journal of Navigation 71.1 Cambridge University Press, 2018, pp. 241–256
- A. Samama “Innovative video analytics for maritime surveillance” In 2010 International WaterSide Security Conference, 2010 International WaterSide Security Conference, 2010, pp. 1–8 DOI: 10.1109/WSSC.2010.5730280
- Evgeny Gershikov, Tzvika Libe and Samuel Kosolapov “Horizon line detection in marine images: which method to choose?” In International Journal on Advances in Intelligent Systems 6.1 Citeseer, 2013
- Gui-Qiu Bao, Shen-Shu Xiong and Zhao-Ying Zhou “Vision-based horizon extraction for micro air vehicle flight control” In IEEE Transactions on Instrumentation and Measurement 54.3, 2005, pp. 1067–1072 DOI: 10.1109/TIM.2005.847234
- “A robust adaptive horizon recognizing algorithm based on projection” In Transactions of the Institute of Measurement and Control 33.6, 2011, pp. 734–751 DOI: 10.1177/0142331209342201
- “A Vision-Based Automatic Safe Landing-Site Detection System” In IEEE Transactions on Aerospace and Electronic Systems 49.1, 2013, pp. 294–311 DOI: 10.1109/TAES.2013.6404104
- “A Hierarchical Horizon Detection Algorithm” In IEEE Geoscience and Remote Sensing Letters 10.1, 2013, pp. 111–114 DOI: 10.1109/LGRS.2012.2194473
- Ilan Lipschutz, Evgeny Gershikov and Benjamin Milgrom “New methods for horizon line detection in infrared and visible sea images” In Int. J. Comput. Eng. Res 3.3 Citeseer, 2013, pp. 1197–1215
- Makoto Ohki, Michael E. Zervakis and Anastasios N. Venetsanopoulos “3-D Digital Filters” In Multidimensional Systems: Signal Processing and Modeling Techniques 69, Control and Dynamic Systems Academic Press, 1995, pp. 49–88 DOI: https://doi.org/10.1016/S0090-5267(05)80038-6
- Bart M Haar Romeny “Front-end vision and multi-scale image analysis: multi-scale computer vision theory and applications, written in mathematica” Springer Science & Business Media, 2008
- “MSCM-LiFe: Multi-scale cross modal linear feature for horizon detection in maritime images” In 2016 IEEE Region 10 Conference (TENCON), 2016 IEEE Region 10 Conference (TENCON), 2016, pp. 1366–1370 DOI: 10.1109/TENCON.2016.7848237
- “MuSCoWERT: multi-scale consistence of weighted edge Radon transform for horizon detection in maritime images” In J. Opt. Soc. Am. A 33.12 OSA, 2016, pp. 2491–2500 DOI: 10.1364/JOSAA.33.002491
- Chi Yoon Jeong, Hyun S Yang and KyeongDeok Moon “Fast horizon detection in maritime images using region-of-interest” In International Journal of Distributed Sensor Networks 14.7, 2018, pp. 1550147718790753 DOI: 10.1177/1550147718790753
- Chiyoon Jeong, Hyun S. Yang and KyeongDeok Moon “A novel approach for detecting the horizon using a convolutional neural network and multi-scale edge detection” In Multidimensional Systems and Signal Processing 30.3, 2019, pp. 1187–1204 DOI: 10.1007/s11045-018-0602-4
- Steven J. Dumble and Peter W. Gibbens “Horizon Profile Detection for Attitude Determination” In Journal of Intelligent & Robotic Systems 68.3, 2012, pp. 339–357 DOI: 10.1007/s10846-012-9684-7
- Terry Donald Cornall, Gregory Kenneth Egan and A Price “Aircraft attitude estimation from horizon video” In Electronics Letters 42.13 IET, 2006, pp. 744–745
- “Towards flight autonomy: Vision-based horizon detection for micro air vehicles” In Florida Conference on Recent Advances in Robotics 2002, 2002
- “Horizon Detection Using Machine Learning Techniques” In 2006 5th International Conference on Machine Learning and Applications (ICMLA’06), 2006 5th International Conference on Machine Learning and Applications (ICMLA’06), 2006, pp. 17–21 DOI: 10.1109/ICMLA.2006.25
- “Fast Image-Based Obstacle Detection From Unmanned Surface Vehicles” In IEEE Transactions on Cybernetics 46.3, 2016, pp. 641–654 DOI: 10.1109/TCYB.2015.2412251
- “Skyline localization for mountain images” In 2013 IEEE International Conference on Multimedia and Expo (ICME), 2013, pp. 1–6 DOI: 10.1109/ICME.2013.6607424
- “Automatic Peak Recognition for Mountain Images” In Advanced Technologies, Embedded and Multimedia for Human-centric Computing Dordrecht: Springer Netherlands, 2014, pp. 1115–1121
- “Elsd: Efficient line segment detector and descriptor” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2969–2978
- “LSD: A Fast Line Segment Detector with a False Detection Control” In IEEE Transactions on Pattern Analysis and Machine Intelligence 32.4, 2010, pp. 722–732 DOI: 10.1109/TPAMI.2008.300
- “EDLines: A real-time line segment detector with a false detection control” In Pattern Recognition Letters 32.13, 2011, pp. 1633–1642 DOI: https://doi.org/10.1016/j.patrec.2011.06.001
- Manzoor Ahmed Hashmani and Muhammad Umair “A Novel Visual-Range Sea Image Dataset for Sea Horizon Line Detection in Changing Maritime Scenes” In Journal of Marine Science and Engineering 10.2, 2022 DOI: 10.3390/jmse10020193
- “A Horizon Line Annotation Software for Streamlining Autonomous Sea Navigation Experiments” In Adv. Artif. Intell. Mach. Learn. 3.4, 2023, pp. 1768–1786 DOI: 10.54364/AAIML.2023.11102
- “Array programming with NumPy” In Nature 585.7825 Nature Publishing Group, 2020, pp. 357–362
- “OpenCV” In Dr. Dobb’s journal of software tools 3, 2000, pp. 2