Online Informative Sampling using Semantic Features in Underwater Environments (2402.03636v1)
Abstract: The underwater world remains largely unexplored, with Autonomous Underwater Vehicles (AUVs) playing a crucial role in sub-sea explorations. However, continuous monitoring of underwater environments using AUVs can generate a significant amount of data. In addition, sending live data feed from an underwater environment requires dedicated on-board data storage options for AUVs which can hinder requirements of other higher priority tasks. Informative sampling techniques offer a solution by condensing observations. In this paper, we present a semantically-aware online informative sampling (ON-IS) approach which samples an AUV's visual experience in real-time. Specifically, we obtain visual features from a fine-tuned object detection model to align the sampling outcomes with the desired semantic information. Our contributions are (a) a novel Semantic Online Informative Sampling (SON-IS) algorithm, (b) a user study to validate the proposed approach and (c) a novel evaluation metric to score our proposed algorithm with respect to the suggested samples by human subjects
- “Video Summarization using Deep Semantic Features,” CoRR, vol. abs/1609.08758, 2016.
- “Autonomous adaptive exploration using realtime online spatiotemporal topic modeling,” The International Journal of Robotics Research, vol. 33, no. 4, pp. 645–657, Apr. 2014, Publisher: SAGE Publications Ltd STM.
- “Efficient on-line data summarization using extremum summaries,” in 2012 IEEE International Conference on Robotics and Automation, May 2012, pp. 3490–3496, ISSN: 1050-4729.
- Yogesh Girdhar, Unsupervised Semantic Perception, Summarization, and Autonomous Exploration, for Robots in Unstructured Environments List of Tables, Ph.D. thesis, McGill University, 2014.
- “Real-time summarization of user-generated videos based on semantic recognition,” in Proceedings of the 22nd ACM international conference on Multimedia, 2014, pp. 849–852.
- “Comprehensive Video Understanding: Video Summarization with Content-Based Video Recommender Design,” in 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), Oct. 2019, pp. 1562–1569, IEEE.
- “Orb: An efficient alternative to sift or surf,” in 2011 International Conference on Computer Vision, 2011, pp. 2564–2571.
- “Online Video Summarization: Predicting Future to Better Summarize Present,” in 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Jan. 2019, pp. 471–480, ISSN: 1550-5790.
- “Creating summaries from user videos,” in Computer Vision – ECCV 2014, David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, Eds., Cham, 2014, pp. 505–520, Springer International Publishing.
- “VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method,” Pattern Recognition Letters, vol. 32, no. 1, pp. 56–68, 2011, Image Processing, Computer Vision and Pattern Recognition in Latin America.
- “Unsupervised video summarization via dynamic modeling-based hierarchical clustering,” in 2013 12th International Conference on Machine Learning and Applications, 2013, vol. 2, pp. 303–308.
- “Video summarization via minimum sparse reconstruction,” Pattern Recognition, vol. 48, no. 2, pp. 522–533, 2015.
- Y.-G. Jiang, “Super: Towards real-time event recognition in internet videos,” in Proceedings of ICMR, 2012.
- “End-to-End Object Detection with Transformers,” CoRR, vol. abs/2005.12872, 2020.
- “Faster r-cnn: Towards real-time object detection with region proposal networks,” 2016.
- “Attention is all you need,” 2017.
- “Deep residual learning for image recognition,” CoRR, vol. abs/1512.03385, 2015.
- “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
- “Microsoft COCO: Common Objects in Context,” CoRR, vol. abs/1405.0312, 2014.
- “Detection of marine animals in a new underwater dataset with varying visibility,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019.