Papers
Topics
Authors
Recent
Search
2000 character limit reached

Spatialyze: A Geospatial Video Analytics System with Spatial-Aware Optimizations

Published 7 Aug 2023 in cs.DB and cs.CV | (2308.03276v5)

Abstract: Videos that are shot using commodity hardware such as phones and surveillance cameras record various metadata such as time and location. We encounter such geospatial videos on a daily basis and such videos have been growing in volume significantly. Yet, we do not have data management systems that allow users to interact with such data effectively. In this paper, we describe Spatialyze, a new framework for end-to-end querying of geospatial videos. Spatialyze comes with a domain-specific language where users can construct geospatial video analytic workflows using a 3-step, declarative, build-filter-observe paradigm. Internally, Spatialyze leverages the declarative nature of such workflows, the temporal-spatial metadata stored with videos, and physical behavior of real-world objects to optimize the execution of workflows. Our results using real-world videos and workflows show that Spatialyze can reduce execution time by up to 5.3x, while maintaining up to 97.1% accuracy compared to unoptimized execution.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Aqeel Anwar. 2022. What are Intrinsic and Extrinsic Camera Parameters in Computer Vision? https://towardsdatascience.com/what-are-intrinsic-and-extrinsic-camera-parameters-in-computer-vision-7071b72fb8ec. Accessed: 2023-07-25.
  2. MIRIS: Fast Object Track Queries in Video. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD ’20). Association for Computing Machinery, New York, NY, USA, 1907–1921. https://doi.org/10.1145/3318464.3389692
  3. SkyQuery: An Aerial Drone Video Sensing Platform. In Proceedings of the 2021 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Chicago, IL, USA) (Onward! 2021). Association for Computing Machinery, New York, NY, USA, 56–67. https://doi.org/10.1145/3486607.3486750
  4. Favyen Bastani and Samuel Madden. 2022. OTIF: Efficient Tracker Pre-Processing over Large Video Datasets. In Proceedings of the 2022 International Conference on Management of Data (Philadelphia, PA, USA) (SIGMOD ’22). Association for Computing Machinery, New York, NY, USA, 2091–2104. https://doi.org/10.1145/3514221.3517835
  5. Simple online and realtime tracking. In 2016 IEEE International Conference on Image Processing (ICIP). 3464–3468. https://doi.org/10.1109/ICIP.2016.7533003
  6. Gary Bradski. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools 25, 11 (Nov. 2000), 120, 122–125. http://www.ddj.com/ftp/2000/2000_11/opencv.txt
  7. Mikel Broström. 2022. Real-time multi-camera multi-object tracker using YOLOv5 and StrongSORT with OSNet. https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet.
  8. nuScenes: A multimodal dataset for autonomous driving. https://doi.org/10.48550/ARXIV.1903.11027
  9. StrongSORT: Make DeepSORT Great Again. arXiv:2202.13514 [cs.CV]
  10. Scenic: A Language for Scenario Specification and Scene Generation. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (Phoenix, AZ, USA) (PLDI 2019). Association for Computing Machinery, New York, NY, USA, 63–78. https://doi.org/10.1145/3314221.3314633
  11. Scenic: a language for scenario specification and data generation. Machine Learning 112, 10 (01 Oct 2023), 3805–3849. https://doi.org/10.1007/s10994-021-06120-5
  12. A new performance measure and evaluation benchmark for road detection algorithms. In 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013). 1693–1700. https://doi.org/10.1109/ITSC.2013.6728473
  13. Demonstration of Apperception: A Database Management System for Geospatial Video Data. Proc. VLDB Endow. 14, 12 (oct 2021), 2767–2770. https://doi.org/10.14778/3476311.3476340
  14. Ross Girshick. 2015. Fast R-CNN. arXiv:1504.08083 [cs.CV]
  15. Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv:1311.2524 [cs.CV]
  16. Derek Gloudemans and Daniel B. Work. 2021. Fast Vehicle Turning-Movement Counting using Localization-based Tracking. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 4150–4159. https://doi.org/10.1109/CVPRW53098.2021.00469
  17. Digging Into Self-Supervised Monocular Depth Estimation. arXiv:1806.01260 [cs.CV]
  18. Timothy Griffin and Monica K. Miller. 2008. Child Abduction, AMBER Alert, and Crime Control Theater. Criminal Justice Review 33, 2 (2008), 159–176. https://doi.org/10.1177/0734016808316778 arXiv:https://doi.org/10.1177/0734016808316778
  19. VisualWorldDB: A DBMS for the Visual World. In Conference on Innovative Data Systems Research. 6.
  20. ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation. https://doi.org/10.5281/zenodo.7347926
  21. NoScope: Optimizing Neural Network Queries over Video at Scale. Proc. VLDB Endow. 10, 11 (aug 2017), 1586–1597. https://doi.org/10.14778/3137628.3137664
  22. VIVA: An End-to-End System for Interactive Video Analytics. In 12th Conference on Innovative Data Systems Research, CIDR 2022, Chaminade, CA, USA, January 9-12, 2022. www.cidrdb.org, 9. https://www.cidrdb.org/cidr2022/papers/p75-kang.pdf
  23. EagerMOT: 3D Multi-Object Tracking via Sensor Fusion. arXiv:2104.14682 [cs.CV]
  24. Querying Labelled Data with Scenario Programs for Sim-to-Real Validation. arXiv:2112.00206 [cs.CV]
  25. H. W. Kuhn. 1955. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly 2, 1-2 (1955), 83–97. https://doi.org/10.1002/nav.3800020109 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/nav.3800020109
  26. Jack B. Kuipers. 1999. Quaternions and rotation sequences : a primer with applications to orbits, aerospace, and virtual reality. Princeton Univ. Press, Princeton, NJ. http://www.worldcat.org/title/quaternions-and-rotation-sequences-a-primer-with-applications-to-orbits-aerospace-and-virtual-reality/oclc/246446345
  27. Laura Leal-Taixé. 2014. Multiple object tracking with context awareness. (11 2014), 15–25.
  28. HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. International Journal of Computer Vision 129, 2 (oct 2020), 548–578. https://doi.org/10.1007/s11263-020-01375-2
  29. Satellites can reveal global extent of forced labor in the world’s fishing fleet. Proceedings of the National Academy of Sciences 118, 3 (2021), e2016238117. https://doi.org/10.1073/pnas.2016238117 arXiv:https://www.pnas.org/doi/pdf/10.1073/pnas.2016238117
  30. Illuminating dark fishing fleets in North Korea. Science Advances 6, 30 (2020), eabb1197. https://doi.org/10.1126/sciadv.abb1197 arXiv:https://www.science.org/doi/pdf/10.1126/sciadv.abb1197
  31. Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. arXiv:1804.02767 [cs.CV]
  32. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv:1506.01497 [cs.CV]
  33. Optimizing Video Analytics with Declarative Model Relationships. Proc. VLDB Endow. 16, 3 (nov 2022), 447–460. https://doi.org/10.14778/3570690.3570695
  34. Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds. arXiv:1904.07537 [cs.CV]
  35. Jack Sklansky. 1982. Finding the Convex Hull of a Simple Polygon. Pattern Recogn. Lett. 1, 2 (dec 1982), 79–83. https://doi.org/10.1016/0167-8655(82)90016-2
  36. Robert Sparks. 2023. 11 Security Camera Statistics & Data - 2023 update. https://opticsmag.com/security-camera-statistics/
  37. Suramya Tomar. 2006. Converting Video Formats with FFmpeg. Linux J. 2006, 146 (jun 2006), 10.
  38. Simple Online and Realtime Tracking with a Deep Association Metric. arXiv:1703.07402 [cs.CV]
  39. Simon Wright. 2023. Autonomous cars generate more than 300 TB of data per year. https://www.tuxera.com/blog/autonomous-cars-300-tb-of-data-per-year/
  40. EVA: A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized Views. In Proceedings of the 2022 International Conference on Management of Data (Philadelphia, PA, USA) (SIGMOD ’22). Association for Computing Machinery, New York, NY, USA, 602–616. https://doi.org/10.1145/3514221.3526142
  41. Zhengyou Zhang. 2014. Camera Parameters (Intrinsic, Extrinsic). Springer US, Boston, MA, 81–85. https://doi.org/10.1007/978-0-387-31439-6_152
  42. Omni-Scale Feature Learning for Person Re-Identification. arXiv:1905.00953 [cs.CV]
  43. MobilityDB: A Mobility Database Based on PostgreSQL and PostGIS. ACM Trans. Database Syst. 45, 4, Article 19 (dec 2020), 42 pages. https://doi.org/10.1145/3406534
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up