Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
91 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
o3 Pro
5 tokens/sec
GPT-4.1 Pro
15 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
Gemini 2.5 Flash Deprecated
12 tokens/sec
2000 character limit reached

BiSwift: Bandwidth Orchestrator for Multi-Stream Video Analytics on Edge (2312.15740v2)

Published 25 Dec 2023 in cs.NI, cs.CV, and cs.LG

Abstract: High-definition (HD) cameras for surveillance and road traffic have experienced tremendous growth, demanding intensive computation resources for real-time analytics. Recently, offloading frames from the front-end device to the back-end edge server has shown great promise. In multi-stream competitive environments, efficient bandwidth management and proper scheduling are crucial to ensure both high inference accuracy and high throughput. To achieve this goal, we propose BiSwift, a bi-level framework that scales the concurrent real-time video analytics by a novel adaptive hybrid codec integrated with multi-level pipelines, and a global bandwidth controller for multiple video streams. The lower-level front-back-end collaborative mechanism (called adaptive hybrid codec) locally optimizes the accuracy and accelerates end-to-end video analytics for a single stream. The upper-level scheduler aims to accuracy fairness among multiple streams via the global bandwidth controller. The evaluation of BiSwift shows that BiSwift is able to real-time object detection on 9 streams with an edge device only equipped with an NVIDIA RTX3070 (8G) GPU. BiSwift improves 10%$\sim$21% accuracy and presents 1.2$\sim$9$\times$ throughput compared with the state-of-the-art video analytics pipelines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” in arXiv:2004.10934, 2020.
  2. N. Ahn, B. Kang, and K.-A. Sohn, “Fast, accurate, and lightweight super-resolution with cascading residual network,” in Proc. of ECCV, 2018, pp. 252–268.
  3. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” Advances in Neural Information Processing Systems, vol. 28, 2015.
  4. H. Wang, X. Jiang, H. Ren, Y. Hu, and S. Bai, “Swiftnet: Real-time video object segmentation,” in Proc. of CVPR, 2021, pp. 1296–1305.
  5. K. Du, A. Pervaiz, X. Yuan, A. Chowdhery, Q. Zhang, H. Hoffmann, and J. Jiang, “Server-driven video streaming for deep learning inference,” in Proc. of ACM SIGCOMM, 2020, pp. 557–570.
  6. Y. Li, A. Padmanabhan, P. Zhao, Y. Wang, G. H. Xu, and R. Netravali, “Reducto: On-camera filtering for resource-efficient real-time video analytics,” in Proc. of ACM SIGCOMM, 2020, pp. 359–376.
  7. A. Padmanabhan, N. Agarwal, A. Iyer, G. Ananthanarayanan, Y. Shu, N. Karianakis, G. H. Xu, and R. Netravali, “Gemel: Model merging for memory-efficient, real-time video analytics at the edge,” in Proc. of NSDI, 2023.
  8. Yoda, “Crossroad,” https://yoda.cs.uchicago.edu/videos/cross-road.mp4.
  9. “Speedtest market report,” https://www.speedtest.net/reports/zh/, 2021.
  10. S. G. Davani and N. J. Sarhan, “Experimental analysis of optimal bandwidth allocation in computer vision systems,” TCSVT, vol. 31, no. 10, pp. 4121–4130, 2021.
  11. C. Wang, S. Zhang, Y. Chen, Z. Qian, J. Wu, and M. Xiao, “Joint configuration adaptation and bandwidth allocation for edge-based real-time video analytics,” in IEEE INFOCOM 2020-IEEE Conference on Computer Communications.   IEEE, 2020, pp. 257–266.
  12. D. Meiling, X. Siya, S. Sujie, G. Shaoyong, Q. Xuesong, and X. Ao, “Blockchain-based reliable fog-cloud service solution for iiot,” Chinese Journal of Electronics, vol. 30, no. 2, pp. 359–366, 2021.
  13. L. Peng, Y. Xiaotian, X. He, and W. Ruchuan, “Secure localization technology based on dynamic trust management in wireless sensor networks,” Chinese Journal of Electronics, vol. 30, no. 4, pp. 759–768, 2021.
  14. L. Lingshu, W. Jiangxing, H. Hongchao, L. Wenyan, and G. Zehua, “Secure cloud architecture for 5g core network,” Chinese Journal of Electronics, vol. 30, no. 3, pp. 516–522, 2021.
  15. K. Hsieh, G. Ananthanarayanan, P. Bodik, S. Venkataraman, P. Bahl, M. Philipose, P. B. Gibbons, and O. Mutlu, “Focus: Querying large video datasets with low latency and low cost,” in Proc. of OSDI, 2018, pp. 269–286.
  16. T. Zhang, A. Chowdhery, P. Bahl, K. Jamieson, and S. Banerjee, “The design and implementation of a wireless video surveillance system,” in Proc. of MobiCom, 2015, pp. 426–438.
  17. D. Kang, J. Emmons, F. Abuzaid, P. Bailis, and M. Zaharia, “Noscope: Optimizing neural network queries over video at scale,” Proc. VLDB Endow., vol. 10, no. 11, p. 1586–1597, 2017.
  18. L. Liu, H. Li, and M. Gruteser, “Edge assisted real-time object detection for mobile augmented reality,” in Proc. of MobiCom, 2019, pp. 1–16.
  19. X. Xie and K.-H. Kim, “Source compression with bounded DNN perception loss for iot edge computer vision,” in Proc. of MobiCom, 2019, pp. 1–16.
  20. K. Du, Q. Zhang, A. Arapin, H. Wang, Z. Xia, and J. Jiang, “Accmpeg: Optimizing video encoding for video analytics,” in Proc. of MLSys, 2022.
  21. H. Yeo, C. J. Chong, Y. Jung, J. Ye, and D. Han, “Nemo: enabling neural-enhanced video streaming on commodity mobile devices,” in Proc. of MobiCom, 2020, pp. 1–14.
  22. J. Kim, Y. Jung, H. Yeo, J. Ye, and D. Han, “Neural-enhanced live streaming: Improving live video ingest via online learning,” in Proc. of ACM SIGCOMM, 2020, pp. 107–125.
  23. H. Yeo, Y. Jung, J. Kim, J. Shin, and D. Han, “Neural adaptive content-aware internet video delivery,” in Proc. of OSDI 18, 2018, pp. 645–661.
  24. J. Yi, S. Kim, J. Kim, and S. Choi, “Supremo: Cloud-assisted low-latency super-resolution in mobile devices,” IEEE Transactions on Mobile Computing, pp. 1847–1860, 2020.
  25. H. Yeo, H. Lim, J. Kim, Y. Jung, J. Ye, and D. Han, “Neuroscaler: Neural video enhancement at scale,” in Proc. of SIGCOMM, 2022.
  26. H. Yeo, S. Do, and D. Han, “How will deep learning change internet video delivery?” in Proc. of ACM HotNets, 2017, pp. 57–64.
  27. J. Yi, S. Choi, and Y. Lee, “EagleEye: Wearable camera-based person identification in crowded urban spaces,” in Proc. of MobiCom, 2020.
  28. T. Yuan, L. Mi, W. Wang, H. Dai, and X. Fu, “Accdecoder: Accelerated decoding for neural-enhanced video analytics,” arXiv preprint arXiv:2301.08664, 2023.
  29. B. Zhang, X. Jin, S. Ratnasamy, J. Wawrzynek, and E. A. Lee, “Awstream: Adaptive wide-area streaming analytics,” in Proc. of ACM SIGCOMM, 2018, pp. 236–252.
  30. J. Jiang, G. Ananthanarayanan, P. Bodik, S. Sen, and I. Stoica, “Chameleon: scalable adaptation of video analytics,” in Proc. of ACM SIGCOMM, 2018, pp. 253–266.
  31. Google, “Webrtc official website,” https://webrtc.org/, 2011.
  32. IETF, “Google congestion control (gcc) specification.” https://datatracker.ietf.org/doc/html/draft-ietf-rmcat-gcc-02, 2017.
  33. ——, “Real-time transport protocol (rtp) specification.” https://tools.ietf.org/html/rfc3550, 2003.
  34. T. Stockhammer, “Dynamic adaptive streaming over http– standards and design principles,” in Pro. of MMSys, 2011, pp. 133–144.
  35. Y. Wang, W. Wang, D. Liu, X. Jin, J. Jiang, and K. Chen, “Enabling edge-cloud video analytics for robotics applications,” in Proc. of IEEE INFOCOM, 2022, pp. 1–10.
  36. Z. Zhang and V. Sze, “Fast: A framework to accelerate super-resolution processing on compressed videos,” in Proc. of CVPRW, 2017, pp. 19–28.
  37. “Fcc broadband bandwidth measurement,” https://www.fcc.gov/reports-research/reports/measuring-broadband-america/raw-data-measuring-broadband-america-eighth.
  38. D. Dias and L. H. M. K. Costa, “Edsr weights.” Downloaded from https://github.com/sanghyun-son/EDSR-PyTorch.
  39. NVIDIA, “nvjpeg libraries,” https://developer.nvidia.com/nvjpeg.
  40. Youtube, “Gebhardt insurance traffic cam round trip bike shop,” https://www.youtube.com/watch?v=_XBMMTQVj68.
  41. Yoda, “Highway,” https://yoda.cs.uchicago.edu/videos/highway.mp4.
  42. R. Liu, J. Gao, J. Zhang, D. Meng, and Z. Lin, “Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 10 045–10 067, 2021.
  43. S. Zhang, C. Wang, Y. Jin, J. Wu, Z. Qian, M. Xiao, and S. Lu, “Adaptive configuration selection and bandwidth allocation for edge-based video analytics,” IEEE/ACM Transaction on Networking, vol. 30, no. 1, p. 285–298, aug 2021.
  44. T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 560–576, 2003.
  45. Google, “libvpx official github repository,” https://github.com/webmproject/ libvpx/.
  46. Open Source, “Jm 19.0,” https://iphome.hhi.de/suehring/.
  47. “Wondershaper,” https://github.com/magnific0/wondershaper, accessed: 2023-08-01.
  48. Youtube, “Jackson hole wyoming usa town square live cam,” https://www.youtube.com/watch?v=1EiC9bvVGnk.
  49. Youtube, “City of auburn toomer’s corner webcam,” https://www.youtube.com/watch?v=hMYIc5ZPJL4.
  50. Yoda, “Motorway,” https://yoda.cs.uchicago.edu/videos/motorway.mp4.
  51. T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proc. of ICML, 2018, pp. 1861–1870.
  52. J. Redmon and A. Farhadi, “Yolo9000: better, faster, stronger.” in Proc. of CVPR, 2017.
  53. C. dataset, “highway video,” https://cocodataset.org/.
  54. NVIDIA, “Nvidia nsight,” https://developer.nvidia.com/nsight-systems.
  55. T. Y.-H. Chen, L. Ravindranath, S. Deng, P. Bahl, and H. Balakrishnan, “Glimpse: Continuous, real-time object recognition on mobile devices,” in Proc. of SenSys, 2015, pp. 155–168.
  56. M. Meddeb, “Region-of-interest-based video coding for video conference applications,” p. 172.
  57. M. Dasari, A. Bhattacharya, S. Vargas, P. Sahu, A. Balasubramanian, and S. R. Das, “Streaming 360-degree videos using super-resolution,” in Proc. of IEEE INFOCOM, 2020, pp. 1977–1986.
  58. Y. Wang, W. Wang, J. Zhang, J. Jiang, and K. Chen, “Bridging the edge-cloud barrier for real-time advanced vision analytics,” in Proc. of HotCloud, 2019.
  59. S. Jain, X. Zhang, Y. Zhou, G. Ananthanarayanan, J. Jiang, Y. Shu, V. Bahl, and J. Gonzalez, “Spatula: Efficient cross-camera video analytics on large camera networks,” in Proc. ACM SEC, 2020, pp. 110–124.
  60. C. Canel, T. Kim, G. Zhou, C. Li, H. Lim, D. G. Andersen, M. Kaminsky, and S. Dulloor, “Scaling video analytics on constrained edge nodes,” in Proc. of MLSys, 2019.
  61. W. Cui, H. Zhao, Q. Chen, N. Zheng, J. Leng, J. Zhao, Z. Song, T. Ma, Y. Yang, C. Li et al., “Enable simultaneous dnn services based on deterministic operator overlap and precise latency prediction,” in Proc. of SC, 2021, pp. 1–15.
  62. Y. Yuan, W. Wang, Y. Wang, S. S. Adhatarao, B. Ren, K. Zheng, and X. Fu, “VSiM: Improving QoE fairness for video streaming in mobile environments,” in Proc. of IEEE INFOCOM, 2022, pp. 1309–1318.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets