Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 89 tok/s
Gemini 2.5 Pro 53 tok/s Pro
GPT-5 Medium 26 tok/s Pro
GPT-5 High 25 tok/s Pro
GPT-4o 93 tok/s Pro
Kimi K2 221 tok/s Pro
GPT OSS 120B 457 tok/s Pro
Claude Sonnet 4 38 tok/s Pro
2000 character limit reached

BONES: Near-Optimal Neural-Enhanced Video Streaming (2310.09920v2)

Published 15 Oct 2023 in eess.SY, cs.LG, cs.NI, and cs.SY

Abstract: Accessing high-quality video content can be challenging due to insufficient and unstable network bandwidth. Recent advances in neural enhancement have shown promising results in improving the quality of degraded videos through deep learning. Neural-Enhanced Streaming (NES) incorporates this new approach into video streaming, allowing users to download low-quality video segments and then enhance them to obtain high-quality content without violating the playback of the video stream. We introduce BONES, an NES control algorithm that jointly manages the network and computational resources to maximize the quality of experience (QoE) of the user. BONES formulates NES as a Lyapunov optimization problem and solves it in an online manner with near-optimal performance, making it the first NES algorithm to provide a theoretical performance guarantee. Comprehensive experimental results indicate that BONES increases QoE by 5\% to 20\% over state-of-the-art algorithms with minimal overhead. Our code is available at https://github.com/UMass-LIDS/bones.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Challenges in cloud based ingest and encoding for high quality streaming media. In 2015 IEEE International Conference on Image Processing (ICIP). 1732–1736. https://doi.org/10.1109/ICIP.2015.7351097
  2. Eirikur Agustsson and Radu Timofte. 2017. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
  3. Oboe: Auto-Tuning Video ABR Algorithms to Network Conditions. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (Budapest, Hungary) (SIGCOMM ’18). Association for Computing Machinery, New York, NY, USA, 44–58. https://doi.org/10.1145/3230543.3230558
  4. DeepStream: Video Streaming Enhancements using Compressed Deep Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology (2022), 1–1. https://doi.org/10.1109/TCSVT.2022.3229079
  5. Akram Ansari and Mea Wang. 2023. IStream Player: A Versatile Video Player Framework. In Proceedings of the 33rd Workshop on Network and Operating System Support for Digital Audio and Video (Vancouver, BC, Canada) (NOSSDAV ’23). Association for Computing Machinery, New York, NY, USA, 65–71. https://doi.org/10.1145/3592473.3592569
  6. Rate-Distortion Hint Tracks for Adaptive Video Streaming. IEEE Trans. Circuits and Systems for Video Technology 15, 10 (Oct. 2005), 1257–1269.
  7. J. Chakareski and P.A. Chou. 2006. RaDiO Edge: Rate-Distortion Optimized Proxy-Driven Streaming from the Network Edge. IEEE/ACM Trans. Networking 14, 6 (Dec. 2006), 1302–1312.
  8. Millimeter Wave and Free-Space-Optics for Future Dual-Connectivity 6DOF Mobile Multi-User VR Streaming. ACM Transactions on Multimedia Computing Communications and Applications 19, 2(15) (feb 2023), 1–25.
  9. Towards Enabling Next Generation Societal Virtual Reality Applications for Virtual Human Teleportation. IEEE Signal Processing Magazine 39, 5 (2022), 22–41.
  10. BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4947–4956.
  11. BasicVSR++: Improving Video Super-Resolution With Enhanced Propagation and Alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5972–5981.
  12. SR360: Boosting 360-Degree Video Streaming with Super-Resolution. In Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (Istanbul, Turkey) (NOSSDAV ’20). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3386290.3396929
  13. Michele Claus and Jan van Gemert. 2019. ViDeNN: Deep Blind Video Denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
  14. Federal Communications Commission. 2016. Raw Data - Measuring Broadband America 2016. https://www.fcc.gov/reports-research/reports/measuring-broadband-america/raw-data-measuring-broadband-america-2016 https://www.fcc.gov/reports-research/reports/measuring-broadband-america/raw-data-measuring-broadband-america-2016.
  15. Streaming 360-Degree Videos Using Super-Resolution. In IEEE INFOCOM 2020 - IEEE Conference on Computer Communications. 1977–1986. https://doi.org/10.1109/INFOCOM41043.2020.9155477
  16. Robert G. Gallager. 2012. Discrete Stochastic Processes. Springer New York, NY. https://doi.org/10.1007/978-1-4615-2329-1
  17. mmWave Networking and Edge Computing for Scalable 360-Degree Video Multi-User Virtual Reality. IEEE Trans. Image Processing 32 (2023), 377–391.
  18. Dejavu: Enhancing Videoconferencing with Prior Knowledge. In Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications (Santa Cruz, CA, USA) (HotMobile ’19). Association for Computing Machinery, New York, NY, USA, 63–68. https://doi.org/10.1145/3301293.3302373
  19. A Buffer-Based Approach to Rate Adaptation: Evidence from a Large Video Streaming Service. SIGCOMM Comput. Commun. Rev. 44, 4 (aug 2014), 187–198. https://doi.org/10.1145/2740070.2626296
  20. PF-Net: Point Fractal Network for 3D Point Cloud Completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  21. Lightweight Image Super-Resolution with Information Multi-Distillation Network. In Proceedings of the 27th ACM International Conference on Multimedia (Nice, France) (MM ’19). Association for Computing Machinery, New York, NY, USA, 2024–2032. https://doi.org/10.1145/3343031.3351084
  22. Improving Fairness, Efficiency, and Stability in HTTP-Based Adaptive Video Streaming with FESTIVE. In Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies (Nice, France) (CoNEXT ’12). Association for Computing Machinery, New York, NY, USA, 97–108. https://doi.org/10.1145/2413176.2413189
  23. Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning. In Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication (Virtual Event, USA) (SIGCOMM ’20). Association for Computing Machinery, New York, NY, USA, 107–125. https://doi.org/10.1145/3387514.3405856
  24. PU-GAN: A Point Cloud Upsampling Adversarial Network. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).
  25. Efficient Meta-Tuning for Content-Aware Neural Video Delivery. In Computer Vision – ECCV 2022, Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer Nature Switzerland, Cham, 308–324.
  26. SwinIR: Image Restoration Using Swin Transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops. 1833–1844.
  27. Enhanced Deep Residual Networks for Single Image Super-Resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
  28. Chamara Madarasingha and Kanchana Thilakarathna. 2022. Edge Assisted Frame Interpolation and Super Resolution for Efficient 360-Degree Video Delivery. In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking (Sydney, NSW, Australia) (MobiCom ’22). Association for Computing Machinery, New York, NY, USA, 856–858. https://doi.org/10.1145/3495243.3558261
  29. Neural Adaptive Video Streaming with Pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (Los Angeles, CA, USA) (SIGCOMM ’17). Association for Computing Machinery, New York, NY, USA, 197–210. https://doi.org/10.1145/3098822.3098843
  30. Xatu: Richer Neural Network Based Prediction for Video Streaming. Proc. ACM Meas. Anal. Comput. Syst. 5, 3, Article 44 (dec 2021), 26 pages. https://doi.org/10.1145/3491056
  31. M. Neely. 2010. Stochastic Network Optimization with Application to Communication and Queueing Systems. Morgan & Claypool Publishers. https://books.google.com/books?id=sZpeAQAAQBAJ
  32. Michael J. Neely. 2013. Dynamic Optimization and Learning for Renewal Systems. IEEE Trans. Automat. Control 58, 1 (2013), 32–46. https://doi.org/10.1109/TAC.2012.2204831
  33. NVIDIA. 2023. Video Processing Framework. https://github.com/NVIDIA/VideoProcessingFramework
  34. Commute Path Bandwidth Traces from 3G Networks: Analysis and Applications. In Proceedings of the 4th ACM Multimedia Systems Conference (Oslo, Norway) (MMSys ’13). Association for Computing Machinery, New York, NY, USA, 114–118. https://doi.org/10.1145/2483977.2483991
  35. Sandvine. 2023. 2023 Global Internet Phenomena Report. https://www.sandvine.com/global-internet-phenomena-report-2023 https://www.sandvine.com/global-internet-phenomena-report-2023.
  36. Sophon: Super-Resolution Enhanced 360° Video Streaming with Visual Saliency-Aware Prefetch. In Proceedings of the 30th ACM International Conference on Multimedia (Lisboa, Portugal) (MM ’22). Association for Computing Machinery, New York, NY, USA, 3124–3133. https://doi.org/10.1145/3503161.3547750
  37. Learning for Unconstrained Space-Time Video Super-Resolution. IEEE Transactions on Broadcasting 68, 2 (2022), 345–358. https://doi.org/10.1109/TBC.2021.3131875
  38. From Theory to Practice: Improving Bitrate Adaptation in the DASH Reference Player. ACM Trans. Multimedia Comput. Commun. Appl. 15, 2s, Article 67 (jul 2019), 29 pages. https://doi.org/10.1145/3336497
  39. BOLA: Near-Optimal Bitrate Adaptation for Online Videos. IEEE/ACM Transactions on Networking 28, 4 (2020), 1698–1711. https://doi.org/10.1109/TNET.2020.2996964
  40. Thomas Stockhammer. 2011. Dynamic Adaptive Streaming over HTTP –: Standards and Design Principles. In Proceedings of the Second Annual ACM Conference on Multimedia Systems (San Jose, CA, USA) (MMSys ’11). Association for Computing Machinery, New York, NY, USA, 133–144. https://doi.org/10.1145/1943552.1943572
  41. Resolution-Robust Large Mask Inpainting With Fourier Convolutions. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2149–2159.
  42. FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  43. Prioritized Distributed Video Delivery with Randomized Network Coding. IEEE Trans. Multimedia 13, 4 (Aug. 2011), 776–787.
  44. HTTP/2-Based Adaptive Streaming of HEVC Video Over 4G/LTE Networks. IEEE Communications Letters 20, 11 (2016), 2177–2180. https://doi.org/10.1109/LCOMM.2016.2601087
  45. Revisiting Super-Resolution for Internet Video Streaming. In Proceedings of the 32nd Workshop on Network and Operating Systems Support for Digital Audio and Video (Athlone, Ireland) (NOSSDAV ’22). Association for Computing Machinery, New York, NY, USA, 8–14. https://doi.org/10.1145/3534088.3534344
  46. Learning in situ: a randomized experiment in video streaming. 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’20) ([n. d.]). https://par.nsf.gov/biblio/10186616
  47. NEMO: Enabling Neural-Enhanced Video Streaming on Commodity Mobile Devices. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (London, United Kingdom) (MobiCom ’20). Association for Computing Machinery, New York, NY, USA, Article 28, 14 pages. https://doi.org/10.1145/3372224.3419185
  48. Neural Adaptive Content-Aware Internet Video Delivery. In Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation (Carlsbad, CA, USA) (OSDI’18). USENIX Association, USA, 645–661.
  49. NeuroScaler: Neural Video Enhancement at Scale (SIGCOMM ’22). Association for Computing Machinery, New York, NY, USA, 795–811. https://doi.org/10.1145/3544216.3544218
  50. A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP. SIGCOMM Comput. Commun. Rev. 45, 4 (aug 2015), 325–338. https://doi.org/10.1145/2829988.2787486
  51. PU-Net: Point Cloud Upsampling Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  52. Zijie Yue and Miaojing Shi. 2023. Enhancing Space-time Video Super-resolution via Spatial-temporal Feature Interaction. arXiv:2207.08960 [cs.CV]
  53. Learning Joint Spatial-Temporal Transformations for Video Inpainting. In Computer Vision – ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 528–543.
  54. Video Super-Resolution and Caching—An Edge-Assisted Adaptive Video Streaming Solution. IEEE Transactions on Broadcasting 67, 4 (2021), 799–812. https://doi.org/10.1109/TBC.2021.3071010
  55. YuZu: Neural-Enhanced Volumetric Video Streaming. In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22). USENIX Association, Renton, WA, 137–154. https://www.usenix.org/conference/nsdi22/presentation/zhang-anlan
  56. Improving Quality of Experience by Adaptive Video Streaming with Super-Resolution. In IEEE INFOCOM 2020 - IEEE Conference on Computer Communications. 1957–1966. https://doi.org/10.1109/INFOCOM41043.2020.9155384
  57. PreSR: Neural-Enhanced Adaptive Streaming of VBR-Encoded Videos With Selective Prefetching. IEEE Transactions on Broadcasting 69, 1 (2023), 49–61. https://doi.org/10.1109/TBC.2022.3227419
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb On Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - UMass-LIDS/bones (6 stars)

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube