Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Beyond the Frame: Single and mutilple video summarization method with user-defined length (2401.10254v1)

Published 23 Dec 2023 in cs.CV and cs.LG

Abstract: Video smmarization is a crucial method to reduce the time of videos which reduces the spent time to watch/review a long video. This apporach has became more important as the amount of publisehed video is increasing everyday. A single or multiple videos can be summarized into a relatively short video using various of techniques from multimodal audio-visual techniques, to natural language processing approaches. Audiovisual techniques may be used to recognize significant visual events and pick the most important parts, while NLP techniques can be used to evaluate the audio transcript and extract the main sentences (timestamps) and corresponding video frames from the original video. Another approach is to use the best of both domain. Meaning that we can use audio-visual cues as well as video transcript to extract and summarize the video. In this paper, we combine a variety of NLP techniques (extractive and contect-based summarizers) with video processing techniques to convert a long video into a single relatively short video. We design this toll in a way that user can specify the relative length of the summarized video. We have also explored ways of summarizing and concatenating multiple videos into a single short video which will help having most important concepts from the same subject in a single short video. Out approach shows that video summarizing is a difficult but significant work, with substantial potential for further research and development, and it is possible thanks to the development of NLP models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. D. DeMenthon, V. Kobla, and D. Doermann, “Video summarization by curve simplification,” in Proceedings of the sixth ACM international conference on Multimedia, 1998, pp. 211–218.
  2. L. He, E. Sanocki, A. Gupta, and J. Grudin, “Auto-summarization of audio-video presentations,” in Proceedings of the seventh ACM international conference on Multimedia (Part 1), 1999, pp. 489–498.
  3. L. Yuan, F. E. H. Tay, P. Li, and J. Feng, “Unsupervised video summarization with cycle-consistent adversarial lstm networks,” IEEE Transactions on Multimedia, vol. 22, no. 10, pp. 2711–2722, 2019.
  4. Z. Ji, K. Xiong, Y. Pang, and X. Li, “Video summarization with attention-based encoder–decoder networks,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 6, pp. 1709–1717, 2019.
  5. C. Guntuboina, A. Porwal, P. Jain, and H. Shingrakhia, “Video summarization for multiple sports using deep learning,” in Proceedings of the International e-Conference on Intelligent Systems and Signal Processing: e-ISSP 2020.   Springer, 2022, pp. 643–656.
  6. M. Ajmal, M. H. Ashraf, M. Shakir, Y. Abbas, and F. A. Shah, “Video summarization: Techniques and classification.” in ICCVG, 2012, pp. 1–13.
  7. U. N. Yoon, M. D. Hong, and G.-S. Jo, “Unsupervised video summarization based on deep reinforcement learning with interpolation,” Sensors, vol. 23, no. 7, p. 3384, 2023.
  8. A. E. Dirik, J. Lai, and M. Topkara, “Automatic static video summarization,” Apr. 1 2014, uS Patent 8,687,941.
  9. S. Chakraborty, O. Tickoo, and R. Iyer, “Adaptive keyframe selection for video summarization,” in 2015 IEEE winter conference on applications of computer vision.   IEEE, 2015, pp. 702–709.
  10. M. Guironnet, D. Pellerin, N. Guyader, and P. Ladret, “Video summarization based on camera motion and a subjective evaluation method,” EURASIP Journal on Image and Video Processing, vol. 2007, pp. 1–12, 2007.
  11. N. Nazari and M. Mahdavi, “A survey on automatic text summarization,” Journal of AI and Data Mining, vol. 7, no. 1, pp. 121–135, 2019.
  12. D. Yadav, J. Desai, and A. K. Yadav, “Automatic text summarization methods: A comprehensive review,” arXiv preprint arXiv:2204.01849, 2022.
  13. A. Nagy, B. Bial, and J. Ács, “Automatic punctuation restoration with bert models,” arXiv preprint arXiv:2101.07343, 2021.
  14. H. Zhang, J. Xu, and J. Wang, “Pretraining-based natural language generation for text summarization,” arXiv preprint arXiv:1902.09243, 2019.
  15. S. Tomar, “Converting video formats with ffmpeg,” Linux journal, vol. 2006, no. 146, p. 10, 2006.
  16. Z. Xia, P. Yi, Y. Liu, B. Jiang, W. Wang, and T. Zhu, “Genpass: A multi-source deep learning model for password guessing,” IEEE Transactions on Multimedia, vol. 22, pp. 1323–1332, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:203186587
  17. Y. Pan, S. Li, Y. Zhang, and T. Zhu, “Cda: Coordinating data dissemination and aggregation in heterogeneous iot networks using ctc,” J. Netw. Comput. Appl., vol. 171, p. 102797, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:225252122
  18. W. Wang, T. Xie, X. Liu, and T. Zhu, “Ect: Exploiting cross-technology concurrent transmission for reducing packet delivery delay in iot networks,” IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, pp. 369–377, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:52963794
  19. W. Wang, X. Liu, Y. Yao, Y. Pan, Z. Chi, and T. Zhu, “Crf: Coexistent routing and flooding using wifi packets in heterogeneous iot networks,” IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pp. 19–27, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:86513247
  20. Z. Chi, Y. Li, H. Sun, Y. Yao, Z. Lu, and T. Zhu, “B2w2: N-way concurrent communication for iot devices,” Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:8208501
  21. Z. Chi, Z. Huang, Y. Yao, T. Xie, H. Sun, and T. Zhu, “Emf: Embedding multiple flows of information in existing traffic for concurrent communication among heterogeneous iot devices,” IEEE INFOCOM 2017 - IEEE Conference on Computer Communications, pp. 1–9, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:958378
  22. Z. Chi, Y. Li, X. Liu, Y. Yao, Y. Zhang, and T. Zhu, “Parallel inclusive communication for connecting heterogeneous iot devices at the edge,” Proceedings of the 17th Conference on Embedded Networked Sensor Systems, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:207907186
  23. Z. Chi, Y. Yao, T. Xie, X. Liu, Z. Huang, W. Wang, and T. Zhu, “Ear: Exploiting uncontrollable ambient rf signals in heterogeneous networks for gesture recognition,” Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:53092538
  24. Z. Chi, Y. Li, Y. Yao, and T. Zhu, “Pmc: Parallel multi-protocol communication to heterogeneous iot radios within a single wifi channel,” 2017 IEEE 25th International Conference on Network Protocols (ICNP), pp. 1–10, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:20261532
  25. D. P. Khatri, G. Song, and T.-L. Zhu, “Heterogeneous computing systems,” ArXiv, vol. abs/2212.14418, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:255341042
  26. Y. Li, Z. Chi, X. Liu, and T. Zhu, “Chiron: Concurrent high throughput communication for iot devices,” Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:49668907
  27. Z. Chi, Y. Li, H. Sun, Y. Yao, and T. Zhu, “Concurrent cross-technology communication among heterogeneous iot devices,” IEEE/ACM Transactions on Networking, vol. 27, pp. 932–947, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:150300159
  28. Y. Pan, S. Li, B. Li, Y. Zhang, Z. Yang, B. Guo, and T. Zhu, “Cdd: Coordinating data dissemination in heterogeneous iot networks,” IEEE Communications Magazine, vol. 58, pp. 84–89, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:220605679
  29. W. Iqbal, W. Wang, and T. Zhu, “Machine learning and artificial intelligence in next-generation wireless network,” ArXiv, vol. abs/2202.01690, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:246485682
  30. D. Han, A. Li, L. Zhang, Y. Zhang, J. Li, T. Li, T. Zhu, and Y. Zhang, “Deep learning-guided jamming for cross-technology wireless networks: Attack and defense,” IEEE/ACM Transactions on Networking, vol. 29, pp. 1922–1932, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:236678111
  31. Z. Ning, H. N. Iradukunda, Q. Zhang, and T. Zhu, “Benchmarking machine learning: How fast can your algorithms go?” ArXiv, vol. abs/2101.03219, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:231573015
  32. W. Wang, E. Sallenback, Z. Ning, H. N. Iradukunda, W. Lu, Q. Zhang, and T. Zhu, “Mailleak: Obfuscation-robust character extraction using transfer learning,” 2021 IEEE/ACM Symposium on Edge Computing (SEC), pp. 459–464, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:229349006
  33. Q. Xie, Q. Zhang, X. Zhang, D. Tian, R. Wen, T. Zhu, P. Yi, and X. Li, “A context-centric chatbot for cryptocurrency using the bidirectional encoder representations from transformers neural networks,” 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:235996896
  34. S. Lola, R. Dhadvai, W. Wang, and T. Zhu, “Chatbot for fitness management using ibm watson,” ArXiv, vol. abs/2112.15167, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:245634752
  35. G. Song and T. Zhu, “Ml-based secure low-power communication in adversarial contexts,” ArXiv, vol. abs/2212.13689, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:255186145
  36. J. Xu, P. Yi, W. Wang, and T. Zhu, “A power storage station placement algorithm for power distribution based on electric vehicle,” International Journal of Distributed Sensor Networks, vol. 13, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:39936922
  37. S. Chandrasekaran, J. M. Reginald, W. Wang, and T. Zhu, “Computer vision based parking optimization system,” ArXiv, vol. abs/2201.00095, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:245650529
  38. X. Liu, W. Wang, G. Song, and T. Zhu, “Lightthief: Your optical communication information is stolen behind the wall,” in USENIX Security Symposium, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:260777714
  39. H. Gopal, G. Song, and T.-L. Zhu, “Security, privacy and challenges in microservices architecture and cloud computing- survey,” ArXiv, vol. abs/2212.14422, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:255340828
  40. W. Wang, Y. Yao, X. Liu, X. Li, P. Hao, and T. Zhu, “I can see the light: Attacks on autonomous vehicles using invisible lights,” Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:244077769
  41. Z. Zhou, X. Kuang, L. Sun, L. Zhong, and C. Xu, “Endogenous security defense against deductive attack: When artificial intelligence meets active defense for online service,” IEEE Communications Magazine, vol. 58, pp. 58–64, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:220606123
  42. Y. Pan, S. Li, J. L. Chang, Y. Yan, S. Xu, Y. An, and T. Zhu, “An unmanned aerial vehicle navigation mechanism with preserving privacy,” ICC 2019 - 2019 IEEE International Conference on Communications (ICC), pp. 1–6, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:198169333
  43. “2018 third international conference on security of smart cities, industrial control system and communications (ssic),” 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:240141684
  44. A. Li, J. Li, D. Han, Y. Zhang, T. Li, and T. Zhu, “Phyauth: Physical-layer message authentication for zigbee networks,” in USENIX Security Symposium, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:260340570
  45. E. Miller, N. Banerjee, and T. Zhu, “Smart homes that detect sneeze, cough, and face touching,” Smart Health (Amsterdam, Netherlands), vol. 19, pp. 100 170 – 100 170, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:230541358
  46. A. Gawade, A. Sanap, V. S. Baviskar, R. Jahnige, Q. Zhang, and T. Zhu, “Indoor air quality improvement,” ArXiv, vol. abs/2012.15387, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:229923861
  47. J. Gao, P. Yi, Z. Chi, and T. Zhu, “A smart medical system for dynamic closed-loop blood glucose-insulin control,” Smart Health, pp. 18–33, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:80388593
  48. J. Wang, Z. Huang, W. Zhang, A. Patil, K. Patil, T. Zhu, E. J. Shiroma, M. A. Schepps, and T. B. Harris, “2016 ieee international conference on big data (big data) wearable sensor based human posture recognition.” [Online]. Available: https://api.semanticscholar.org/CorpusID:10607990
  49. S. Li, P. Yi, Z. Huang, T. Xie, and T. Zhu, “Energy scheduling and allocation in electric vehicles energy internet,” 2016 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), pp. 1–5, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:26187112
  50. Y. Li and T. Zhu, “Using wi-fi signals to characterize human gait for identification and activity monitoring,” 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 238–247, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:5049216
  51. J. Gao, P. Yi, Z. Chi, and T. Zhu, “Enhanced wearable medical systems for effective blood glucose control,” 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 199–208, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:5049985
  52. Z. Huang and T. Zhu, “eair: an energy efficient air quality management system in residential buildings: poster abstract,” Proceedings of the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings, 2014. [Online]. Available: https://api.semanticscholar.org/CorpusID:117747
  53. W. Zhong, Z. Huang, T. Zhu, Y. J. Gu, Q. Zhang, P. Yi, D. Jiang, and S. Xiao, “ides: Incentive-driven distributed energy sharing in sustainable microgrids,” International Green Computing Conference, pp. 1–10, 2014. [Online]. Available: https://api.semanticscholar.org/CorpusID:11873749
  54. Y. Sui, P. Yi, X. Liu, and T. Zhu, “Energy transport station deployment in electric vehicles energy internet,” IEEE Access, vol. 7, pp. 97 986–97 995, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:198315553
  55. Y. Pan, B. K. Bhargava, Z. Ning, N. Slavov, S. Li, J. Liu, S. Xu, C. Li, and T. Zhu, “Safe and efficient uav navigation near an airport,” ICC 2019 - 2019 IEEE International Conference on Communications (ICC), pp. 1–6, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:198169074

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets