Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

VVS: Video-to-Video Retrieval with Irrelevant Frame Suppression (2303.08906v2)

Published 15 Mar 2023 in cs.CV

Abstract: In content-based video retrieval (CBVR), dealing with large-scale collections, efficiency is as important as accuracy; thus, several video-level feature-based studies have actively been conducted. Nevertheless, owing to the severe difficulty of embedding a lengthy and untrimmed video into a single feature, these studies have been insufficient for accurate retrieval compared to frame-level feature-based studies. In this paper, we show that appropriate suppression of irrelevant frames can provide insight into the current obstacles of the video-level approaches. Furthermore, we propose a Video-to-Video Suppression network (VVS) as a solution. VVS is an end-to-end framework that consists of an easy distractor elimination stage to identify which frames to remove and a suppression weight generation stage to determine the extent to suppress the remaining frames. This structure is intended to effectively describe an untrimmed video with varying content and meaningless information. Its efficacy is proved via extensive experiments, and we show that our approach is not only state-of-the-art in video-level approaches but also has a fast inference time despite possessing retrieval capabilities close to those of frame-level approaches. Code is available at https://github.com/sejong-rcv/VVS

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. 2015. Call for Proposals for Compact Descriptors for Video Analysis.
  2. AC-SUM-GAN: Connecting actor-critic and generative adversarial networks for unsupervised video summarization. IEEE Transactions on Circuits and Systems for Video Technology.
  3. Unsupervised video summarization via attention-driven adversarial learning. In MultiMedia Modeling: 26th International Conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, Proceedings, Part I 26.
  4. Summarizing videos using concentrated attention and considering the uniqueness and diversity of the video frames. In Proceedings of the 2022 International Conference on Multimedia Retrieval.
  5. A stepwise, label-based approach for improving the adversarial training in unsupervised video summarization. In Proceedings of the 1st International Workshop on AI for Smart TV Content Production, Access and Delivery.
  6. NetVLAD: CNN architecture for weakly supervised place recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  7. LAMV: Learning to align and match videos with kernelized temporal layers. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  8. Parametric correspondence and chamfer matching: Two new techniques for image matching. In Proceedings of Image Understanding Workshop.
  9. A clustering method for information retrieval. Technical ReportIR-0199, Laboratoire d’Informatique d’Avignon, France.
  10. Pattern-based near-duplicate video retrieval and localization on web-scale videos. IEEE Transactions on Multimedia.
  11. Circulant temporal encoding for video retrieval and temporal alignment. International Journal of Computer Vision.
  12. Enhanced deep video summarization network. In BMVC.
  13. Using interdocument similarity information in document retrieval systems. Journal of the American Society for Information Science.
  14. Symmetrical synthesis for deep metric learning. In Proceedings of the AAAI Conference on Artificial Intelligence.
  15. Creating summaries from user videos. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VII 13.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  17. Unsupervised video summarization with attentive conditional generative adversarial networks. In Proceedings of the 27th ACM International Conference on multimedia.
  18. What makes a video a video: Analyzing temporal information in video understanding models and datasets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  19. Negative evidences and co-occurences in image retrieval: The benefit of PCA and whitening. In European Conference on Computer Vision.
  20. VCDB: a large-scale database for partial copy detection in videos. In European Conference on Computer Vision.
  21. Simultaneous Video Retrieval and Alignment. IEEE Access.
  22. Exploring the Temporal Cues to Enhance Video Retrieval on Standardized CDVA. IEEE Access.
  23. Combination of multiple global descriptors for image retrieval. arXiv preprint arXiv:1903.10663.
  24. Discriminative feature learning for unsupervised video summarization. In Proceedings of the AAAI Conference on artificial intelligence.
  25. Global-and-local relative position embedding for unsupervised video summarization. In European Conference on Computer Vision.
  26. Unsupervised video summarization via multi-source features. In Proceedings of the 2021 International Conference on Multimedia Retrieval.
  27. UBoCo: Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  28. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  29. Embedding expansion: Augmentation in embedding space for deep metric learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  30. A benchmark on tricks for large-scale image retrieval. arXiv preprint arXiv:1907.11854.
  31. FIVR: Fine-grained incident video retrieval. IEEE Transactions on Multimedia.
  32. ViSiL: Fine-grained spatio-temporal video similarity learning. In Proceedings of IEEE Conference on Computer Vision.
  33. Near-duplicate video retrieval by aggregating intermediate cnn layers. In Proceedings of International Conference on Multimedia Modeling.
  34. Near-duplicate video retrieval with deep metric learning. In Proceedings of IEEE Conference on Computer Vision Workshops.
  35. DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval. International Journal of Computer Vision.
  36. Exploring global diverse attention via pairwise temporal relation for video summarization. Pattern Recognition.
  37. Hnip: Compact deep invariant representations for video matching, localization, and retrieval. IEEE Transactions on Multimedia.
  38. Network in network. arXiv preprint arXiv:1312.4400.
  39. Cluster-based retrieval using language models. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval.
  40. VRAG: Region Attention Graphs for Content-Based Video Retrieval. arXiv preprint arXiv:2205.09068.
  41. Tempered sigmoid activations for deep learning with differential privacy. In Proceedings of the AAAI Conference on Artificial Intelligence.
  42. Self-attention recurrent summarization network with reinforcement learning for video summarization task. In 2021 IEEE International Conference on Multimedia and Expo (ICME).
  43. Temporal matching kernel with explicit feature maps. In Proceedings of ACM International Conference on Multimedia.
  44. Event retrieval in large video collections with circulant temporal encoding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  45. Video summarization by learning from unpaired data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
  46. Video summarization using fully convolutional sequence networks. In Proceedings of the European conference on computer vision.
  47. Temporal context aggregation for video retrieval with contrastive learning. In Proceedings of IEEE Winter Conference on Applications of Computer Vision.
  48. Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Transactions on Multimedia.
  49. Tvsum: Summarizing web videos using titles. In Proceedings of the IEEE conference on computer vision and pattern recognition.
  50. Strzalkowski, T. 1995. Natural language information retrieval. Information Processing & Management.
  51. Scalable detection of partial near-duplicate videos by visual-temporal consistency. In Proceedings of ACM International Conference on Multimedia.
  52. Particular Object Retrieval With Integral Max-Pooling of CNN Activations. In Proceedings of International Conference on Learning Representations.
  53. Graph attention networks. arXiv preprint arXiv:1710.10903.
  54. Era: Entity relationship aware video summarization with wasserstein gan. arXiv preprint arXiv:2109.02625.
  55. Real-time near-duplicate elimination for web video search with content and context. IEEE Transactions on Multimedia.
  56. Using independently recurrent networks for reinforcement learning based unsupervised video summarization. Multimedia Tools and Applications.
  57. Deep multi-task representation learning: A tensor factorisation approach. arXiv preprint arXiv:1605.06391.
  58. Reconstructive sequence-graph network for video summarization. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  59. Property-constrained dual learning for video summarization. IEEE transactions on neural networks and learning systems.
  60. Deep reinforcement learning for unsupervised video summarization with diversity-representativeness reward. In Proceedings of the AAAI Conference on Artificial Intelligence.
  61. Zhu, M. 2004. Recall, precision and average precision. Department of Statistics and Actuarial Science, University of Waterloo, Waterloo.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - sejong-rcv/VVS: [AAAI-24] VVS : Video-to-Video Retrieval With Irrelevant Frame Suppression (19 stars)