Gaussian Mixture Proposals with Pull-Push Learning Scheme to Capture Diverse Events for Weakly Supervised Temporal Video Grounding
Abstract: In the weakly supervised temporal video grounding study, previous methods use predetermined single Gaussian proposals which lack the ability to express diverse events described by the sentence query. To enhance the expression ability of a proposal, we propose a Gaussian mixture proposal (GMP) that can depict arbitrary shapes by learning importance, centroid, and range of every Gaussian in the mixture. In learning GMP, each Gaussian is not trained in a feature space but is implemented over a temporal location. Thus the conventional feature-based learning for Gaussian mixture model is not valid for our case. In our special setting, to learn moderately coupled Gaussian mixture capturing diverse events, we newly propose a pull-push learning scheme using pulling and pushing losses, each of which plays an opposite role to the other. The effects of components in our scheme are verified in-depth with extensive ablation studies and the overall scheme achieves state-of-the-art performance. Our code is available at https://github.com/sunoh-kim/pps.
- Iterative Proposal Refinement for Weakly-Supervised Video Grounding. In CVPR, 6524–6534.
- Quo vadis, action recognition? a new model and the kinetics dataset. In CVPR, 6299–6308.
- Explore Inter-contrast between Videos via Composition for Weakly Supervised Temporal Sentence Grounding. In AAAI.
- Towards bridging event captioner and sentence localizer for weakly supervised dense event captioning. In CVPR, 8425–8435.
- Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT.
- Dual encoding for zero-example video retrieval. In CVPR, 9346–9355.
- Weakly supervised dense event captioning in videos. In NeurIPS, volume 31.
- Tall: Temporal activity localization via language query. In ICCV, 5267–5275.
- Cross-sentence temporal and semantic relations in video activity localisation. In ICCV, 7199–7208.
- SWAG-Net: Semantic Word-Aware Graph Network for Temporal Video Grounding. In ACM CIKM, 982–992.
- Position-aware Location Regression Network for Temporal Video Grounding. In AVSS, 1–8.
- Adam: A method for stochastic optimization. In ICLR.
- Dense-captioning events in videos. In ICCV, 706–715.
- A simple unified framework for detecting out-of-distribution samples and adversarial attacks. NeurIPS, 31.
- A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130.
- Weakly-supervised video moment retrieval via semantic completion network. In AAAI, volume 34, 11539–11546.
- Gaussian temporal awareness networks for action localization. In CVPR, 344–353.
- Vlanet: Video-language alignment network for weakly-supervised video moment retrieval. In ECCV, 156–171. Springer.
- A user attention model for video summarization. In ACM MM, 533–542.
- Weakly supervised video moment retrieval from text queries. In CVPR, 11592–11601.
- Efficient non-maximum suppression. In ICPR, volume 3, 850–855. IEEE.
- GloVe: Global Vectors for Word Representation. In EMNLP, 1532–1543.
- Temporal gaussian mixture layer for videos. In ICML, 5152–5161. PMLR.
- Learning transferable visual models from natural language supervision. In ICML, 8748–8763. PMLR.
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
- Weakly-supervised multi-level attentional reconstruction network for grounding textual queries in videos. arXiv preprint arXiv:2003.07048.
- Logan: Latent graph co-attention network for weakly-supervised video moment retrieval. In WACV, 2083–2092.
- Learning spatiotemporal features with 3d convolutional networks. In ICCV, 4489–4497.
- Attention is all you need. In NeurIPS, volume 30.
- Object-aware video-language pre-training for retrieval. In CVPR, 3313–3322.
- Learning fine-grained image similarity with deep ranking. In CVPR, 1386–1393.
- Weakly supervised temporal adjacent network for language grounding. IEEE Transactions on Multimedia.
- Fine-grained semantic alignment network for weakly supervised temporal language grounding. In EMNLP.
- Visual co-occurrence alignment learning for weakly-supervised video moment retrieval. In ACM MM, 1459–1468.
- Reinforcement learning for weakly supervised temporal grounding of natural language in untrimmed videos. In ACM MM, 1283–1291.
- Local correspondence network for weakly supervised temporal sentence grounding. IEEE Transactions on Image Processing, 30: 3252–3262.
- A closer look at temporal sentence grounding in videos: Dataset and metric. In Proceedings of the 2nd International Workshop on Human-centric Multimedia Analysis, 13–21.
- Regularized two-branch proposal networks for weakly-supervised moment retrieval in videos. In ACM MM, 4098–4106.
- Counterfactual contrastive learning for weakly-supervised vision-language grounding. In NeurIPS, volume 33, 18123–18134.
- Weakly supervised video moment localization with contrastive negative sample mining. In AAAI, volume 1, 3.
- Weakly Supervised Temporal Sentence Grounding With Gaussian-Based Contrastive Proposal Learning. In CVPR, 15555–15564.
- Embracing uncertainty: Decoupling and de-bias for robust temporal grounding. In CVPR, 8445–8454.
- Zhou, Z.-H. 2021. Ensemble learning. In Machine learning, 181–210. Springer.
- Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In ICLR.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.