---
title: 'SportsMoments: Fine-Grained Sports Video Retrieval'
url: https://www.emergentmind.com/topics/sportsmoments
type: topic
---

# SportsMoments: Fine-Grained Sports Video Retrieval

SportsMoments is a benchmark for fine-grained **video-to-video moment retrieval** in sports, introduced to localize, within a target match video, the start and end times of a moment that semantically matches a query clip. It was proposed because existing resources such as **Sports-1M** and **ActivityNet-VRL** do not adequately capture fine-grained sports moments such as **“Cover Drive,” “Ducking a Bouncer,” “Goal Kick,”** and **“Penalty.”** Built from **176.6 hours** of complete match footage drawn from **80 full-length** YouTube videos in **soccer** and **cricket**, SportsMoments contains about **770K query-target pairs** over **29 action classes** and evaluates retrieval under **disjoint action classes across train, validation, and test splits** [2508.15439]. In adjacent literature, the phrase “sports moments” is also used more broadly for meaningful segments surfaced from tracking data, completion instants of actions, active-play intervals, highlight shots, and momentum shifts within matches [1611.09158][1805.06749][1906.11720][2509.01439][2009.05830].

## 1. Concept and scope

SportsMoments is defined through the task of **video-to-video moment retrieval**: given a **query video** \(V_q\) and a **target video** \(V_t\), the system must retrieve the temporally localized moment in \(V_t\) that matches the query. The benchmark is domain-specific, using only **soccer** and **cricket**, and it is explicitly motivated by the claim that prior datasets are either too broad or too coarse for realistic sports retrieval. In particular, the benchmark is positioned against **Sports-1M**, which is described as primarily for broad video classification, and **ActivityNet-VRL**, whose sports coverage includes coarse categories such as **“Playing Polo”** rather than fine-grained within-sport events [2508.15439].

The benchmark’s design makes the retrieval problem difficult in the way sports footage is difficult. The target videos are **full-length match videos**, not short trimmed clips, so the model must search through long, repetitive, context-heavy broadcasts in which many moments are visually similar. The retrieval objective is therefore not category recognition alone; it is **cross-video temporal-semantic alignment** under variation in speed, viewpoint, and surrounding context. The paper emphasizes that this is especially natural for sports, where a user may have an example clip but may not know the precise vocabulary needed to describe a highly specific event [2508.15439].

## 2. Corpus, annotation, and split design

The dataset is constructed from **176.6 hours** of complete match footage covering **80** full-length videos from YouTube. It contains **29 action classes** in total: **13 soccer** actions and **16 cricket** actions. Representative examples named in the paper include **“Cover Drive,” “Ducking a Bouncer,” “Goal Kick,”** and **“Penalty.”** The annotation unit is a temporal segment in a full match, and **two annotators** with **strong knowledge of these sports** marked the **start and end timestamps** for the specified actions [2508.15439].

| Statistic | Value |
|---|---:|
| Match footage | 176.6 hours |
| Full-length videos | 80 |
| Sports | Soccer, cricket |
| Action classes | 29 |
| Soccer classes | 13 |
| Cricket classes | 16 |
| Query-target pairs | \(\sim 770\)K |

The split structure is central to the benchmark. The **training split** contains approximately **750K pairs** across **16 classes**; the **validation split** contains **10K pairs** across **4 classes**; and the **test split** contains **10K pairs** across **9 classes**. The paper states that **action classes are disjoint across train, validation, and test**, and that each split includes **both cricket and soccer actions**, producing a “well-rounded distribution.” This makes SportsMoments an unseen-class retrieval benchmark rather than a same-class memorization test. The paper does **not** report SportsMoments-specific summary statistics for average clip duration, median clip duration, or minimum/maximum clip duration [2508.15439].

## 3. Retrieval formulation and the MATR model

SportsMoments supports a retrieval problem in which the query is itself a video segment. The paper represents the target video by \(M\) uniformly sampled frames and the query video by \(N\) uniformly sampled frames, with frames sampled **every 2 seconds**. The proposed model, **MATR** (**Moment Alignment TRansformer**), begins from frozen **CLIP ViT-B/32** features and learns a query-conditioned representation of the target video through an encoder, a dual-stage alignment mechanism, and a decoder with temporal prediction heads [2508.15439].

The input embeddings are written as
\[
\mathrm{E}_t \in \mathbb{R}^{M \times d}, \qquad \mathrm{E}_q \in \mathbb{R}^{N \times d},
\]
and concatenated as
\[
\mathrm{E}_c = [\mathrm{E}_t;\mathrm{E}_q] \in \mathbb{R}^{(M+N)\times d}.
\]
A Transformer encoder produces fused query-conditioned target and query representations, after which the model performs **dual-stage sequence alignment** using **soft-DTW**: first on the pre-fusion representations, then again on the post-fusion representations. The paper’s ablations show that **post-fusion alignment contributes more than pre-fusion alignment**, but that the two together are strongest.

After alignment, a Transformer decoder refines the aligned target subsequence. Two prediction heads then operate on the final representation: a **foreground/background classification head** and a **boundary prediction head**. The overall training loss combines foreground classification, boundary prediction, and the two alignment losses:
\[
\mathcal{L} = \frac{1}{S} \sum_{i=1}^{S} \left( \lambda_{fg}\mathcal{L}_{fg} + \lambda_{\text{seg}}\mathcal{L}_{\text{seg}} + \lambda_{\text{align}}^{\text{pre}}\mathcal{L}_{\text{align}}^{\text{pre}} + \lambda_{\text{align}}^{\text{post}}\mathcal{L}_{\text{align}}^{\text{post}} \right).
\]

The model is further strengthened by a task-specific **self-supervised pretraining** stage. A random clip is sampled from within a target video and used as a query, and the model is trained to localize that clip back inside the source video. The augmentations used in pretraining are **reversing frames**, **adding Gaussian noise**, **slowing down the action**, and **speeding up the action**. For SportsMoments, the implementation details reported are: hidden dimension **1024**, **4** encoder layers, **4** decoder layers, Transformer dropout **0.1**, linear projection dropout **0.5**, \(l=10\) learnable decoder queries, **AdamW** with learning rate **\(1\mathrm{e}{-4}\)**, weight decay **\(1\mathrm{e}{-4}\)**, **40** training epochs, batch size **40**, and all \(\lambda\) weights set to **1** [2508.15439].

## 4. Evaluation protocol and empirical results

SportsMoments is evaluated with standard **video-to-video moment retrieval** metrics: **mIoU** and **R@1, IoU=0.5**. The strongest implemented baseline reported on SportsMoments is **UniVTG Finetuned+V**, while the proposed model is reported in zero-shot, trained, and fine-tuned forms. The paper’s main claim is that MATR is substantially stronger than all adapted baselines, with especially large gains on this fine-grained sports benchmark [2508.15439].

| Method | mIoU | R@1 |
|---|---:|---:|
| UniVTG Finetuned+V | 43.6 | 41.8 |
| MATR Zero-shot | 31.8 | 30.7 |
| MATR Trained+V | 56.2 | 52.7 |
| MATR Finetuned+V | 59.2 | 56.5 |

Relative to **UniVTG Finetuned+V**, **MATR Finetuned+V** improves by **+15.6 mIoU** and **+14.7 R@1** according to the table values. The abstract reports a **14.7% gain in R@1** and a **14.4% gain in mIoU** over strong baselines, while the detailed discussion states **+15.6 mIoU** and **+14.7 R@1** over UniVTG; the table-based comparison is therefore the more specific one.

The ablations explain where these gains come from. Without either alignment stage, performance is **49.1 mIoU** and **46.6 R@1**; with both pre-fusion and post-fusion alignment, it rises to **56.2 mIoU** and **52.7 R@1**. Replacing **no alignment** with **soft-DTW** raises performance from **52.1 mIoU / 48.6 R@1** to **59.2 mIoU / 56.5 R@1**. Self-supervised pretraining with augmentation further improves over no pretraining, from **56.2 mIoU / 52.7 R@1** to **59.2 mIoU / 56.5 R@1**. The paper also shows that predicting directly from alignment matrices is much weaker than using the decoder plus dedicated prediction heads: the best alignment-only variant reaches **36.4 mIoU / 33.5 R@1**, far below the full model [2508.15439].

## 5. Related uses of “sports moments” in adjacent research

In adjacent sports analytics literature, “sports moments” also denotes meaningful segments identified from movement or event data rather than video-to-video retrieval. One line of work uses interactive motion charts to surface candidate moments from player-tracking trajectories. In a basketball case study, `gvisMotionChart` from the `googleVis` R package is used to animate player trajectories, inspect offensive and defensive spacing, and isolate short windows as candidate “moments.” The same study then validates visual impressions with team-level metrics such as **average inter-player distance** and **Voronoi area**, reporting **Defense: 28.47 \(m^2\), 5.68 m** and **Attack: 42.59 \(m^2\), 7.25 m** [1611.09158].

A second line treats moments as **action-completion instants**. “Action Completion: A Temporal Model for Moment Detection” formulates the task as locating the first frame at which an action’s goal is achieved, or declaring the sequence incomplete. Its best **classification-pre / regression-post** model detects the completion moment within **30 frames (1 second)** in **89%** of tested sequences, within **15 frames (0.5 second)** in **74%**, and at the exact frame in **30.4%** of cases [1805.06749].

In sensor-based basketball analytics, “moments” may instead denote **active game periods**. A rule-based method removes inactive periods using three criteria—wrong number of players on court, free-throw situations, and sustained low movement of all five on-court players—and then labels active moments as **offensive**, **defensive**, or **transition**. On a case study from the **Italian Basketball Cup Final Eight 2017**, the tuned parameters are reported as \(V_{min}=9.25\) km/h, \(T_{vel}=2\), with best **AUC = 0.8329** for active/inactive classification [1906.11720].

In soccer broadcast understanding, moments become **summary segments** rather than isolated events. **SoccerHigh** introduces a benchmark of **237 matches** from the Spanish, French, and Italian leagues, aligned to official highlight reels. Its baseline, using **VideoMAEv2 ViT-Giant** features and a **2-layer Transformer encoder**, achieves a test **F1 = 0.3956**, while the length-constrained **F1 Score@T** is **0.3883** [2509.01439].

Statistical work on tennis and football uses the language of moments to describe **momentum** or **change points** rather than retrieval targets. In Grand Slam tennis, generalized linear mixed-effect models applied to data from **2014 through 2019** find **strong evidence of carryover effects at the set, game, and point level**, including the result that winning the previous set increases the odds of winning the current set by **56.6%** for AO men, **114.5%** for AO women, **108.6%** for FO men, **182.1%** for FO women, **65.5%** for WB men, **187.2%** for WB women, **79.0%** for USO men, and **42.8%** for USO women [2009.05830]. In football, a **copula-based multivariate hidden Markov model** fitted to **34 Bundesliga matches** of Borussia Dortmund in **2017/18** infers latent states interpretable as low control, balanced play, and high control, thereby modeling momentum shifts as changes in latent match-control regimes [2002.01193].

## 6. Limitations, reuse, and position in the literature

SportsMoments is deliberately specialized. Its experimental domain is limited to **soccer** and **cricket**, and its source videos are **full-length match videos from YouTube**. The paper does not provide SportsMoments-specific descriptive statistics for average query duration or target clip duration, nor does it spell out the exact combinatorics used to generate the approximately **770K query-target pairs**. These omissions do not change the benchmark’s scale, but they do constrain reproducibility at the level of fine-grained data generation [2508.15439].

At the same time, later work indicates that the benchmark is reusable. A subsequent paper on **Generalized Moment Retrieval** builds **Soccer-GMR** partly from **Sportsmoments** and reports, on **100 randomly sampled clips**, a **mean boundary deviation < 2s** after human verification. That later benchmark also generalizes the retrieval setting from “exactly one matching moment” to a setting in which a query may correspond to **multiple** or **no** moments, a direction that is especially natural for sports search interfaces built around repeated event types such as corners, saves, offsides, or fouls [2605.02623].

Taken together, the literature positions SportsMoments as a benchmark for **fine-grained cross-video temporal-semantic alignment** in sports. It is distinct from tracking backbones, movement visualization tools, or momentum models, but it interfaces naturally with them: tracking and pose estimation can provide lower-level temporal structure, while completion detection, summarization, and momentum analysis define adjacent notions of what counts as an important sports moment. Within that landscape, SportsMoments contributes a concrete retrieval problem: given one sports event as a video example, find its counterpart in another full match video [2508.15439].

Source: https://www.emergentmind.com/topics/sportsmoments