---
title: 'JumpScore: Temporal Localization & Action Scoring'
url: https://www.emergentmind.com/topics/jumpscore
type: topic
---

# JumpScore: Temporal Localization & Action Scoring

JumpScore is a term that denotes two distinct but related paradigms in the analysis and evaluation of jumping actions in video: (1) a temporal localization benchmark and metric for fine-grained event detection in dense, cyclic motion settings, introduced in the context of vision-language models for perceptual intelligence; and (2) a system for quantitative action quality assessment in judged sporting events relying on spatiotemporal representation learning and regression. Both paradigms have emerged within the last decade and have specific methodological foundations, metric definitions, data protocols, and application contexts. 

## 1. Formal Definition of JumpScore as Temporal Localization Metric

JumpScore was introduced as a temporal-localization benchmark and metric specifically targeting fine-grained grounding in high-frequency, densely repeated motion. The reference implementation centers on the problem of detecting precise cycle-boundary events in jump-rope sequences—an underrepresented regime in previous video understanding evaluations. For each annotated video, let $G = \{g_1, \dots, g_N\} \subset \mathbb{R}$ denote the set of ground-truth cycle-start times (in seconds), and let $P = \{p_1, \dots, p_M\} \subset \mathbb{R}$ be the set of predicted timestamps provided by a model.

For a prescribed temporal tolerance $\delta > 0$, a one-to-one greedy matching is performed between $P$ and $G$. Each $p_j$ is assigned to the nearest unmatched $g_i$ such that $|p_j - g_i| \leq \delta$. The numbers of true positives ($\mathrm{TP}_\delta$), false positives ($\mathrm{FP}_\delta$), and false negatives ($\mathrm{FN}_\delta$) are then computed. Precision and recall are evaluated over the ranked list of predictions (ranked chronologically, as all predictions are scored equally), and the standard Average Precision (AP) is computed for each $\delta$:

\[
\mathrm{AP}(\delta) = \sum_{k=1}^{M} \left[ R(k) - R(k-1) \right] \cdot P(k)
\]
where $P(k) = \frac{\text{TP up to }k}{k}$, $R(k) = \frac{\text{TP up to }k}{N}$.

To balance fine temporal accuracy with resilience to minor annotation noise, JumpScore takes the unweighted mean of AP across three tolerances ($\delta \in \{0.1, 0.2, 0.3\}$ seconds):

\[
\mathrm{JumpScore} = \frac{1}{3} [\mathrm{AP}(0.1) + \mathrm{AP}(0.2) + \mathrm{AP}(0.3)]
\]

This multi-band construction rewards precise cycle-boundary localization while also promoting completeness and robustness to small temporal offsets [2605.25979].

## 2. Dataset Composition and Evaluation Protocol

The JumpScore benchmark dataset consists of 189 "in-the-wild" jump-rope videos, each 30–90 seconds in duration, with native resolutions of at least $1280 \times 720$ (the majority at $1920 \times 1080$ or higher). Every cycle start is annotated by human raters to decimal-second granularity, validated against the original frame rate for reproducibility. 

For evaluation, a fixed prompt instructs the model: "List the start timestamps in s of each jump rope the main character does in the video. The start is defined as the moment the rope is behind the legs." Greedy one-to-one matching within each $\delta$ is used to designate true positives, false positives, and false negatives. 

Input representations are systematically evaluated under two regimes:
- Uniform frame sampling (extracting evenly spaced RGB frames)
- Codec-stream tokenization (variable-length GOPs, motion–residual-driven patch selection), both with matched token/frame budgets.

Performance is reported as mean AP over the prescribed tolerances for each input regime [2605.25979].

## 3. High-Frequency Repeated Action Localization

JumpScore is tailored for settings characterized by highly regular, subtly differentiated cyclic motions where adjacent events are visually similar and discriminative cues are confined to specific transition frames. In the jump-rope scenario, only the moment when the rope passes behind the legs is informative. There is no temporal grouping or down-sampling in the evaluation protocol; each cycle is an independent, granular event. The scoring system penalizes coarse predictions or uniformly spaced guesses, emphasizing models' ability to resolve fine temporal boundaries repeatedly—a regime where prior video benchmarks are less discriminative.

By aggregating AP across sub-second bands, JumpScore remains sensitive to precise errors and also highlights completeness of coverage in dense-event settings [2605.25979]. 

## 4. Empirical Results, Model Comparisons, and Representational Ablations

Tabulated results from [2605.25979] demonstrate the discriminative power of JumpScore and the effect of input representation strategies. On JumpScore:

| Model                   | JumpScore mAP |
|-------------------------|--------------|
| LLaVA-OneVision-2-8B    | 74.9         |
| Qwen3-VL-8B             | 30.1         |

Codec-stream tokenization, which adapts token allocation according to bit-cost and motion residuals, produces substantial gains relative to uniform sampling under fixed visual budgets. For instance:

| Input Type                  | JumpScore @128 frames |
|-----------------------------|----------------------|
| Uniform Sampling            | 45.4                 |
| Codec-Stream Tokenization   | 74.9                 |

A broader sweep shows codec allocation maintaining strong gains even at lower token budgets:

| Frames | Uniform | Codec  |
|--------|---------|--------|
|   4    | 32.5    | 39.4   |
|   8    | 35.2    | 40.2   |
|  16    | 36.7    | 46.9   |
|  32    | 37.6    | 58.3   |
|  64    | 39.9    | 71.3   |
| 128    | 45.4    | 74.9   |

Codec-stream tokenization alone leads to an average absolute gain of +9.7 points over uniform sampling on standard (non-repeated-motion) temporal grounding benchmarks, with even larger benefits (+17.3 or up to ~30 points) on dense repeated-motion metrics such as JumpScore [2605.25979].

## 5. Interpretation, Robustness, and Failure Modes

The large +44.8 point performance gap between LLaVA-OneVision-2 and Qwen3-VL-8B (74.9 vs 30.1 JumpScore mAP) reflects significant advances in temporal-boundary discrimination. These gains derive from the LLaVA-OneVision-2's use of a unified 3D RoPE for visual token positional encoding and a codec-stream approach for allocating the token budget to high-evidence intervals. Codec-stream tokenization enables the model to concentrate representation on event-bearing transitions, mitigating the upper-bound limitations imposed by uniform frame allocation.

JumpScore is highly sensitive to temporal misalignment and local errors. For example, if boundary predictions systematically drift beyond the strictest $\delta = 0.1$ s band, AP collapses, revealing model biases undetected by looser tolerances. The design also exposes failures in handling very irregular periodicities—outlier intervals may not be tokenized adequately unless budget allocation remains adaptive. Budget sensitivity analyses show that codec-stream achieves high JumpScore mAP at moderate budgets ($\geq 32$ frames), while uniform sampling saturates well below state-of-the-art performance.

Best practices for deployment include using codec-stream tokenization at $>$32 GOP slots and reporting AP across all $\delta$ to distinguish models' tolerance to temporal precision. Combining codec-stream canvases for event coverage with sparse uniform keys can augment appearance-sensitive tasks [2605.25979].

## 6. JumpScore for Action Quality Assessment in Sports (SVR/LSTM Paradigm)

An orthogonal but relevant usage of "JumpScore" has arisen in the context of predicting action quality scores in judged sports, exemplified by the framework of Parmar & Morris [1611.05125]. Here, JumpScore denotes a system for mapping video of athletic jumps (e.g., ski jump, high jump, long jump) to a predicted execution quality score, based on spatiotemporal feature learning and regression:

- **C3D-SVR**: Averaged 4096-dimensional features from a pretrained 3D-CNN (C3D) form the input to a support vector regressor.
- **C3D-LSTM**: Sequences of clip-level features are processed by an LSTM trained under either final-label or incremental-label supervision.
- **C3D-LSTM-SVR**: An SVR regressor is trained atop the LSTM's final hidden state or an aggregation thereof.

Evaluation is based on Spearman's rank correlation $\rho$ between ground-truth and predicted execution scores. The C3D-SVR approach consistently yields the highest $\rho$ across all tested sports; for instance, on the MIT-Dive dataset, $\rho = 0.74$, outperforming pose-based and entropy-based baselines.

Extensions include loss functions tailored for ordinal targets, phase-aware temporal modeling (e.g., via hierarchical LSTMs), and the fusion of pose or sensor features for increased explainability and cross-domain robustness. Feedback to athletes is readily enabled by backtracing score gradients to video intervals with maximal predicted deductions [1611.05125].

## 7. Applications, Limitations, and Recommended Practices

JumpScore as a temporal localization metric has established itself as a rigorous, high-precision evaluator for dense repeated motion, exposing the limits of uniform visual evidence allocation and elevating the standard for fine-grained temporal grounding tasks in video-language models. As an action quality assessment system, JumpScore (in the SVR/LSTM paradigm) provides a data-efficient, interpretable, and extendable method for automatic judging in sports with limited labeled samples.

Recommended protocols for the localization metric include: employing codec-stream tokenization with budgets of at least 32 slots, reporting AP at all $\delta$ to expose coarse/fine errors, and hybridizing representations for specialized video content. For action scoring, C3D-SVR should serve as the primary regressor, with LSTM-based auxiliary branches enabling error localization and interpretability, supported by a regime of domain-adaptive pretraining, data augmentation, and robust validation (Spearman’s $\rho$).

JumpScore’s precise design choices, especially its focus on dense-cycle event detection and budget-aware evidence allocation, render it distinct from broader video-localization or event-quality frameworks. Its adoption has advanced both the evaluation of next-generation vision-language models and the development of robust, explainable video-based sports analytics [2605.25979, 1611.05125].

Source: https://www.emergentmind.com/topics/jumpscore