SV3.3B: Lightweight Sports Video Analysis

• SV3.3B is a sports video analysis model with 3.3B parameters that integrates motion-aware keyframe extraction via DWT-VGG16-LDA with a self-supervised V-DWT-JEPA2 encoder and a LoRA-adapted LLaMA decoder.
• It preserves fine-grained temporal fidelity by selecting 16 representative frames through wavelet-based motion differentiation and clustering, ensuring key biomechanical transitions are maintained.
• The architecture improves technical action description quality and outperforms baseline multimodal models, achieving higher GT and IR scores on sports-specific evaluation metrics.

Searching arXiv for the primary paper and closely related video-LLMs mentioned in the supplied data. SV3.3B is a lightweight 3.3B-parameter sports video understanding model introduced for automated sports video analysis, with emphasis on preserving fine-grained biomechanical transitions and generating technical action descriptions from video clips (Kodathala et al., 23 Jul 2025). Its design couples a motion-aware keyframe selection stage, termed DWT-VGG16-LDA, with a self-supervised V-DWT-JEPA2 video encoder and a LoRA-adapted LLaMA-3.2-3B decoder. The model is motivated by the observation that sports actions often unfold through preparation, execution, and follow-through within seconds, so coarse temporal pooling or extreme token compression can erase analytically important transitions.

1. Research setting and stated objectives

SV3.3B is framed around three explicit objectives: fine-grained temporal fidelity, on-device efficiency, and data efficiency through self-supervised pretraining (Kodathala et al., 23 Jul 2025). The target domain is sports video, where rapid multi-phase motion makes conventional video-language compression especially lossy. The paper argues that existing multimodal LLM pipelines often rely on server-side computation and compress temporal information so aggressively that subtle athletic mechanics are not preserved.

The resulting system is described as a lightweight end-to-end architecture with approximately 3.3B total parameters, combining a 300M video encoder and a 3B LLM decoder. The paper’s subtitle refers to action recognition, but the reported pipeline, training setup, and evaluation are centered on technical sports action description generation. In that sense, SV3.3B occupies an intermediate position between event recognition and video captioning: it is not limited to assigning a coarse action label, yet it is also not a general-purpose long-form captioning system.

The paper positions the model against prior video-language approaches that use uniform temporal pooling or extreme token reduction, and argues that a motion-aware frame selection stage can retain biomechanical structure while keeping the encoded visual budget fixed at 16 frames. This suggests that the core research claim is not simply parameter reduction, but controlled temporal compression guided by motion salience.

2. Formal problem statement and overall pipeline

The paper formulates the input video as $X \in \mathbb{R}^{H \times W \times 3 \times N}$, where $N$ is the number of frames, and defines the task as selecting $K \ll N$ keyframes and generating a textual description $y$ (Kodathala et al., 23 Jul 2025). In SV3.3B, $K=16$, and this fixed-size selection is aligned with the downstream V-DWT-JEPA2 encoder.

The high-level pipeline is:

Component	Specification	Role
Keyframe extractor	DWT-VGG16-LDA, $K=16$	Select representative frames
Video encoder	V-DWT-JEPA2, ViT-L, $\approx 300$M params	Produce 1024-d spatiotemporal representation
Projection head	Two-layer MLP, $1024 \rightarrow 512 \rightarrow d_{\mathrm{LLM}}$	Map video features to LLM embedding space
Decoder	LLaMA-3.2-3B with LoRA rank 16	Generate technical action descriptions

Operationally, the system first computes wavelet-domain motion differences between consecutive frames, then extracts appearance and motion features with a dual-path VGG-16 setup, clusters the fused features, and selects one representative frame per cluster. The selected 16 keyframes are then encoded by the frozen V-DWT-JEPA2 encoder, projected into the LLaMA embedding space, and decoded into natural-language sports descriptions.

A notable architectural choice is that the temporal bottleneck is introduced before transformer encoding rather than inside the transformer. A plausible implication is that the model depends heavily on the quality of keyframe selection, because the encoder never sees the discarded frames.

3. DWT-VGG16-LDA keyframe extraction

The keyframe extraction stage is the most distinctive component of SV3.3B. For each frame $F_i$, the method performs a 2-level Haar discrete wavelet transform and retains the approximation coefficients $A_i$ from the LL band (Kodathala et al., 23 Jul 2025). Motion difference between consecutive frames is then defined in the wavelet domain as

$N$0

These $N$1 tensors are converted into 3-channel motion maps for feature extraction. Appearance features are computed as $N$2, while motion features are computed as $N$3, where $N$4 denotes the 3-channel conversion of the motion map. The fused feature for frame $N$5 is then

$N$6

The selection procedure uses K-means on the fused features to produce $N$7 clusters and corresponding pseudo-labels. LDA is then applied as a discriminative dimensionality-reduction stage, after which one frame per cluster is chosen as the nearest neighbor to the cluster center in the LDA space:

$N$8

The paper also includes the canonical LDA objective for context,

$N$9

with $K \ll N$0 and $K \ll N$1 denoting between-class and within-class scatter matrices induced by the K-means pseudo-labels. The details explicitly note that this canonical objective is provided for reference rather than as an explicit equation from the paper.

Several implementation details remain unspecified. The paper does not provide explicit 2D DWT coefficient formulas, does not identify which VGG-16 layers are used for feature extraction, and does not report the dimensionality of the appearance or motion features before concatenation. Even so, the procedural role of the module is clear: it is intended to surface frames that best preserve the multi-phase structure of sports actions rather than uniformly sampling time.

4. V-DWT-JEPA2 encoder and language decoder

After keyframe selection, the 16 frames are passed to the V-DWT-JEPA2 encoder, described as a ViT-L model with approximately 300M parameters and a 1024-dimensional output representation (Kodathala et al., 23 Jul 2025). The encoder operates on 16-frame clips at $K \ll N$2 resolution, with patch size 16 and tubelet size 2.

Pretraining uses JEPA2-style mask-denoising objectives. The masking policy applies two spatial mask scales over blocks: 8 blocks with spatial scale $K \ll N$3 and 2 blocks with $K \ll N$4, with temporal scale $K \ll N$5 and aspect ratios in $K \ll N$6. The paper does not provide the exact mathematical form of the JEPA2 loss, so the training objective is specified only at the level of mask-denoising in representation space.

The encoder output is mapped into the LLM embedding space by a two-layer MLP with the structure

$K \ll N$7

using ReLU, dropout, and layer normalization. The decoder is LLaMA-3.2-3B, fine-tuned with LoRA of rank 16 applied to all linear layers, specifically Q/K/V/O and MLP gate/up/down projections.

During decoder fine-tuning, the video encoder is frozen. This produces a modular architecture in which video representation learning and language adaptation are separated. The decoder loss is not explicitly stated in the paper. Cross-entropy for next-token prediction is plausible, but the details note that the paper does not confirm it.

5. Training regimen, data, and reproducibility

The encoder is pretrained on unlabeled sports-domain videos using 16-frame clips selected by DWT-VGG16-LDA (Kodathala et al., 23 Jul 2025). Reported pretraining settings are batch size 4, bfloat16 precision, 10 epochs, and 300 iterations per epoch, for a total of 3000 iterations. Resolution is $K \ll N$8, with patch size 16 and tubelet size 2.

The decoder fine-tuning stage uses the frozen encoder, the 2-layer projection MLP, and the LoRA-adapted LLaMA-3.2-3B decoder. Reported settings are batch size 8, learning rate $K \ll N$9, and the use of warmup steps and gradient clipping, although the warmup counts and clipping thresholds are not specified.

Evaluation is performed on a subset of the NSVA basketball dataset. The original NSVA is described as containing 32,019 clips from 132 games across 10 teams in the 2018–2019 season, curated from NBA public web content. The SV3.3B subset contains 1,315 clips, split into 1,050 training clips and 265 validation clips. Player names are removed from annotations to avoid identity bias, and the action strings are concise, such as “MISS 3' Layup,” “OFFENSIVE REBOUND,” and “17' Jump Shot.” Preprocessing uses 4 fps sampling to construct 16-frame clips.

Reproducibility is partial rather than complete. The details state that random seeds, optimizer types, weight decay, clipping thresholds, augmentation details, total training time, and checkpoint URLs are not provided. No example inference code is included, though a high-level inference procedure is given.

6. Evaluation methodology and reported metrics

The paper evaluates SV3.3B with both standard text-generation metrics and sports-specific metrics (Kodathala et al., 23 Jul 2025). The traditional metrics are ROUGE-L F1, BLEU, BERT F1, Content F1, and Semantic Similarity. These are aggregated into the Ground Truth Validation Score, abbreviated GT, defined as the sum of those five metrics.

Sports-specific evaluation is organized around six metrics: Information Density, Action Complexity, Measurement Precision, Sequence Length, Vocabulary Richness, and Technical Coverage. Information Density is explicitly defined as the ratio of technical terms to total words. If $y$0 is the set of technical tokens and $y$1 is the total token set in a generated description, then

$y$2

The Information Richness Score, abbreviated IR, aggregates those six sports-specific metrics. The Combined Score is then defined as

$y$3

This metric design reflects the paper’s aim to reward descriptions that are not merely semantically similar to references, but also dense in technical terminology, chronologically structured, and quantitatively specific. In evaluation terms, SV3.3B is therefore optimized for analytically rich action narration rather than generic caption fluency alone.

7. Reported performance, positioning, and limitations

On the NSVA subset, SV3.3B reports a GT Validation Score of 2.1239, an IR Score of 160.6970, and a Combined Score of 162.8209 (Kodathala et al., 23 Jul 2025). The individual text metrics are reported as ROUGE-L F1 $y$4, BLEU $y$5, BERT F1 $y$6, Content F1 $y$7, and Semantic Similarity $y$8. The sports-specific metrics are Information Density $y$9, Action Complexity $K=16$0, Measurement Precision $K=16$1, Sequence Length $K=16$2, Vocabulary Richness $K=16$3, and Technical Coverage $K=16$4.

Relative to baselines using uniformly sampled frames, the paper reports GPT-4o at GT $K=16$5, GPT-4o Mini at GT $K=16$6, and GPT-o4. Mini at GT $K=16$7. The abstract states that SV3.3B achieves a 29.2% improvement over GPT-4o in GT metrics, and the details note that this is consistent with the Table I calculation $K=16$8. Statistical comparisons are also reported: against GPT-4o, the GT difference is $K=16$9 with Cohen’s $K=16$0 and 95% CI $K=16$1; against GPT-4o Mini, the GT difference is $K=16$2 with Cohen’s $K=16$3 and 95% CI $K=16$4; against GPT-o4. Mini, the GT difference is $K=16$5 with Cohen’s $K=16$6 and 95% CI $K=16$7.

The paper also reports qualitative evidence that DWT-VGG16-LDA keyframes better capture biomechanical phases than uniform sampling or color histogram-based selection in examples such as basketball free throws and golf swings. However, no quantitative ablation tables are given for varying $K=16$8, DWT levels, or masking ratios.

Several limitations are explicitly acknowledged in the details. No FLOPs, memory footprint, latency, throughput, energy measurements, or hardware-specific on-device benchmarks are reported, so on-device feasibility is argued architecturally rather than empirically. Failure modes are not itemized, robustness to lighting, occlusion, and motion blur is not quantitatively assessed, and the paper does not provide detailed optimizer or checkpoint information. There is also an evaluation caveat: the reported tables contain inconsistent labels or values for GPT-4o variants across Table I and Table III. That inconsistency does not invalidate the central claim that the model outperforms the listed baselines on the reported NSVA subset, but it does narrow the precision with which baseline ranking should be interpreted.

In methodological terms, SV3.3B is best understood as a motion-aware video-language system for sports analysis in which temporal compression is delegated to a wavelet-and-clustering front end, representation learning is handled by a JEPA2-pretrained ViT-L encoder, and domain-adapted text generation is produced by a LoRA-tuned 3B LLM. Its future directions are stated as expansion to additional sports domains, handling longer temporal sequences, and integrating real-time feedback mechanisms for coaching.