---
title: 'SoccerNet: Benchmark for Soccer Video Understanding'
url: https://www.emergentmind.com/topics/soccernet
type: topic
---

# SoccerNet: Benchmark for Soccer Video Understanding

SoccerNet is an open benchmark family for football broadcast video understanding that couples large-scale annotated data with standardized tasks, metrics, baselines, and annual challenges. It originated as a scalable action-spotting dataset of 500 complete soccer games from six main European leagues, covering three seasons from 2014 to 2017 and a total duration of 764 hours, with sparse temporal anchors for goals, yellow/red cards, and substitutions, and it later expanded into broader releases with around 300,000 manual annotations, 17 action classes, replay and camera-production labels, player-centric annotations, audio commentary transcripts, and game-state labels [1804.04527] [2011.13367] [2405.07354]. In current usage, SoccerNet denotes both the underlying datasets and the recurring open benchmarking effort spanning broadcast video understanding, field understanding, and player understanding [2309.06006] [2409.10587] [2607.07320].

## 1. Origins, scope, and benchmark identity

SoccerNet started in 2018 and has since grown through several versions rather than remaining a single fixed dataset [2410.01304]. The original release was motivated by the lack of large-scale, soccer-specific video understanding benchmarks and defined action spotting as the task of finding the anchors of soccer events in long untrimmed broadcasts [1804.04527]. Later releases broadened the scope from sparse event localization to holistic understanding of broadcast soccer videos, including replay grounding, camera-shot segmentation, camera calibration, tracking, re-identification, dense captioning, game state reconstruction, and multimodal audio-text augmentation [2011.13367] [2210.02365] [2309.06006].

A common misconception is that SoccerNet refers only to the original three-class action-spotting benchmark. The later challenge reports and dataset papers show that it functions as a benchmark suite with multiple task-specific branches, each with its own annotations and evaluation protocols [2210.02365] [2309.06006] [2508.19182].

| Component | Main content | Representative scale |
|---|---|---|
| Original SoccerNet | Action spotting on full matches | 500 games, 764 hours, 6,637 events |
| SoccerNet-v2 | Holistic broadcast annotations | around 300,000 annotations on 500 videos |
| SoccerNet-Tracking | Multiple object tracking | 201 sequences, 225,375 frames, ~3.65 million boxes |
| SoccerNet-GSR | Game state reconstruction on a minimap | 200 clips, 9.37 million line points, 2.36 million athlete positions |
| SoccerNet-Echoes | ASR commentary transcripts | 1100 half-match videos, roughly 825 hours of audio |
| SoccerNet-v3D | 3D scene understanding and ball localization | 4,051 multi-view systems |

This progression shows a shift from sparse temporal semantics toward joint temporal, geometric, player-centric, and multimodal reasoning. A plausible implication is that SoccerNet’s main scientific role is standardization across heterogeneous football-video tasks rather than optimization for any single annotation type.

## 2. Dataset generations and annotation regimes

The original SoccerNet benchmark is composed of 500 complete soccer games from the English Premier League, La Liga, Ligue 1, Bundesliga, Serie A, and the UEFA Champions League, covering the 2014–2017 period [1804.04527]. Its 6,637 temporal annotations were automatically parsed from online match reports at a one minute resolution for three main classes of events—Goal, Yellow/Red Card, and Substitution—and then manually refined to a one second resolution by anchoring them at a single timestamp following well-defined soccer rules. With an average of one event every 6.9 minutes, the original benchmark is explicitly designed around very sparse events in long videos [1804.04527].

SoccerNet-v2 substantially broadened annotation density and semantic scope. It releases around 300,000 manual annotations within 500 untrimmed broadcast soccer videos: 110,458 action timestamps over 17 action classes, 158,493 camera-shot changes across 13 camera types for 200 fully annotated games, and 32,932 replay-shot annotations linking replay segments back to an action timestamp [2011.13367]. The 17 action classes are ball out of play, throw-in, foul, indirect free-kick, clearance, shot on target, shot off target, corner, substitution, kick-off, yellow card, offside, direct free-kick, goal, penalty, yellow→red, and red card. For each action, a binary visibility flag distinguishes shown from unshown events, with approximately 18% marked unshown; for shot transitions, transition types are abrupt, fade, or logo, and shot context is live, replay-of-action, or other-replay [2011.13367].

The SoccerNet action-spotting line later used more than 550 complete broadcast games annotated with almost all types of actions that can occur in a football game, with 500 games public and 50 held out as a challenge test set [2410.01304]. This versioning is important: some papers describe the public corpus of 500 games, whereas others describe the action-spotting track as 500 public games plus 50 held-out challenge games [2011.13367] [2410.01304].

The annotation philosophy across versions is consistent. SoccerNet prefers precise anchor timestamps instead of full temporal intervals for many semantic events, and later branches preserve this design choice while adding specialized labels such as replay links, camera types, player identities, jersey numbers, pitch geometry, and commentary text [1804.04527] [2011.13367] [2405.07354].

## 3. Core task formulations and evaluation protocols

The foundational SoccerNet tasks are segment classification and action spotting [1804.04527]. In segment classification, one predicts the set of event classes occurring in a fixed-length chunk, originally one minute. In spotting, the objective is sharper: predict a single timestamp, or spot, for every occurrence of a class in the entire untrimmed broadcast. For the original three-class benchmark, both tasks are evaluated with mean Average Precision. For spotting, a prediction is correct if it lies within a tolerance window $\delta$ around the ground-truth anchor, and performance is summarized by
$$
\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),
$$
with $\Delta=\{5,10,20,30,40,50,60\}$ seconds [1804.04527].

SoccerNet-v2 generalizes this formalism to 17 classes and adds two broadcast-production tasks [2011.13367]. Action spotting retains the same structure, with
$$
\mathrm{mAP}(\delta)=\frac{1}{17}\sum_{c=1}^{17} AP_c(\delta).
$$
Camera-shot segmentation is evaluated with mean IoU over 13 camera classes, and boundary detection uses the same spotting machinery with a single tolerance $\delta=1$ second. Replay grounding takes a replay segment and the full game video as input and seeks the original live anchor time; it is evaluated by Average-AP over replay instances [2011.13367].

The 2022 SoccerNet challenges redefined action-spotting and replay-grounding evaluation to include tighter temporal accuracies [2210.02365]. For action spotting and replay grounding, the benchmark distinguishes a loose regime over $\delta\in\{5,10,15,\ldots,60\}$ seconds and a tight regime over $\delta\in\{1,2,3,4,5\}$ seconds. This change made precise temporal localization more central to leaderboard performance [2210.02365].

Baseline results document the increasing difficulty of the benchmark. On the original three-class task, the best model for classifying temporal segments of length one minute reaches 67.8% mAP, and the best spotting baseline reaches 49.7% Average-mAP [1804.04527]. On SoccerNet-v2, the reproduced action-spotting baselines report 18.6% for MaxPool, 31.4% for NetVLAD, 39.9% for AudioVid, and 40.7% for CALF on Average-mAP over the 17-class benchmark [2011.13367]. These numbers quantify the gap between the sparse three-class formulation and the denser, semantically broader 17-class setting.

## 4. Spatial reasoning, calibration, tracking, and game state reconstruction

SoccerNet’s spatial turn began with large-scale calibration and player-localization resources. A 2021 study distilled a commercial calibration tool, EVSXeebra, into a public neural calibration network trained on approximately 12,000 frames sampled at 2 fps from the 500 SoccerNet games, each associated with a homography $H_{\text{gt}}\in\mathbb{R}^{3\times3}$ mapping the field plane to the image [2104.09333]. That work released three representations of calibration plus player localization—top-view images, feature vectors, and player graphs—and integrated them into the CALF action-spotting pipeline. The reported test-set Average-mAP of the best fusion variant, “CC + ResNet-34 + FCL,” is 46.8%, compared with 40.7% for CALF [2104.09333]. This result links geometric understanding back to temporal event recognition.

SoccerNet-Tracking formalized soccer multiple object tracking with 200 sequences of 30 seconds each, representative of challenging soccer scenarios, together with one complete 45-minute half-time for long-term tracking studies [2204.06918]. The dataset contains 201 sequences, 225,375 frames, and approximately 3.65 million bounding boxes. It annotates five object classes—Player, Goalkeeper, Referee, Ball, and Other—and includes tracklet IDs, team side tags, and jersey numbers when visible [2204.06918]. The paper reports modest performance on the public test set, with the best HOTA approximately 58% without ground-truth detections and approximately 71% with ground-truth detections, and identifies fast motion and severe occlusion as persistent failure modes [2204.06918].

Single-image camera calibration was later benchmarked at larger scale in the SoccerNet Camera Calibration Challenge 2023. The associated dataset contains 25,506 frames at 960×540 px drawn from 500 real English Premier League games, with field-marking semantic labels and evaluation by Final Score $=\mathrm{Acc@5}\times \mathrm{CR}$, where $\mathrm{CR}$ is the completeness ratio [2410.07401]. The top private-leaderboard score reported for “Sportlight” is 0.5535 [2410.07401].

Game State Reconstruction (GSR) extends spatial reasoning from calibration and tracking to minimap reconstruction [2404.11335]. SoccerNet-GSR is composed of 200 video sequences of 30 seconds, annotated with 9.37 million line points for pitch localization and camera calibration and over 2.36 million athlete positions on the pitch with role, team, jersey number, and track identity. A GSR method must output, for each frame, detections of the form
$$
d_i^t=\{\text{pitch\_x},\ \text{pitch\_y},\ \text{role},\ \text{team},\ \text{jersey\_number}\}.
$$
The benchmark introduces GS-HOTA, which replaces IoU-based matching with a product of localization and identity similarity:
$$
\mathrm{Sim}_{\mathrm{GS}}(P,G)=\mathrm{LocSim}(P,G)\times \mathrm{IdSim}(P,G),
$$
where
$$
\mathrm{LocSim}(P,G)=\exp\Bigl(\ln(0.05)\,\frac{\|P-G\|_2^2}{\tau^2}\Bigr)
$$
with $\tau=5$ m, and $\mathrm{IdSim}(P,G)=1$ only if role, team, and jersey match exactly [2404.11335]. The released end-to-end baseline reports 22.26% GS-HOTA on the test split, and the main failure cases are insufficient pitch lines causing calibration drift and small, occluded jerseys causing OCR misses [2404.11335]. A later challenge-winning pipeline reported 63.81 GS-HOTA, 49.52 GS-DetA, and 82.23 GS-AssA, showing that post-processing, calibration, and attribute consistency dominate performance in this task [2504.06357].

## 5. Multimodal, 3D, and anticipatory extensions

SoccerNet-Echoes augments the benchmark with automatically generated transcriptions of audio commentaries from soccer game broadcasts, using Whisper and machine translation [2405.07354]. It covers all 1100 half-match videos in SoccerNet, corresponding to 550 full games and roughly 825 hours of broadcast audio, across six leagues and four seasons: 2014–15, 2015–16, 2016–17, and 2019–20 [2405.07354]. Each JSON file stores timestamped ASR segments, and the paper reports transcription performance against GOAL ground truth on 40 half-videos: Whisper large-v1 achieves WER 0.443, CER 0.261, and BLEU 54.50; Whisper large-v2 achieves WER 0.458, CER 0.269, and BLEU 52.59; Whisper large-v3 achieves WER 0.551, CER 0.341, and BLEU 47.97 [2405.07354]. The paper does not report multimodal ablations, but it explicitly positions the textual stream as support for action spotting, automatic caption generation, and game summarization.

SoccerNet-v3D extends SoccerNet-v3 with field-line-based camera calibration and multi-view synchronization for 3D scene understanding [2504.10106]. After filtering synchronized main-camera and replay pairs, SoccerNet-v3D retains 4,051 multi-view systems, split into 3,240 train and 811 test frames, each with a 3D ball center and an optimized 2D bounding box. The companion ISSIA-3D dataset contains 10,544 frames with 3D ball positions and generated boxes [2504.10106]. The benchmark defines a monocular 3D ball localization task and reports, on SoccerNet-v3D test, that a YOLO-based detector trained on optimized boxes plus Oracle field localization reaches AP@0.5 = 0.81, MAE$_m$ = 4.2 m, MAE$_\%$ = 5.5%, and $P_{2m}=0.30$ [2504.10106]. These results make explicit how replay-derived correspondences can be repurposed for 3D reconstruction.

Action anticipation pushes SoccerNet from retrospective localization toward future-event prediction [2504.12021]. The SoccerNet Ball Action Anticipation dataset is built from SoccerNet Ball Action Spotting and reuses the nine broadcast games of that source corpus, with 12,433 actions across 11.4 hours; for anticipation, the benchmark excludes “free-kick” and “goal,” yielding $C=10$ classes [2504.12021]. The task predicts future actions within a five- or ten-second anticipation window and introduces $mAP@\delta$ for temporal precision and $mAP@\infty$ for occurrence prediction. The FAANTRA baseline, adapted from FUTR, reports on test that the best variant using SN-BAA+SN-AS training and a 400MF backbone reaches an average 24.08 for $T_a=5$ s and 19.90 for $T_a=10$ s, while a T-DEED upper bound on observed future frames reaches 56.69 for $T_a=5$ s [2504.12021]. This suggests that future-action prediction remains substantially harder than post hoc spotting even when the action vocabulary is narrowed to ball-related events.

## 6. Annual challenges, leaderboard structure, and research trajectory

SoccerNet’s challenge series is central to its identity as an open benchmark. The 2022 edition contained six vision-based tasks: action spotting, replay grounding, pitch localization, camera calibration, player re-identification, and multiple object tracking [2210.02365]. The 2023 edition expanded to seven tasks and introduced ball action spotting, dense video captioning, and jersey number recognition while strengthening camera calibration and moving tracking toward end-to-end evaluation [2309.06006]. The 2024 edition focused on four vision-based tasks: Ball Action Spotting, Dense Video Captioning, Multi-View Foul Recognition, and Game State Reconstruction [2409.10587]. The 2025 edition contained Team Ball Action Spotting, Monocular Depth Estimation, Multi-View Foul Recognition, and Game State Reconstruction [2508.19182]. The 2026 edition contained Ball Action Anticipation, Player-Centric Ball Action Spotting, Novel View Synthesis, Spiideo SoccerNet Synloc, and Visual Question Answering, and recorded 427 teams, 1,129 submissions, and 28 reviewed technical reports [2607.07320].

The leaderboards quantify rapid progress within task-specific protocols. In SoccerNet 2025, Team Ball Action Spotting improved from a baseline Team-mAP@1 of 51.72 to a winning 60.03; Monocular Depth Estimation improved from RMSE $3.757\times10^{-3}$ to $2.418\times10^{-3}$; Multi-View Foul Recognition improved from Combined BA 36.99% to 52.22%; and Game State Reconstruction improved from 29.01% GS-HOTA to 63.90% [2508.19182]. In SoccerNet 2026, Ball Action Anticipation improved from a FAANTRA baseline of 16.76 mAP$_{avg}$ to 24.08; Player-Centric Ball Action Spotting improved from 46.41 to 58.94 Macro F1@0.15; Novel View Synthesis improved from a 3DGS baseline PSNR of 26.74 to 29.89; Synloc improved from 77.30 mAP-LocSim to 97.67; and VQA improved from a random baseline of 25.0% to 98.0% accuracy [2607.07320].

The action-spotting survey reports that more than 60 methods were developed or published over the past five years, improving on the first baselines and making action spotting a viable option for the sports industry [2410.01304]. That statement captures SoccerNet’s broader role: it is not merely a repository of football videos, but an evaluation framework in which task definitions, tolerances, public baselines, and yearly challenge tracks jointly determine what counts as progress.

Source: https://www.emergentmind.com/topics/soccernet