SoccerNet: Benchmark for Soccer Video Understanding
- SoccerNet is a comprehensive benchmark defining soccer video understanding with annotated datasets covering action spotting, tracking, and game state reconstruction.
- It integrates varied annotation regimes—from sparse event timestamps to detailed player and camera calibration—paired with rigorous evaluation protocols.
- The evolving challenge series drives progress in multimodal analysis, spatial reasoning, and anticipatory action prediction in soccer broadcasts.
SoccerNet is an open benchmark family for football broadcast video understanding that couples large-scale annotated data with standardized tasks, metrics, baselines, and annual challenges. It originated as a scalable action-spotting dataset of 500 complete soccer games from six main European leagues, covering three seasons from 2014 to 2017 and a total duration of 764 hours, with sparse temporal anchors for goals, yellow/red cards, and substitutions, and it later expanded into broader releases with around 300,000 manual annotations, 17 action classes, replay and camera-production labels, player-centric annotations, audio commentary transcripts, and game-state labels (Giancola et al., 2018, Deliège et al., 2020, Gautam et al., 2024). In current usage, SoccerNet denotes both the underlying datasets and the recurring open benchmarking effort spanning broadcast video understanding, field understanding, and player understanding (Cioppa et al., 2023, Cioppa et al., 2024, Cioppa et al., 8 Jul 2026).
1. Origins, scope, and benchmark identity
SoccerNet started in 2018 and has since grown through several versions rather than remaining a single fixed dataset (Giancola et al., 2024). The original release was motivated by the lack of large-scale, soccer-specific video understanding benchmarks and defined action spotting as the task of finding the anchors of soccer events in long untrimmed broadcasts (Giancola et al., 2018). Later releases broadened the scope from sparse event localization to holistic understanding of broadcast soccer videos, including replay grounding, camera-shot segmentation, camera calibration, tracking, re-identification, dense captioning, game state reconstruction, and multimodal audio-text augmentation (Deliège et al., 2020, Giancola et al., 2022, Cioppa et al., 2023).
A common misconception is that SoccerNet refers only to the original three-class action-spotting benchmark. The later challenge reports and dataset papers show that it functions as a benchmark suite with multiple task-specific branches, each with its own annotations and evaluation protocols (Giancola et al., 2022, Cioppa et al., 2023, Giancola et al., 26 Aug 2025).
| Component | Main content | Representative scale |
|---|---|---|
| Original SoccerNet | Action spotting on full matches | 500 games, 764 hours, 6,637 events |
| SoccerNet-v2 | Holistic broadcast annotations | around 300,000 annotations on 500 videos |
| SoccerNet-Tracking | Multiple object tracking | 201 sequences, 225,375 frames, ~3.65 million boxes |
| SoccerNet-GSR | Game state reconstruction on a minimap | 200 clips, 9.37 million line points, 2.36 million athlete positions |
| SoccerNet-Echoes | ASR commentary transcripts | 1100 half-match videos, roughly 825 hours of audio |
| SoccerNet-v3D | 3D scene understanding and ball localization | 4,051 multi-view systems |
This progression shows a shift from sparse temporal semantics toward joint temporal, geometric, player-centric, and multimodal reasoning. A plausible implication is that SoccerNet’s main scientific role is standardization across heterogeneous football-video tasks rather than optimization for any single annotation type.
2. Dataset generations and annotation regimes
The original SoccerNet benchmark is composed of 500 complete soccer games from the English Premier League, La Liga, Ligue 1, Bundesliga, Serie A, and the UEFA Champions League, covering the 2014–2017 period (Giancola et al., 2018). Its 6,637 temporal annotations were automatically parsed from online match reports at a one minute resolution for three main classes of events—Goal, Yellow/Red Card, and Substitution—and then manually refined to a one second resolution by anchoring them at a single timestamp following well-defined soccer rules. With an average of one event every 6.9 minutes, the original benchmark is explicitly designed around very sparse events in long videos (Giancola et al., 2018).
SoccerNet-v2 substantially broadened annotation density and semantic scope. It releases around 300,000 manual annotations within 500 untrimmed broadcast soccer videos: 110,458 action timestamps over 17 action classes, 158,493 camera-shot changes across 13 camera types for 200 fully annotated games, and 32,932 replay-shot annotations linking replay segments back to an action timestamp (Deliège et al., 2020). The 17 action classes are ball out of play, throw-in, foul, indirect free-kick, clearance, shot on target, shot off target, corner, substitution, kick-off, yellow card, offside, direct free-kick, goal, penalty, yellow→red, and red card. For each action, a binary visibility flag distinguishes shown from unshown events, with approximately 18% marked unshown; for shot transitions, transition types are abrupt, fade, or logo, and shot context is live, replay-of-action, or other-replay (Deliège et al., 2020).
The SoccerNet action-spotting line later used more than 550 complete broadcast games annotated with almost all types of actions that can occur in a football game, with 500 games public and 50 held out as a challenge test set (Giancola et al., 2024). This versioning is important: some papers describe the public corpus of 500 games, whereas others describe the action-spotting track as 500 public games plus 50 held-out challenge games (Deliège et al., 2020, Giancola et al., 2024).
The annotation philosophy across versions is consistent. SoccerNet prefers precise anchor timestamps instead of full temporal intervals for many semantic events, and later branches preserve this design choice while adding specialized labels such as replay links, camera types, player identities, jersey numbers, pitch geometry, and commentary text (Giancola et al., 2018, Deliège et al., 2020, Gautam et al., 2024).
3. Core task formulations and evaluation protocols
The foundational SoccerNet tasks are segment classification and action spotting (Giancola et al., 2018). In segment classification, one predicts the set of event classes occurring in a fixed-length chunk, originally one minute. In spotting, the objective is sharper: predict a single timestamp, or spot, for every occurrence of a class in the entire untrimmed broadcast. For the original three-class benchmark, both tasks are evaluated with mean Average Precision. For spotting, a prediction is correct if it lies within a tolerance window around the ground-truth anchor, and performance is summarized by
$\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$
with seconds (Giancola et al., 2018).
SoccerNet-v2 generalizes this formalism to 17 classes and adds two broadcast-production tasks (Deliège et al., 2020). Action spotting retains the same structure, with
Camera-shot segmentation is evaluated with mean IoU over 13 camera classes, and boundary detection uses the same spotting machinery with a single tolerance second. Replay grounding takes a replay segment and the full game video as input and seeks the original live anchor time; it is evaluated by Average-AP over replay instances (Deliège et al., 2020).
The 2022 SoccerNet challenges redefined action-spotting and replay-grounding evaluation to include tighter temporal accuracies (Giancola et al., 2022). For action spotting and replay grounding, the benchmark distinguishes a loose regime over seconds and a tight regime over seconds. This change made precise temporal localization more central to leaderboard performance (Giancola et al., 2022).
Baseline results document the increasing difficulty of the benchmark. On the original three-class task, the best model for classifying temporal segments of length one minute reaches 67.8% mAP, and the best spotting baseline reaches 49.7% Average-mAP (Giancola et al., 2018). On SoccerNet-v2, the reproduced action-spotting baselines report 18.6% for MaxPool, 31.4% for NetVLAD, 39.9% for AudioVid, and 40.7% for CALF on Average-mAP over the 17-class benchmark (Deliège et al., 2020). These numbers quantify the gap between the sparse three-class formulation and the denser, semantically broader 17-class setting.
4. Spatial reasoning, calibration, tracking, and game state reconstruction
SoccerNet’s spatial turn began with large-scale calibration and player-localization resources. A 2021 study distilled a commercial calibration tool, EVSXeebra, into a public neural calibration network trained on approximately 12,000 frames sampled at 2 fps from the 500 SoccerNet games, each associated with a homography mapping the field plane to the image (Cioppa et al., 2021). That work released three representations of calibration plus player localization—top-view images, feature vectors, and player graphs—and integrated them into the CALF action-spotting pipeline. The reported test-set Average-mAP of the best fusion variant, “CC + ResNet-34 + FCL,” is 46.8%, compared with 40.7% for CALF (Cioppa et al., 2021). This result links geometric understanding back to temporal event recognition.
SoccerNet-Tracking formalized soccer multiple object tracking with 200 sequences of 30 seconds each, representative of challenging soccer scenarios, together with one complete 45-minute half-time for long-term tracking studies (Cioppa et al., 2022). The dataset contains 201 sequences, 225,375 frames, and approximately 3.65 million bounding boxes. It annotates five object classes—Player, Goalkeeper, Referee, Ball, and Other—and includes tracklet IDs, team side tags, and jersey numbers when visible (Cioppa et al., 2022). The paper reports modest performance on the public test set, with the best HOTA approximately 58% without ground-truth detections and approximately 71% with ground-truth detections, and identifies fast motion and severe occlusion as persistent failure modes (Cioppa et al., 2022).
Single-image camera calibration was later benchmarked at larger scale in the SoccerNet Camera Calibration Challenge 2023. The associated dataset contains 25,506 frames at 960×540 px drawn from 500 real English Premier League games, with field-marking semantic labels and evaluation by Final Score , where is the completeness ratio (Falaleev et al., 2024). The top private-leaderboard score reported for “Sportlight” is 0.5535 (Falaleev et al., 2024).
Game State Reconstruction (GSR) extends spatial reasoning from calibration and tracking to minimap reconstruction (Somers et al., 2024). SoccerNet-GSR is composed of 200 video sequences of 30 seconds, annotated with 9.37 million line points for pitch localization and camera calibration and over 2.36 million athlete positions on the pitch with role, team, jersey number, and track identity. A GSR method must output, for each frame, detections of the form
$\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$0
The benchmark introduces GS-HOTA, which replaces IoU-based matching with a product of localization and identity similarity:
$\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$1
where
$\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$2
with $\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$3 m, and $\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$4 only if role, team, and jersey match exactly (Somers et al., 2024). The released end-to-end baseline reports 22.26% GS-HOTA on the test split, and the main failure cases are insufficient pitch lines causing calibration drift and small, occluded jerseys causing OCR misses (Somers et al., 2024). A later challenge-winning pipeline reported 63.81 GS-HOTA, 49.52 GS-DetA, and 82.23 GS-AssA, showing that post-processing, calibration, and attribute consistency dominate performance in this task (Golovkin et al., 8 Apr 2025).
5. Multimodal, 3D, and anticipatory extensions
SoccerNet-Echoes augments the benchmark with automatically generated transcriptions of audio commentaries from soccer game broadcasts, using Whisper and machine translation (Gautam et al., 2024). It covers all 1100 half-match videos in SoccerNet, corresponding to 550 full games and roughly 825 hours of broadcast audio, across six leagues and four seasons: 2014–15, 2015–16, 2016–17, and 2019–20 (Gautam et al., 2024). Each JSON file stores timestamped ASR segments, and the paper reports transcription performance against GOAL ground truth on 40 half-videos: Whisper large-v1 achieves WER 0.443, CER 0.261, and BLEU 54.50; Whisper large-v2 achieves WER 0.458, CER 0.269, and BLEU 52.59; Whisper large-v3 achieves WER 0.551, CER 0.341, and BLEU 47.97 (Gautam et al., 2024). The paper does not report multimodal ablations, but it explicitly positions the textual stream as support for action spotting, automatic caption generation, and game summarization.
SoccerNet-v3D extends SoccerNet-v3 with field-line-based camera calibration and multi-view synchronization for 3D scene understanding (Gutiérrez-Pérez et al., 14 Apr 2025). After filtering synchronized main-camera and replay pairs, SoccerNet-v3D retains 4,051 multi-view systems, split into 3,240 train and 811 test frames, each with a 3D ball center and an optimized 2D bounding box. The companion ISSIA-3D dataset contains 10,544 frames with 3D ball positions and generated boxes (Gutiérrez-Pérez et al., 14 Apr 2025). The benchmark defines a monocular 3D ball localization task and reports, on SoccerNet-v3D test, that a YOLO-based detector trained on optimized boxes plus Oracle field localization reaches [email protected] = 0.81, MAE$\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$5 = 4.2 m, MAE$\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$6 = 5.5%, and $\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$7 (Gutiérrez-Pérez et al., 14 Apr 2025). These results make explicit how replay-derived correspondences can be repurposed for 3D reconstruction.
Action anticipation pushes SoccerNet from retrospective localization toward future-event prediction (Dalal et al., 16 Apr 2025). The SoccerNet Ball Action Anticipation dataset is built from SoccerNet Ball Action Spotting and reuses the nine broadcast games of that source corpus, with 12,433 actions across 11.4 hours; for anticipation, the benchmark excludes “free-kick” and “goal,” yielding $\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$8 classes (Dalal et al., 16 Apr 2025). The task predicts future actions within a five- or ten-second anticipation window and introduces $\mathrm{Average\mbox{-}mAP}=\frac{1}{|\Delta|}\sum_{\delta\in\Delta}\mathrm{mAP}(\delta),$9 for temporal precision and 0 for occurrence prediction. The FAANTRA baseline, adapted from FUTR, reports on test that the best variant using SN-BAA+SN-AS training and a 400MF backbone reaches an average 24.08 for 1 s and 19.90 for 2 s, while a T-DEED upper bound on observed future frames reaches 56.69 for 3 s (Dalal et al., 16 Apr 2025). This suggests that future-action prediction remains substantially harder than post hoc spotting even when the action vocabulary is narrowed to ball-related events.
6. Annual challenges, leaderboard structure, and research trajectory
SoccerNet’s challenge series is central to its identity as an open benchmark. The 2022 edition contained six vision-based tasks: action spotting, replay grounding, pitch localization, camera calibration, player re-identification, and multiple object tracking (Giancola et al., 2022). The 2023 edition expanded to seven tasks and introduced ball action spotting, dense video captioning, and jersey number recognition while strengthening camera calibration and moving tracking toward end-to-end evaluation (Cioppa et al., 2023). The 2024 edition focused on four vision-based tasks: Ball Action Spotting, Dense Video Captioning, Multi-View Foul Recognition, and Game State Reconstruction (Cioppa et al., 2024). The 2025 edition contained Team Ball Action Spotting, Monocular Depth Estimation, Multi-View Foul Recognition, and Game State Reconstruction (Giancola et al., 26 Aug 2025). The 2026 edition contained Ball Action Anticipation, Player-Centric Ball Action Spotting, Novel View Synthesis, Spiideo SoccerNet Synloc, and Visual Question Answering, and recorded 427 teams, 1,129 submissions, and 28 reviewed technical reports (Cioppa et al., 8 Jul 2026).
The leaderboards quantify rapid progress within task-specific protocols. In SoccerNet 2025, Team Ball Action Spotting improved from a baseline Team-mAP@1 of 51.72 to a winning 60.03; Monocular Depth Estimation improved from RMSE 4 to 5; Multi-View Foul Recognition improved from Combined BA 36.99% to 52.22%; and Game State Reconstruction improved from 29.01% GS-HOTA to 63.90% (Giancola et al., 26 Aug 2025). In SoccerNet 2026, Ball Action Anticipation improved from a FAANTRA baseline of 16.76 mAP6 to 24.08; Player-Centric Ball Action Spotting improved from 46.41 to 58.94 Macro [email protected]; Novel View Synthesis improved from a 3DGS baseline PSNR of 26.74 to 29.89; Synloc improved from 77.30 mAP-LocSim to 97.67; and VQA improved from a random baseline of 25.0% to 98.0% accuracy (Cioppa et al., 8 Jul 2026).
The action-spotting survey reports that more than 60 methods were developed or published over the past five years, improving on the first baselines and making action spotting a viable option for the sports industry (Giancola et al., 2024). That statement captures SoccerNet’s broader role: it is not merely a repository of football videos, but an evaluation framework in which task definitions, tolerances, public baselines, and yearly challenge tracks jointly determine what counts as progress.