FineBadminton: Hierarchical Badminton Analysis

• FineBadminton is a hierarchically annotated badminton video dataset that organizes play into foundational actions, tactical semantics, and decision evaluations.
• It combines multi-modal inputs from broadcast footage with structured data such as shuttle trajectories and player localization to support nuanced spatio-temporal reasoning.
• The accompanying FBBench benchmark enables multi-frame evaluation and domain-specific training, significantly enhancing model performance in detailed sports analytics.

FineBadminton is a large-scale, multi-modal, and hierarchically annotated badminton video dataset designed for fine-grained sports video understanding, especially in settings where multimodal LLMs must move beyond coarse stroke recognition to nuanced spatio-temporal reasoning, tactic comprehension, and rally-level evaluation. It was introduced together with FBBench, a benchmark derived from the dataset, to address the scarcity of badminton resources that encode execution nuances, tactical semantics, and decision quality in a unified framework. Its central organizing principle is a three-level semantic hierarchy—Foundational Actions, Tactical Semantics, and Decision Evaluation—which mirrors expert badminton analysis from stroke identification through tactical interpretation to rally-wide judgment (He et al., 11 Aug 2025).

1. Concept and research motivation

FineBadminton was created to address a specific gap in badminton analytics: earlier resources typically stop at stroke-level labels or basic tracking and do not encode the subtleties of execution, intent, or decision quality required for expert analysis. The dataset therefore formalizes badminton understanding as a layered semantic problem in which low-level strokes are linked to trajectory and intent, and then to rally narratives and shot quality. This layered design is explicitly presented as a mirror of how human experts analyze matches: identify strokes, interpret tactical meanings, and assess decisions (He et al., 11 Aug 2025).

This design positions FineBadminton within a broader shift in sports video understanding. Earlier broadcast-video pipelines emphasized point segmentation, player detection, identification, and coarse stroke segmentation from standard broadcast footage rather than special camera systems or sensors (Ghosh et al., 2017). Later real-time systems extended this broadcast-only paradigm to live rally segmentation, player tracking, and movement analytics such as distance, speed, and heatmaps (Nilesh et al., 2023). FineBadminton inherits the broadcast-video setting but changes the representational target: instead of ending at movement or stroke recognition, it introduces hierarchical semantics and evaluative narratives.

A common misconception is to treat FineBadminton as merely another stroke classification corpus. The dataset is explicitly purpose-built to push understanding beyond coarse stroke recognition, and its benchmark is framed around multi-frame spatio-temporal reasoning and higher-order cognition rather than only per-stroke labeling (He et al., 11 Aug 2025).

2. Dataset composition, modalities, and coverage

FineBadminton contains 120 professional singles matches from the past five years, segmented into 3,215 rally clips comprising 33,325 strokes. The average rally lasts 12.4 seconds. Videos come from public BWF broadcast footage on YouTube, including major events such as the Olympics, World Championships, and Open series. The viewpoint is predominantly overhead “behind the court,” with a smaller subset of side-angle clips. All videos are at least 720p, and the frame rate is consistently 25 FPS (He et al., 11 Aug 2025).

The dataset is multi-modal. In addition to RGB video, it includes shuttle trajectory and coordinates obtained with TrackNetV3, court boundary and player localization from Fast R-CNN, detected hit frames, stroke type classification, and shuttle landing spot predictions. The structured data collection is denoted as $D$ and includes ball trajectories, player and court localizations, hit instances, stroke types, and landing spots (He et al., 11 Aug 2025).

Aspect	Value
Matches	120 professional singles matches
Rally clips	3,215
Strokes	33,325
Average rally duration	12.4 seconds
Frame rate	25 FPS
Video quality	At least 720p
Main viewpoint	Predominantly overhead “behind the court”
Modalities	RGB video plus structured data $D$

The paper does not specify a public train/validation/test split for the dataset itself. For model training, the authors constructed a training set of approximately 60,000 QA pairs from FineBadminton. FBBench uses its own fixed set of 2,563 QA pairs, and the paper does not explicitly distinguish public and private subsets. Licensing terms, download procedures, APIs or tools, and data format details are also not specified; the project homepage is the stated entry point for access (He et al., 11 Aug 2025).

3. Multi-level annotation hierarchy

FineBadminton organizes badminton semantics into three levels, and the terminology was designed with professional players. The first level, Foundational Actions, provides the physical building blocks for higher-level understanding by classifying individual strokes with granular execution detail. It comprises 11 primary stroke types expanded into 20 subtypes that capture execution nuances; a stroke such as a smash can, for example, be sub-typed into jump smash or slice smash. A single stroke can belong to multiple subcategories. Court context is discretized into 9 regions for landing or contact interpretation (He et al., 11 Aug 2025).

The second level, Tactical Semantics, connects physical actions to tactical reasoning. It includes trajectory and spatial descriptors such as “straight line,” “cross-court,” “passing shot,” “body shot,” and “overhead shot,” as well as outcome-related descriptors including “out of bounds” and “hitting into the net.” It also encodes player intent through labels such as “deception,” “defensive play,” and “passive/transitional shot.” The taxonomy contains 3 categories of player actions, 9 strategic classifications, 6 shot characteristic types, and 3 classes for the outcome of the last hit (He et al., 11 Aug 2025).

The third level, Decision Evaluation, shifts to rally-wide and decision-focused assessment. Each shot receives a quality score from 1 to 7 and a free-form textual comment averaging 37.4 words. Rally-wise commentary is expressed through “Overall Evaluation” with an average length of 63 words, “Scoring Reason” averaging 41 words, and “Losing Reason” averaging 44 words. These descriptions follow the broadcast viewpoint, so players are often referred to as “top” or “bottom,” and they are temporally aligned to hit frames in order to preserve causal relationships across shots (He et al., 11 Aug 2025).

This hierarchy differs from earlier badminton datasets such as ShuttleSet, which provides stroke-level records, player and shuttle locations, and an 18-class shot taxonomy for tactical analysis but does not introduce the same three-level semantics or shot-wise and rally-wise evaluative text generation (Wang et al., 2023). It also differs from the interval-annotated “Fine-Badminton” temporal action localization dataset, which focuses on temporally localized action intervals across 29 categories rather than hierarchical tactic and decision annotations (Wang et al., 22 May 2026).

4. Annotation pipeline and quality control

The construction pipeline has three stages and combines automated proposal generation with human refinement. In Stage 1, Structural Parsing, the system extracts the spatio-temporal elements required to understand play. Shuttle trajectory detection uses TrackNetV3, court and player localization uses Fast R-CNN, and hit-event detection together with visual feature extraction uses VideoMAE fine-tuned on badminton. These outputs are combined into the structured representation $D$, and this stage undergoes manual verification before higher-level semantics are produced (He et al., 11 Aug 2025).

In Stage 2, Semantic Interpretation, each stroke $j$ is represented by three frames around the hit instance $I_j$ together with associated structured data $D_j$. An MLLM then produces a fine-grained textual description $T_j$ for that stroke, covering execution qualities, trajectory choices, and preliminary assessment. In Stage 3, Evaluative Refinement, a refining LLM processes these per-stroke descriptions within a context window of size $w$ to generate contextualized descriptions $T'_j$, and a summarizing LLM converts the sequence $(T'_1, \ldots, T'_N)$ into a rally-wide evaluation including reasons for scoring or losing (He et al., 11 Aug 2025).

Quality control is multi-stage. In the initial phase, each sample is independently annotated by two trained annotators, and a senior annotator resolves conflicts and consolidates the labels. In the pipeline-assisted phase, automated outputs are reviewed and refined by two human annotators. The paper does not report Cohen’s $D$0 or other inter-annotator agreement statistics; instead, it emphasizes multiple rounds of review and refinement by trained annotators to ensure consistency and quality (He et al., 11 Aug 2025).

The reported failure modes are also integral to the design. Per-stroke descriptions can be myopic, which motivates the contextual window in Stage 3. Tracking and occlusion are addressed by choosing TrackNetV3 for robustness and by manually verifying structural outputs before semantic generation. Detailed prompt engineering for the pipeline is not disclosed, but the human-in-the-loop design is presented as a safeguard against misclassification and over-interpretation by MLLMs (He et al., 11 Aug 2025).

5. FBBench: benchmark design and evaluation protocol

FBBench is the evaluation benchmark derived from FineBadminton. It contains 12 task types grouped into four domains. The Count domain includes T1 Hitting Count, T2 Round Count, and T3 Detailed Hitting Count. The Action domain includes T4 Action Prediction, T5 Action Grounding, and T6 Action Classification. The Position domain includes T7 Moving Recognition, T8 Hitting Localization, and T9 Landing Prediction. The Cognition domain includes T10 Hitting Comment, T11 Round Comment, and T12 Pointer Recognition, which targets rally outcome reasoning (He et al., 11 Aug 2025).

The benchmark contains 2,563 QA pairs in total, of which 2,413 are multiple-choice and 150 are open-ended. Distractors are rule-based or LLM-generated, and answer length supervision is used to improve robustness. In the multiple-choice protocol, models return one letter; options are shuffled on re-evaluation to mitigate position bias, and GLM-4-air is used for answer normalization when outputs deviate from the expected format. Open-ended responses are free-form text and are scored by GPT-4.1 (He et al., 11 Aug 2025).

The evaluation metrics are deliberately simple and task-specific. For multiple-choice tasks, the primary metric is accuracy,

$D$1

Open-ended tasks are scored from 0 to 10 per response based on relevance, correctness, and completeness versus a reference answer, and the reported score is the sum over 150 questions for a maximum of 1,500. Other common metrics such as Precision, Recall, F1, mAP, IoU, and AUC are not used in FBBench. The paper gives the standard definition

$D$2

but explicitly notes that it is not part of the reported evaluation (He et al., 11 Aug 2025).

The tasks are designed to require multi-frame reasoning rather than isolated frame recognition. They include counting strokes across rallies, localizing hits temporally, identifying movement patterns, predicting landing using trajectory and context, and explaining rally outcomes using causal inference over several shots. This suggests that FBBench functions not only as a dataset-derived QA benchmark but also as a probe of whether a model can connect action recognition, spatial reasoning, and tactical interpretation within a single rally representation.

6. Baseline modeling and empirical results

The baseline proposed with FineBadminton is an optimized video-to-token transformation $D$3 intended to reduce redundancy from static backgrounds and non-critical frames while focusing the MLLM on pivotal moments and salient visual cues. Given a raw video $D$4 and query $D$5, the framework produces a compact token sequence $D$6 and then answers with $D$7. The transformation has two stages:

$D$8

where $D$9 is Hit-Centric Keyframe Selection and $D$0 is Coordinate-Guided Condensation (He et al., 11 Aug 2025).

Hit-Centric Keyframe Selection detects hit probability per frame using VideoMAE features, an attention-pooling projection, and a transformer temporal encoder to produce $D$1. It then selects $D$2 anchor frames at hits, partitions the video into inter-hit segments, and samples query frames from each segment. Coordinate-Guided Condensation preserves anchor-frame tokens and distills tokens from query frames using a ReSampler with learnable queries and coordinate-guided cross-attention. The coordinate-bias tensor $D$3 boosts attention to tokens aligned with detected game elements such as the ball, players, and court (He et al., 11 Aug 2025).

The condensed segment representation is defined as

$D$4

and the final token sequence is constructed by temporal concatenation:

$D$5

yielding

$D$6

The training set contains about 60,000 QA pairs, roughly 10% multiple-choice and the rest open-ended, and fine-tuning is performed as sequence-to-sequence generation. The visual encoder and LLM are fine-tuned via LoRA, while the ReSampler and hit-centric projection are fully trained. Open-source models were trained on 4 A100-40G GPUs (He et al., 11 Aug 2025).

The benchmark results are reported as correctly answered multiple-choice items and summed open-ended scores over 150 questions. They show that strong commercial models struggle, while badminton-specific adaptation produces large gains for open-source models.

Model	Multiple-choice	Open-ended
Gemini 2.5 Pro	872/2413 (36.14%)	731/1500 (48.73%)
Gemini 2.5 Pro + S	932/2413 (38.62%)	759/1500 (50.60%)
GPT-4.1	743/2413 (30.79%)	710/1500 (47.33%)
GPT-4.1 + S	782/2413 (32.41%)	715/1500 (47.67%)
Qwen2.5VL-7B	557/2413 (23.08%)	235/1500 (15.67%)
Qwen2.5VL-7B + S + C (fine-tuned)	1015/2413 (42.06%)	655/1500 (43.67%)
VideoLLaMA3-7B	359/2413 (14.88%)	208/1500 (13.87%)
VideoLLaMA3-7B + S + C (fine-tuned)	954/2413 (39.54%)	629/1500 (41.93%)

Ablation studies on VideoLLaMA3 multiple-choice performance further isolate the contribution of domain adaptation and the two strategies. “No strategies, no fine-tuning” achieves 359 overall; “Fine-tuning only” achieves 627; “Fine-tuning + S” achieves 737; “Fine-tuning + C” achieves 856; and “Fine-tuning + S + C” achieves 954. The paper states that these results show domain-specific fine-tuning is crucial, and that $D$7 and $D$8 yield complementary gains, with the combination best (He et al., 11 Aug 2025).

The most challenging tasks are T3 Detailed Hitting Count, T6 Action Classification, and the cognition tasks T10–T12. The reported pattern is that hit-centric sampling helps commercial models on counting and localization, whereas open-source models benefit substantially when both $D$9 and $j$0 are added and fine-tuned, to the point of surpassing commercial models in overall multiple-choice accuracy (He et al., 11 Aug 2025).

7. Relation to adjacent work, practical access, and limitations

FineBadminton sits within a larger badminton-analysis literature but occupies a distinct niche. ShuttleSet is the largest publicly available badminton singles dataset with annotated stroke-level records, containing 44 matches, 104 sets, 3,685 rallies, and 36,492 strokes with 18 shot classes, player positions, shuttle locations, and benchmark tasks such as shot influence, stroke forecasting, and movement forecasting (Wang et al., 2023). FineBadminton extends beyond that regime by introducing hierarchical semantics, free-form textual annotations at stroke and rally levels, and benchmark tasks aimed at tactical and decision-focused reasoning rather than only structured prediction (He et al., 11 Aug 2025).

A separate but similarly named resource, “Fine-Badminton,” is a temporal action localization dataset with 31 matches, 2,104 rallies, 27,597 annotated actions, and 29 fine-grained stroke categories, introduced together with the Decoupling Spatio-Temporal Adapter for interval-based action localization (Wang et al., 22 May 2026). FineBadminton and Fine-Badminton are therefore related in subject matter but differ in annotation unit, task definition, and benchmark design. FineBadminton centers on multi-level semantics and QA-style reasoning; Fine-Badminton centers on interval-level TAL.

The relationship to later stroke classification work is also specific. “TemPose-TF-ASF” does not introduce, use, or reference a dataset or framework named FineBadminton; its experiments are conducted on ShuttleSet, though the paper notes that its adjacent-stroke fusion module could be integrated into a FineBadminton-like pipeline (Liu et al., 4 May 2026). This suggests a methodological connection without implying empirical evaluation on FineBadminton itself.

The practical limitations of FineBadminton are explicitly acknowledged. The videos are predominantly overhead broadcast footage, side-angle clips are fewer, the frame rate is fixed at 25 FPS, and only singles matches are covered. Cohen’s $j$1 is not reported. Per-stroke MLLM descriptions can miss broader context, even though contextual window refinement mitigates this problem. Open-ended evaluation depends on GPT-4.1 as a grader, introducing dependence on an external model’s judgment. The paper also does not specify standardized dataset splits, license terms, download procedures, APIs, or official data format documentation, and directs readers to the project homepage for updates (He et al., 11 Aug 2025).

These limitations also imply the most immediate directions for extension. The paper frames future progress around broader viewpoints and match types such as doubles, standardized splits and APIs, explicit agreement metrics, and further refinement of prompting and cross-modal fusion to capture deception and tactical feints at scale. In that sense, FineBadminton is best understood not as a closed benchmark but as the beginning of a badminton-specific ecosystem for fine-grained multimodal reasoning (He et al., 11 Aug 2025).