BSTFormer: Badminton Stroke Transformer

• The paper introduces BSTFormer, a Transformer-based framework that integrates player poses, court positions, and shuttlecock trajectories to achieve state-of-the-art stroke classification.
• It employs a variable-width clip segmentation strategy to capture pre-stroke, stroke, and post-stroke phases, addressing rapid motion and complex broadcast environments.
• Empirical results on ShuttleSet show that incorporating shuttlecock trajectory and position cues significantly improves accuracy and robustness over traditional pose-based models.

Searching arXiv for the BST paper and closely related badminton stroke-recognition work to ground the article in current literature. BSTFormer, abbreviated in the source paper as BST, denotes the Badminton Stroke-type Transformer, a skeleton-based action recognition framework for badminton stroke classification in broadcast match videos. It is designed for the fine-grained recognition setting in which stroke identity is determined not only by a player’s body motion but also by the interaction between player skeleton joints, shuttlecock trajectories, and player positions. The framework couples a clip-construction strategy centered on stroke events with Transformer-based fusion, and the paper reports that its strongest variant, BST-3, achieves the best reported performance on ShuttleSet, the largest publicly available badminton video dataset (Chang, 28 Feb 2025).

1. Task formulation and domain-specific difficulty

BSTFormer addresses the task of classifying the stroke type executed by a player in a badminton singles rally from a broadcast match video. The paper frames this as a fine-grained interaction problem rather than a generic action-recognition problem, because the decisive evidence for a label is distributed across the target player, the opponent, and the shuttlecock trajectory (Chang, 28 Feb 2025).

The paper emphasizes five sources of difficulty. First, badminton contains very fast, brief motions: a stroke is short and often subtle. Second, the label space is fine-grained; examples given in the paper include distinctions such as smash versus wrist smash and return net versus defensive return drive. Third, broadcast-video complexity introduces spectators, referees, camera motion, occlusions, and large player separation in the frame. Fourth, generic skeleton-based action recognition models are not enough, because broad human-action categories tend to collapse badminton-specific swing variations into generic “hitting” motions. Fifth, multi-person interaction matters, since the opponent’s movement and shuttle response are informative for the target player’s stroke type (Chang, 28 Feb 2025).

A central thesis of the method is that shuttlecock trajectory is not just auxiliary but a primary cue for stroke recognition. This claim is foundational to the architecture: rather than treating shuttle dynamics as side information appended late in the pipeline, BSTFormer makes them a principal source for cross-attentional fusion (Chang, 28 Feb 2025).

2. Video segmentation and temporal context design

The full pipeline begins by segmenting a broadcast match into rallies, then splitting each rally into stroke clips. A rally is defined as a sequence of strokes from a serve until the point ends, and each stroke is associated with a hit frame $h_j^{(i)}$, the frame where the shuttle is in contact with or closest to the racket. The paper writes a rally $i$ as

$R_i=\{S_j^{(i)}\}_{j=1}^{n^{(i)}}$

where $n^{(i)}$ is the number of strokes in rally $i$ (Chang, 28 Feb 2025).

The paper contrasts a baseline fixed-width clipping strategy with its proposed variable-width clipping strategy. In the fixed-width case, a clip is centered around the hit frame,

$$C_j^{(i)}=\{F_{h_j^{(i)}-t}, F_{h_j^{(i)}-t+1},\ldots,F_{h_j^{(i)}+t}\},$$

which guarantees that the target stroke is centered but may include either too little context or too many unrelated neighboring strokes (Chang, 28 Feb 2025).

The proposed segmentation instead constructs a clip intended to include the previous stroke, the target stroke, and the next stroke. The boundaries are defined as

$$\hat{h}^{(i)}_{j-1}= \begin{cases} h^{(i)}_{j-1}, & j>1 \ h^{(i)}_j-t, & j=1 \end{cases}$$

and

$$\hat{h}^{(i)}_{j+1}= \begin{cases} h^{(i)}_{j+1}+\epsilon, & j<n^{(i)} \ h^{(i)}_j+t, & j=n^{(i)} \end{cases}$$

with $\epsilon<t$, yielding

$$\hat{C}_j^{(i)}=\{F_{\hat{h}^{(i)}_{j-1}},F_{\hat{h}^{(i)}_{j-1}+1},\ldots,F_{\hat{h}^{(i)}_{j+1}}\}.$$

The paper interprets this clip as capturing three phases: preparation / reaction as the shuttle approaches, the target stroke itself, and opponent response after the stroke (Chang, 28 Feb 2025).

This segmentation mechanism is not an incidental preprocessing step. The reported experiments evaluate models both without the proposed segmentation strategy and after the proposed segmentation strategy, and the paper states that the strategy improves BST variants significantly by focusing on the relevant temporal context (Chang, 28 Feb 2025).

3. Input modalities and feature extraction

After clip construction, BSTFormer extracts three input modalities: player poses, player positions, and shuttlecock trajectories (Chang, 28 Feb 2025).

For pose extraction, the implementation uses MMPose, specifically RTMPose for 2D pose estimation and MotionBERT for 3D pose estimation. The paper ultimately uses 2D poses as the main input because 3D pose estimates were found inferior. The stated reason is that 3D estimation is biased by general pose priors and may distort badminton-specific body orientation; because 3D is inferred from 2D, it can also introduce extra error (Chang, 28 Feb 2025).

For player positions, the method uses court lines to determine which detected skeleton belongs to which player, thereby isolating the two players from other humans in the frame. Court line information is available in ShuttleSet, and the paper notes that if court lines are not available, MonoTrack’s court line method can be used. Position information is later incorporated through Pose Position Fusion (PPF) (Chang, 28 Feb 2025).

For shuttle modeling, the framework uses TrackNetV3 in its attention-based version to extract a sequence of 2D shuttlecock coordinates for each clip. The architecture treats this trajectory stream as central rather than peripheral, and the paper’s ablations argue that shuttlecock trajectory provides a stronger gain than position-only fusion (Chang, 28 Feb 2025).

A practical note concerns upstream event localization. If hit frames or rally boundaries are not provided, the paper notes that they can be estimated using HitNet from MonoTrack for hit-frame detection and rally detection based on larger inter-rally time gaps (Chang, 28 Feb 2025). This implies that BSTFormer is not a single monolithic recognizer; it is situated within a broader sports-video analysis stack in which temporal segmentation, pose estimation, court parsing, and shuttle tracking all affect downstream stroke classification.

4. Transformer architecture and model variants

BST is described as a Transformer-based model derived from TemPose, but with a stronger emphasis on shuttlecock trajectory. The paper defines four variants: BST-0, BST-1, BST-2, and BST-3 (Chang, 28 Feb 2025).

BST-0 is the base model and uses pose + shuttlecock trajectory. It takes three latent streams as input: pose latent for player 1, pose latent for player 2, and shuttlecock trajectory latent. A TCN module first processes the temporal sequence, after which a Transformer Encoder is applied. The encoder is given in pre-norm form: $i$0

$i$1

The model defines multi-head attention through

$i$2

with per-head self-attention

$i$3

and multi-head aggregation

$i$4

According to the paper, attention is intended to learn temporal dependencies within the stroke clip, interactions between player motion and shuttle trajectory, and which parts of the sequence are most informative for stroke type (Chang, 28 Feb 2025).

A key architectural feature is a Cross Transformer layer with Multi-Head Cross Attention (MHCA). In this layer, queries come from player pose latent or player position latent, while keys and values come from the shuttlecock trajectory latent. The design question is explicitly trajectory-centric: given the shuttle trajectory, which player-motion features are most relevant? This produces two pathways, one for trajectory with player 1 and one for trajectory with player 2, followed by a second Transformer Encoder. Final class representations are passed through an MLP head to produce logits, with

$i$5

and training uses cross-entropy loss (Chang, 28 Feb 2025).

For models BST-1, BST-2, and BST-3, the architecture also uses Pose Position Fusion (PPF). The paper describes PPF as an MLP-based module that produces fusion coefficients which are multiplied back into the original pose features, so that positions modulate pose features. The stated utility is that player position conveys which side the target player is on, the approximate relation to shuttle movement, and tactical context of the rally (Chang, 28 Feb 2025).

The four variants are summarized below.

Variant	Distinguishing mechanism	Inputs emphasized
BST-0	Base model with TCN + Transformer + trajectory-centric fusion	Pose + shuttlecock trajectory
BST-1	Clean Gate for trajectory purification	Pose + trajectory + position via PPF
BST-2	Cosine-similarity-based player weighting	Pose + trajectory + position via PPF
BST-3	Combination of BST-1 and BST-2	Pose + trajectory + position via PPF

BST-1 adds a Clean Gate to reduce noise in trajectory information. Its motivation is that shuttle trajectories can be noisy because the opponent’s stroke is also present in the clip. The Clean Gate uses three inputs: the class token for shuttlecock information, the fused class token for shuttle + player 1, and the fused class token for shuttle + player 2. The latter two are processed through min pooling and an MLP to generate gating values that adjust the shuttlecock class token. The intended effect is to “purify” shuttle information using interaction strength with each player (Chang, 28 Feb 2025).

BST-2 instead introduces cosine-similarity-based weighting. It compares cosine similarity between the shuttle token and the player 1 fusion token, and between the shuttle token and the player 2 fusion token. Their difference is denoted $i$6, with

$i$7

and is normalized as

$i$8

Here, $i$9 weights the target player and $R_i=\{S_j^{(i)}\}_{j=1}^{n^{(i)}}$0 weights the opponent, so the player more strongly associated with the shuttle trajectory receives greater influence (Chang, 28 Feb 2025).

BST-3 combines the trajectory purification idea of BST-1 with the player-importance weighting idea of BST-2. The paper identifies BST-3 as the strongest variant (Chang, 28 Feb 2025).

5. Dataset, label space, and evaluation protocol

The principal evaluation benchmark is ShuttleSet, described in the paper as the largest publicly available badminton video dataset. The reported dataset statistics are 104 sets, 3,685 rallies, 36,492 strokes, 44 matches, and 27 top-ranking male and female singles players. After removing incorrect or problematic labels, the working subset contains 40 matches and 33,429 strokes (Chang, 28 Feb 2025).

The split used in the paper is 30 matches for training, 5 matches for validation, and 5 matches for testing. ShuttleSet originally contains 19 stroke categories, including “None”. The paper merges rare driven flight into “None”, and because stroke labels are not originally separated by top versus bottom player, it doubles all stroke categories except “None”. The resulting label space has 35 classes total, including “None” (Chang, 28 Feb 2025).

Evaluation is conducted using Accuracy, Macro-F1, and Top-2 Accuracy (Acc-2). Baselines include ST-GCN, BlockGCN, SkateFormer, ProtoGCN, TemPose-V, TemPose-PF, TemPose-SF, and TemPose-TF. The paper also uses the modality abbreviations J for joints, B for bones, SP for shuttlecock position input, and PP for player position input (Chang, 28 Feb 2025).

This evaluation design reflects the paper’s broader claim that badminton stroke recognition should be treated as a multi-modal, interaction-sensitive recognition problem. Accuracy captures overall label correctness, Macro-F1 is sensitive to category imbalance and class-wise performance, and Top-2 Accuracy is pertinent because several label confusions occur between visually similar stroke types (Chang, 28 Feb 2025).

6. Empirical results, ablations, and limitations

The main quantitative result reported in the paper is that BST-3 after segmentation achieves 0.7710 Accuracy, 0.7042 Macro-F1, and 0.9340 Acc-2 on the test set. Without the proposed segmentation strategy, BST-3 reaches 0.7695 Accuracy, 0.7043 Macro-F1, and 0.9267 Acc-2. The paper states that the strongest result outperforms the prior state of the art and supports the conclusion that shuttlecock information is crucial for badminton stroke recognition (Chang, 28 Feb 2025).

The key post-segmentation results reported for Transformer-based baselines and BST variants are as follows.

Model	Accuracy	Macro-F1
TemPose-SF	0.7534	0.6833
BST-0	0.7655	0.6976
TemPose-TF	0.7580	0.6902
BST-1	0.7704	0.7026
BST-2	0.7687	0.7000
BST-3	0.7710	0.7042

The ablation findings articulate the paper’s hierarchy of informative modalities. First, player positions help: TemPose-PF improves over TemPose-V, which the paper interprets as evidence that player positions implicitly encode shuttle location and tactical behavior. Second, shuttlecock trajectory helps even more: TemPose-SF gives a stronger gain than position-only fusion, supporting the claim that shuttlecock trajectory is a primary signal, not merely auxiliary. Third, combining both is best. Fourth, the segmentation strategy further improves all models by controlling the temporal context seen by the classifier. Fifth, 2D pose is better than 3D pose under the tested setup (Chang, 28 Feb 2025).

The paper’s error analysis shows that the most substantial confusions occur between closely related labels, such as top smash vs top wrist smash and bottom smash vs bottom wrist smash. It also notes that some classes, including top defensive return drive, have especially low recall. This suggests that the model captures broad motion structure but that extremely fine-grained stroke distinctions remain difficult (Chang, 28 Feb 2025).

Several limitations are made explicit. The framework depends on upstream modules, especially accurate hit-frame detection and accurate shuttlecock tracking. It inherits the present limitations of 2D/3D trajectory estimation, with the paper noting that current 3D shuttlecock trajectory estimation is still difficult while 2D tracking is more mature. It also acknowledges dataset granularity issues, since some categories remain hard to distinguish, and a scope restriction: the work focuses on singles badminton, while extension to doubles, tennis, or table tennis is suggested but not demonstrated (Chang, 28 Feb 2025).

A common misconception in this problem area is that pose alone should be sufficient for stroke classification. The reported results argue against that view. In the formulation adopted by BSTFormer, stroke recognition is governed by the joint structure of player motion, court-relative positioning, and shuttle dynamics, with trajectory acting as a principal modality rather than a supplementary one (Chang, 28 Feb 2025).