---
title: 'BSTFormer: Badminton Stroke Transformer'
url: https://www.emergentmind.com/topics/bstformer
type: topic
---

# BSTFormer: Badminton Stroke Transformer

Searching arXiv for the BST paper and closely related badminton stroke-recognition work to ground the article in current literature.
BSTFormer, abbreviated in the source paper as BST, denotes the **Badminton Stroke-type Transformer**, a skeleton-based action recognition framework for **badminton stroke classification in broadcast match videos**. It is designed for the fine-grained recognition setting in which stroke identity is determined not only by a player’s body motion but also by the interaction between **player skeleton joints**, **shuttlecock trajectories**, and **player positions**. The framework couples a clip-construction strategy centered on stroke events with Transformer-based fusion, and the paper reports that its strongest variant, BST-3, achieves the best reported performance on **ShuttleSet**, the largest publicly available badminton video dataset [2502.21085].

## 1. Task formulation and domain-specific difficulty

BSTFormer addresses the task of classifying the **stroke type** executed by a player in a badminton singles rally from a broadcast match video. The paper frames this as a fine-grained interaction problem rather than a generic action-recognition problem, because the decisive evidence for a label is distributed across the target player, the opponent, and the shuttlecock trajectory [2502.21085].

The paper emphasizes five sources of difficulty. First, badminton contains **very fast, brief motions**: a stroke is short and often subtle. Second, the label space is **fine-grained**; examples given in the paper include distinctions such as smash versus wrist smash and return net versus defensive return drive. Third, **broadcast-video complexity** introduces spectators, referees, camera motion, occlusions, and large player separation in the frame. Fourth, **generic skeleton-based action recognition models are not enough**, because broad human-action categories tend to collapse badminton-specific swing variations into generic “hitting” motions. Fifth, **multi-person interaction matters**, since the opponent’s movement and shuttle response are informative for the target player’s stroke type [2502.21085].

A central thesis of the method is that **shuttlecock trajectory is not just auxiliary but a primary cue** for stroke recognition. This claim is foundational to the architecture: rather than treating shuttle dynamics as side information appended late in the pipeline, BSTFormer makes them a principal source for cross-attentional fusion [2502.21085].

## 2. Video segmentation and temporal context design

The full pipeline begins by segmenting a broadcast match into **rallies**, then splitting each rally into **stroke clips**. A rally is defined as a sequence of strokes from a serve until the point ends, and each stroke is associated with a **hit frame** \(h_j^{(i)}\), the frame where the shuttle is in contact with or closest to the racket. The paper writes a rally \(i\) as
\[
R_i=\{S_j^{(i)}\}_{j=1}^{n^{(i)}}
\]
where \(n^{(i)}\) is the number of strokes in rally \(i\) [2502.21085].

The paper contrasts a baseline **fixed-width clipping** strategy with its proposed **variable-width clipping** strategy. In the fixed-width case, a clip is centered around the hit frame,
\[
C_j^{(i)}=\{F_{h_j^{(i)}-t}, F_{h_j^{(i)}-t+1},\ldots,F_{h_j^{(i)}+t}\},
\]
which guarantees that the target stroke is centered but may include either too little context or too many unrelated neighboring strokes [2502.21085].

The proposed segmentation instead constructs a clip intended to include the **previous stroke**, the **target stroke**, and the **next stroke**. The boundaries are defined as
\[
\hat{h}^{(i)}_{j-1}=
\begin{cases}
h^{(i)}_{j-1}, & j>1 \\
h^{(i)}_j-t, & j=1
\end{cases}
\]
and
\[
\hat{h}^{(i)}_{j+1}=
\begin{cases}
h^{(i)}_{j+1}+\epsilon, & j<n^{(i)} \\
h^{(i)}_j+t, & j=n^{(i)}
\end{cases}
\]
with \(\epsilon<t\), yielding
\[
\hat{C}_j^{(i)}=\{F_{\hat{h}^{(i)}_{j-1}},F_{\hat{h}^{(i)}_{j-1}+1},\ldots,F_{\hat{h}^{(i)}_{j+1}}\}.
\]
The paper interprets this clip as capturing three phases: **preparation / reaction** as the shuttle approaches, **the target stroke itself**, and **opponent response after the stroke** [2502.21085].

This segmentation mechanism is not an incidental preprocessing step. The reported experiments evaluate models both **without the proposed segmentation strategy** and **after the proposed segmentation strategy**, and the paper states that the strategy improves BST variants significantly by focusing on the relevant temporal context [2502.21085].

## 3. Input modalities and feature extraction

After clip construction, BSTFormer extracts three input modalities: **player poses**, **player positions**, and **shuttlecock trajectories** [2502.21085].

For pose extraction, the implementation uses **MMPose**, specifically **RTMPose** for 2D pose estimation and **MotionBERT** for 3D pose estimation. The paper ultimately uses **2D poses** as the main input because 3D pose estimates were found inferior. The stated reason is that 3D estimation is biased by general pose priors and may distort badminton-specific body orientation; because 3D is inferred from 2D, it can also introduce extra error [2502.21085].

For player positions, the method uses **court lines** to determine which detected skeleton belongs to which player, thereby isolating the two players from other humans in the frame. Court line information is available in ShuttleSet, and the paper notes that if court lines are not available, MonoTrack’s court line method can be used. Position information is later incorporated through **Pose Position Fusion (PPF)** [2502.21085].

For shuttle modeling, the framework uses **TrackNetV3** in its attention-based version to extract a sequence of **2D shuttlecock coordinates** for each clip. The architecture treats this trajectory stream as central rather than peripheral, and the paper’s ablations argue that shuttlecock trajectory provides a stronger gain than position-only fusion [2502.21085].

A practical note concerns upstream event localization. If hit frames or rally boundaries are not provided, the paper notes that they can be estimated using **HitNet** from MonoTrack for hit-frame detection and rally detection based on larger inter-rally time gaps [2502.21085]. This implies that BSTFormer is not a single monolithic recognizer; it is situated within a broader sports-video analysis stack in which temporal segmentation, pose estimation, court parsing, and shuttle tracking all affect downstream stroke classification.

## 4. Transformer architecture and model variants

BST is described as a Transformer-based model derived from **TemPose**, but with a stronger emphasis on shuttlecock trajectory. The paper defines four variants: **BST-0**, **BST-1**, **BST-2**, and **BST-3** [2502.21085].

BST-0 is the base model and uses **pose + shuttlecock trajectory**. It takes three latent streams as input: pose latent for player 1, pose latent for player 2, and shuttlecock trajectory latent. A **TCN module** first processes the temporal sequence, after which a **Transformer Encoder** is applied. The encoder is given in pre-norm form:
\[
\tilde{X}^{(l)} = X^{(l-1)} + \text{MHSA}(\text{split}(\text{LN}(X^{(l-1)}))),
\]
\[
X^{(l)} = \tilde{X}^{(l)} + \text{FFN}(\text{LN}(\tilde{X}^{(l)})).
\]
The model defines multi-head attention through
\[
Q = [Q_1\ \cdots\ Q_H]=XW_Q,\quad
K = [K_1\ \cdots\ K_H]=XW_K,\quad
V = [V_1\ \cdots\ V_H]=XW_V,
\]
with per-head self-attention
\[
\text{SA}(Q_h,K_h,V_h)=\text{softmax}\!\left(\frac{Q_hK_h^{\top}}{\sqrt{D_A}}\right)V_h,
\]
and multi-head aggregation
\[
\text{MHSA}(Q,K,V)=\left[\cdots \text{SA}(Q_h,K_h,V_h)\cdots\right]W_O.
\]
According to the paper, attention is intended to learn temporal dependencies within the stroke clip, interactions between player motion and shuttle trajectory, and which parts of the sequence are most informative for stroke type [2502.21085].

A key architectural feature is a **Cross Transformer layer** with **Multi-Head Cross Attention (MHCA)**. In this layer, **queries** come from player pose latent or player position latent, while **keys** and **values** come from the shuttlecock trajectory latent. The design question is explicitly trajectory-centric: given the shuttle trajectory, which player-motion features are most relevant? This produces two pathways, one for trajectory with player 1 and one for trajectory with player 2, followed by a second Transformer Encoder. Final class representations are passed through an **MLP head** to produce logits, with
\[
p = \text{softmax}(\text{logits}),
\]
and training uses **cross-entropy loss** [2502.21085].

For models BST-1, BST-2, and BST-3, the architecture also uses **Pose Position Fusion (PPF)**. The paper describes PPF as an MLP-based module that produces fusion coefficients which are multiplied back into the original pose features, so that positions modulate pose features. The stated utility is that player position conveys which side the target player is on, the approximate relation to shuttle movement, and tactical context of the rally [2502.21085].

The four variants are summarized below.

| Variant | Distinguishing mechanism | Inputs emphasized |
|---|---|---|
| BST-0 | Base model with TCN + Transformer + trajectory-centric fusion | Pose + shuttlecock trajectory |
| BST-1 | **Clean Gate** for trajectory purification | Pose + trajectory + position via PPF |
| BST-2 | Cosine-similarity-based player weighting | Pose + trajectory + position via PPF |
| BST-3 | Combination of BST-1 and BST-2 | Pose + trajectory + position via PPF |

BST-1 adds a **Clean Gate** to reduce noise in trajectory information. Its motivation is that shuttle trajectories can be noisy because the opponent’s stroke is also present in the clip. The Clean Gate uses three inputs: the class token for shuttlecock information, the fused class token for shuttle + player 1, and the fused class token for shuttle + player 2. The latter two are processed through **min pooling** and an **MLP** to generate gating values that adjust the shuttlecock class token. The intended effect is to “purify” shuttle information using interaction strength with each player [2502.21085].

BST-2 instead introduces **cosine-similarity-based weighting**. It compares cosine similarity between the shuttle token and the player 1 fusion token, and between the shuttle token and the player 2 fusion token. Their difference is denoted \(\tilde{\alpha}\), with
\[
\tilde{\alpha}\in[-2,2],
\]
and is normalized as
\[
\alpha = \frac{\tilde{\alpha}+2}{4}, \qquad \alpha\in[0,1].
\]
Here, \(\alpha\) weights the **target player** and \(1-\alpha\) weights the **opponent**, so the player more strongly associated with the shuttle trajectory receives greater influence [2502.21085].

BST-3 combines the **trajectory purification** idea of BST-1 with the **player-importance weighting** idea of BST-2. The paper identifies BST-3 as the strongest variant [2502.21085].

## 5. Dataset, label space, and evaluation protocol

The principal evaluation benchmark is **ShuttleSet**, described in the paper as the **largest publicly available badminton video dataset**. The reported dataset statistics are **104 sets**, **3,685 rallies**, **36,492 strokes**, **44 matches**, and **27 top-ranking male and female singles players**. After removing incorrect or problematic labels, the working subset contains **40 matches** and **33,429 strokes** [2502.21085].

The split used in the paper is **30 matches for training**, **5 matches for validation**, and **5 matches for testing**. ShuttleSet originally contains **19 stroke categories**, including “None”. The paper merges rare **driven flight** into “None”, and because stroke labels are not originally separated by top versus bottom player, it doubles all stroke categories except “None”. The resulting label space has **35 classes total**, including “None” [2502.21085].

Evaluation is conducted using **Accuracy**, **Macro-F1**, and **Top-2 Accuracy (Acc-2)**. Baselines include **ST-GCN**, **BlockGCN**, **SkateFormer**, **ProtoGCN**, **TemPose-V**, **TemPose-PF**, **TemPose-SF**, and **TemPose-TF**. The paper also uses the modality abbreviations **J** for joints, **B** for bones, **SP** for shuttlecock position input, and **PP** for player position input [2502.21085].

This evaluation design reflects the paper’s broader claim that badminton stroke recognition should be treated as a multi-modal, interaction-sensitive recognition problem. Accuracy captures overall label correctness, Macro-F1 is sensitive to category imbalance and class-wise performance, and Top-2 Accuracy is pertinent because several label confusions occur between visually similar stroke types [2502.21085].

## 6. Empirical results, ablations, and limitations

The main quantitative result reported in the paper is that **BST-3 after segmentation** achieves **0.7710 Accuracy**, **0.7042 Macro-F1**, and **0.9340 Acc-2** on the test set. Without the proposed segmentation strategy, **BST-3** reaches **0.7695 Accuracy**, **0.7043 Macro-F1**, and **0.9267 Acc-2**. The paper states that the strongest result outperforms the prior state of the art and supports the conclusion that **shuttlecock information is crucial** for badminton stroke recognition [2502.21085].

The key post-segmentation results reported for Transformer-based baselines and BST variants are as follows.

| Model | Accuracy | Macro-F1 |
|---|---:|---:|
| TemPose-SF | 0.7534 | 0.6833 |
| BST-0 | 0.7655 | 0.6976 |
| TemPose-TF | 0.7580 | 0.6902 |
| BST-1 | 0.7704 | 0.7026 |
| BST-2 | 0.7687 | 0.7000 |
| BST-3 | 0.7710 | 0.7042 |

The ablation findings articulate the paper’s hierarchy of informative modalities. First, **player positions help**: TemPose-PF improves over TemPose-V, which the paper interprets as evidence that player positions implicitly encode shuttle location and tactical behavior. Second, **shuttlecock trajectory helps even more**: TemPose-SF gives a stronger gain than position-only fusion, supporting the claim that shuttlecock trajectory is a **primary signal**, not merely auxiliary. Third, **combining both is best**. Fourth, the **segmentation strategy further improves all models** by controlling the temporal context seen by the classifier. Fifth, **2D pose is better than 3D pose** under the tested setup [2502.21085].

The paper’s error analysis shows that the most substantial confusions occur between closely related labels, such as **top smash vs top wrist smash** and **bottom smash vs bottom wrist smash**. It also notes that some classes, including **top defensive return drive**, have especially low recall. This suggests that the model captures broad motion structure but that extremely fine-grained stroke distinctions remain difficult [2502.21085].

Several limitations are made explicit. The framework **depends on upstream modules**, especially accurate hit-frame detection and accurate shuttlecock tracking. It inherits the present limitations of **2D/3D trajectory estimation**, with the paper noting that current 3D shuttlecock trajectory estimation is still difficult while 2D tracking is more mature. It also acknowledges **dataset granularity** issues, since some categories remain hard to distinguish, and a **scope** restriction: the work focuses on **singles badminton**, while extension to **doubles**, **tennis**, or **table tennis** is suggested but not demonstrated [2502.21085].

A common misconception in this problem area is that pose alone should be sufficient for stroke classification. The reported results argue against that view. In the formulation adopted by BSTFormer, stroke recognition is governed by the joint structure of player motion, court-relative positioning, and shuttle dynamics, with trajectory acting as a principal modality rather than a supplementary one [2502.21085].

Source: https://www.emergentmind.com/topics/bstformer