VST-Pose: WiFi-Based Human Pose Estimation

• VST-Pose is a WiFi-based human pose estimation framework that reconstructs joint coordinates from Channel State Information, overcoming visual limitations like occlusion and privacy concerns.
• It employs a dual-stream spatiotemporal attention backbone called ViSTA-Former integrated with an explicit velocity modeling branch to capture short-term keypoint displacements.
• The method achieves high accuracy, with PCK@50 up to 92.25% on a smart home care dataset, demonstrating its effectiveness for privacy-aware indoor applications.

Searching arXiv for VST-Pose and closely related WiFi pose-estimation papers. VST-Pose is a WiFi-based human pose estimation framework for reconstructing human joint coordinates from WiFi Channel State Information rather than from images or depth sensors. It is presented as a non-visual alternative for continuous pose estimation in indoor environments, motivated by penetrability and privacy advantages, and is centered on a dual-stream spatiotemporal attention backbone called ViSTA-Former together with an explicit velocity modeling branch for short-term keypoint displacement learning (Zhang et al., 13 Jul 2025).

1. Definition and scope

VST-Pose addresses human pose estimation from WiFi CSI in settings where camera-based systems are limited by low light, occlusion, or privacy constraints. The framework aims to estimate a human skeleton from wireless signal fluctuations induced by body movement, and it is formulated for both 2D pose estimation on a self-collected smart home care dataset and 3D pose estimation on the public MMFi dataset (Zhang et al., 13 Jul 2025).

The method is described as end-to-end and supervised. Its core components are a CNN-based CSI encoder, a spatiotemporal attention backbone, a velocity modeling branch, late fusion between motion-sensitive and keypoint-sensitive features, and MLP decoders for pose coordinates and velocity. This design targets three difficulties emphasized by the paper: the indirect relation between CSI and body joints, the need for temporally coherent predictions rather than single-frame estimates, and the challenge of capturing subtle motions, especially at articulated distal joints such as wrists and elbows (Zhang et al., 13 Jul 2025).

A plausible implication is that VST-Pose should be understood less as a generic transformer for pose estimation than as a domain-specific architecture for wireless sensing, where temporal continuity and body-structure priors must compensate for the absence of direct visual evidence.

2. Input representation, sensing setup, and supervision

The sensing setup uses two hosts with Intel 5300 NICs in a master-slave configuration, one as transmitter and one as receiver. Both operate at 2.4 GHz with three antennas each, forming a $3 \times 3$ MIMO system under IEEE 802.11n, and CSI is collected on 30 subcarriers. The CSI sampling rate is 150 Hz, while the synchronized video frame rate is 30 Hz. Although amplitude and phase are available, the method uses amplitude only because phase is described as heavily affected by noise (Zhang et al., 13 Jul 2025).

Raw CSI amplitude is denoised by a discrete wavelet transform. The paper states that five consecutive CSI samples are aggregated into one CSI frame. The initial tensor shape is

$3 \times 3 \times 30 \times 5,$

which is reshaped to

$3 \times 90 \times 5.$

A temporal sample is then represented as

$X \in \mathbb{R}^{T \times 3 \times 90 \times 5},$

where $T$ denotes the number of consecutive CSI frames in the temporal window (Zhang et al., 13 Jul 2025).

Ground truth is obtained through synchronized RGB video processed by OpenPose on the self-collected dataset. OpenPose yields 25 joints $(x,y,c)$, from which the framework retains the 17 COCO joints: nose, eyes, ears, neck, shoulders, elbows, wrists, hips, knees, and ankles. The paper describes the model and losses in terms of coordinate regression rather than confidence-aware heatmap estimation (Zhang et al., 13 Jul 2025).

For the self-collected dataset, action sequences are divided into clips containing 9 consecutive CSI frames, and the selected operating setting uses a sliding window of size 3 and stride 2. For MMFi, the sliding window size is 10 and the stride is 3 (Zhang et al., 13 Jul 2025).

3. Architecture and mathematical formulation

The encoder first converts CSI frames into keypoint-wise latent features. A three-layer CNN produces feature maps of shape

$T \times J \times H \times W,$

where $J$ is the number of human keypoints. Flattening over $H \times W$ and applying a fully connected layer yields

$X_c \in \mathbb{R}^{T \times J \times D}.$

The paper further introduces learnable temporal and spatial positional encodings,

$3 \times 3 \times 30 \times 5,$0

which are added to the encoded features before the attention backbone (Zhang et al., 13 Jul 2025).

ViSTA-Former, the backbone of VST-Pose, is a dual-stream spatiotemporal attention architecture inspired by DST-Former. Each DST-Block contains two branches with different operator orderings:

1. ST branch: spatial block followed by temporal block.
2. TS branch: temporal block followed by spatial block.

The two streams do not share weights. The spatial block applies multi-head self-attention over joint tokens at each time step. For a spatial feature tensor $3 \times 3 \times 30 \times 5,$1, the head projections are defined as

$3 \times 3 \times 30 \times 5,$2

and the spatial multi-head self-attention is written as

$3 \times 3 \times 30 \times 5,$3

$3 \times 3 \times 30 \times 5,$4

The temporal block first flattens joint-wise features at each time step into dimension $3 \times 3 \times 30 \times 5,$5, forming a sequence $3 \times 3 \times 30 \times 5,$6, and then applies temporal self-attention across frames (Zhang et al., 13 Jul 2025).

The output of DST-Block $3 \times 3 \times 30 \times 5,$7 is an adaptive weighted fusion of the two streams:

$3 \times 3 \times 30 \times 5,$8

with weights

$3 \times 3 \times 30 \times 5,$9

This formulation makes the backbone explicitly sensitive both to structural relationships among joints and to temporal evolution across CSI frames (Zhang et al., 13 Jul 2025).

4. Velocity integration and learning objective

A distinguishing element of VST-Pose is its velocity modeling branch. Rather than imposing only temporal smoothness, the framework introduces explicit supervision on short-term displacement. The local velocity feature in block $3 \times 90 \times 5.$0 is defined from the TS branch as

$3 \times 90 \times 5.$1

The paper states that velocity features from all blocks are aggregated with skip connections and then passed through a temporal block to produce a global velocity feature $3 \times 90 \times 5.$2. Although the exact aggregation operator is not given, the resulting feature is fused with the final backbone keypoint feature $3 \times 90 \times 5.$3 by

$3 \times 90 \times 5.$4

Each time step of $3 \times 90 \times 5.$5 is fed to an MLP to predict the keypoint sequence

$3 \times 90 \times 5.$6

for 2D estimation, while the final time step of $3 \times 90 \times 5.$7 is fed to another MLP to predict

$3 \times 90 \times 5.$8

For MMFi, the paper indicates the same framework is used for 3D pose estimation, though the decoder equations are not rewritten with explicit 3D dimensionality (Zhang et al., 13 Jul 2025).

Ground-truth velocity is defined as the displacement between the last and first keypoint frames in the temporal window:

$3 \times 90 \times 5.$9

The total loss is

$X \in \mathbb{R}^{T \times 3 \times 90 \times 5},$0

with

$X \in \mathbb{R}^{T \times 3 \times 90 \times 5},$1

The paper argues that this explicit velocity supervision improves sensitivity to subtle motion without the over-smoothing associated with simple temporal continuity regularization (Zhang et al., 13 Jul 2025).

A plausible implication is that the velocity branch functions as a motion-aware auxiliary pathway whose main benefit is not coarse tracking stability but finer localization under weak CSI perturbations.

5. Datasets, protocol, and empirical results

Self-collected smart home care dataset

The self-collected 2D dataset was built for smart home care scenarios. It includes 5 volunteers, with heights from 165 cm to 190 cm and ages 23–32, recorded in three distinct residential environments. The action set contains 15 common in-home actions, divided into daily actions and sleep-related actions: standing, sitting, walking, sitting down, standing up, stretching, bending, jumping, turning left, turning right, getting up, lying down, supine, left lateral, and right lateral (Zhang et al., 13 Jul 2025).

Each action segment has at least 12 repetitions per subject per action, is trimmed to 3 seconds, corresponds to 450 CSI samples at 150 Hz, and yields approximately 90 CSI frames. The final dataset contains 3,300 action segments and approximately 297,000 CSI frames. A frame-level data augmentation strategy divides each action sequence into 10 data clips, producing 33,200 CSI segments, which are split randomly into training and testing with ratio 4:1 (Zhang et al., 13 Jul 2025).

MMFi

For 3D evaluation, VST-Pose is tested on MMFi, which provides 3D joint annotations and includes 27 actions and 40 subjects. The paper follows official Setting 1 under Protocol 3, using a 3:1 random train/test split over action samples. On MMFi, the sliding window size is 10, stride is 3, the ViSTA-Former backbone is reduced to 1 DST-Block, training lasts 50 epochs, and StepLR is used (Zhang et al., 13 Jul 2025).

Main quantitative results

On the self-collected dataset, VST-Pose reports the headline result

$X \in \mathbb{R}^{T \times 3 \times 90 \times 5},$2

with average values

• PCK@50: 92.25
• PCK@40: 88.79
• PCK@30: 82.94
• PCK@20: 71.90
• PCK@10: 48.91

Selected per-joint PCK@50 values include 93.13 for nose, 93.25 and 93.41 for left and right shoulders, 95.35 and 95.64 for left and right hips, 87.03 and 87.25 for wrists, and 91.09 and 90.96 for elbows. The paper notes that torso and lower-limb joints are easier, whereas wrists and elbows are harder because of more complex articulated motion (Zhang et al., 13 Jul 2025).

Compared with prior methods on the same self-collected dataset, the paper reports:

Method	PCK@50	PCK@20	MPJPE	PA-MPJPE
MetaFi++	86.37	56.87	11.5248	6.4905
HPE-Li	85.60	51.70	11.9218	6.4454
DT-Pose	88.24	63.15	10.5659	5.7397
VST-Pose	92.25	71.90	8.0692	4.7016

On MMFi, the reported results are:

Method	PCK@50	PCK@20	MPJPE	PA-MPJPE
MetaFi++	85.93	47.71	187.9	106.5
HPE-Li	86.38	49.52	183.0	105.3
DT-Pose	86.29	49.53	183.3	105.6
VST-Pose	87.37	55.03	169.7	105.9

The self-collected-dataset abstract statement that VST-Pose outperforms existing methods by 8.3% in PCK@50 appears in the paper text, but the tabulated comparison to DT-Pose yields a difference of $X \in \mathbb{R}^{T \times 3 \times 90 \times 5},$3. This suggests that the larger figure refers to a different comparison basis or broader baseline context rather than to the strongest table entry directly (Zhang et al., 13 Jul 2025).

6. Ablations, interpretation, and limitations

The ablation studies isolate several design choices. Increasing the number of DST-Blocks improves performance up to 5 layers, which becomes the default on the self-collected dataset. Table V reports:

• 1 layer: PCK@50 90.08, PCK@20 63.29, MPJPE 9.60, PA-MPJPE 5.46
• 3 layers: PCK@50 91.87, PCK@20 70.17, MPJPE 8.44, PA-MPJPE 4.87
• 5 layers: PCK@50 92.25, PCK@20 71.90, MPJPE 8.06, PA-MPJPE 4.70
• 7 layers: PCK@50 92.07, PCK@20 72.05, MPJPE 8.18, PA-MPJPE 4.73

Velocity modeling is also directly validated. Without the velocity branch, the model achieves PCK@50 91.54, PCK@20 69.40, MPJPE 8.59, and PA-MPJPE 5.05; with it, performance improves to 92.25, 71.90, 8.06, and 4.70, respectively. The source of the velocity branch matters as well: taking it from the TS branch performs better than using the ST branch alone or combining TS and ST. Late fusion is beneficial; removing velocity fusion reduces performance from 92.25 to 91.68 in PCK@50 and from 71.90 to 69.71 in PCK@20 (Zhang et al., 13 Jul 2025).

These results support the paper’s central interpretation: structural and temporal dependencies are complementary, and explicit motion-displacement supervision is especially effective for fine-grained localization. The stronger gains at PCK@20 than at PCK@50 reinforce this reading, since they indicate improved precision rather than only coarse correctness (Zhang et al., 13 Jul 2025).

The paper explicitly identifies several limitations. First, dataset creation requires synchronized CSI and video, which increases collection complexity and cost. Second, the sensing configuration uses a single transmitter-receiver pair in one direction, which limits spatial resolution. Third, cross-domain modeling remains weak, indicating sensitivity to environmental shifts. The paper does not study multi-person estimation, and it acknowledges that complex fast motions and articulated distal joints remain difficult (Zhang et al., 13 Jul 2025).

This suggests that VST-Pose is best understood as a privacy-aware, continuous, single-person indoor pose estimation framework for WiFi sensing rather than a general replacement for visual pose estimation. Its significance lies in demonstrating that explicit spatiotemporal attention and displacement-aware supervision can materially improve wireless human pose estimation in smart-home environments (Zhang et al., 13 Jul 2025).