---
title: 'BodyWave: Egocentric mmWave Pose Estimation'
url: https://www.emergentmind.com/topics/bodywave
type: topic
---

# BodyWave: Egocentric mmWave Pose Estimation

BodyWave is an inside-out body-tracking (IOBT) system that mounts compact millimeter-wave (mmWave) radars on a mixed-reality (MR) headset and uses them, together with a lightweight deep model, to recover full-body 3D pose from the wearer’s own point of view. It was introduced as an egocentric body-tracking approach designed for applications such as gesture control and codec avatar in MR, including augmented reality (AR) and virtual reality (VR), while targeting the core failure modes of camera-based IOBT: limited field of view, self-occlusion, environmental sensitivity, and privacy concerns. The reported system places radars just 4–5 cm from the cheeks in a Meta Quest 3-like form factor, processes raw radar signals into high-resolution range profiles, and predicts 13 3D keypoints in a headset-centric coordinate system [2509.07995].

## 1. Problem setting and sensing rationale

Egocentric or inside-out body tracking estimates the wearer’s own body pose using sensors mounted on the head or glasses. In MR headsets, the sensors look outward from the head and must infer the full-body pose—including limbs and lower body—from this self-view. This is substantially harder than exocentric tracking because the headset viewpoint is sparse, strongly self-occluded, and frequently excludes lower-body parts from direct view.

BodyWave addresses a formulation stated as full-body, fine-grained 3D pose estimation from a headset under strong self-occlusion and limited field of view, with privacy, robustness, and compact hardware. The motivation is tied to several application domains. Codec avatars and telepresence require full-body motion to convey non-verbal cues. Gesture control in MR benefits from upper- and lower-body tracking rather than controller-centric interaction. Health, fitness, and rehabilitation scenarios require motion capture in everyday environments without multi-camera installations.

The system is explicitly positioned against camera-based IOBT. Even with fisheye lenses of roughly \(200^\circ\) horizontal FoV, head-mounted cameras primarily capture the upper body and hands, while the lower body is often out of frame. Self-occlusion is severe during motions such as kicking or squatting. Cameras also depend on line-of-sight and illumination, and they capture detailed visual appearance of users and environments.

mmWave radar is used because it offers non-line-of-sight sensing, low SWAP+C, robustness to lighting and several environmental factors, and enhanced privacy relative to cameras. The BodyWave prototype uses 60 GHz FMCW radars, with the paper arguing that this modality can detect reflections off multiple body surfaces and via multi-path, allowing partial sensing of otherwise occluded body parts [2509.07995].

## 2. Hardware configuration and signal representation

The prototype is modeled after the Meta Quest 3 form factor. It uses a Quest 3 strap, a facial interface, and a custom 3D-printed frame in HP MJF nylon. Two Infineon XENSIV BGT60ATR24C 60 GHz FMCW radars are mounted at the left bottom edge and right bottom edge of the headset, both downward facing and aligned roughly with the nose bridge. Each radar is a 22 mm \(\times\) 13 mm PCB, and the distance to the cheek is 4–5 cm.

The radar operating configuration is specified as follows: ADC sample rate 1 MHz, frame rate 30 Hz, 4 FMCW chirps per frame, 128 ADC samples per chirp, and a 58–63 GHz sweep corresponding to \(B = 5\text{ GHz}\). The chirp repetition time is \(5 \times 10^{-4}\) s. The IF band-pass filter is 80 kHz–500 kHz, the TX power is maximum SDR setting with unit 31, and the IF gain is 33 dB. The paper states that all 4 TX and 2 RX are enabled with bit masks 15 and 3, while also describing the chip as 2 TX / 4 RX with 8 virtual channels per sensor; this indicates that the effective representation is organized around 8 channels per sensor pipeline as reported in the system overview.

The FMCW transmit signal is written as

$$
s_{\text{TX}}(t) = A \sin\left(2\pi\left( f_0 t + \frac{f_1 - f_0}{2T} t^2 \right)\right),
$$

with \(f_0 = 58\text{ GHz}\), \(f_1 = 63\text{ GHz}\), and \(T\) the chirp duration. The range resolution is

$$
\Delta R = \frac{c}{2B},
$$

which yields \(3\text{ cm}\) for \(B = 5 \times 10^9 \text{ Hz}\). The maximum unambiguous range, given 128 samples, is reported as

$$
R_{\max} = \frac{N_{\text{samples}} \,\Delta R}{2} \approx 1.92 \text{ m},
$$

which aligns with the typical distance from headset to feet during motion [2509.07995].

Rather than constructing point clouds, BodyWave uses FFT-based range profiles. For IF samples \(x[n]\), the range FFT is

$$
X[k] = \sum_{n=0}^{N-1} x[n] \, e^{-j2\pi kn/N}, \quad k = 0, \dots, N-1.
$$

The magnitude \(|X[k]|\) yields amplitude versus range bin, and successive chirps over time form range-time spectrograms. The paper argues that this preserves dense multi-target reflections and temporal structure that would be discarded by thresholded point-cloud generation.

Two processing steps are central. First, successive subtraction suppresses static clutter:

$$
P'(t, r) = P(t, r) - P(t-1, r).
$$

Second, frame unwarping processes each chirp independently rather than aggregating chirps per frame, increasing temporal granularity within the 30 Hz frame envelope. The resulting network input for each time window is an \(8 \times 60 \times 65\) tensor, and the model uses 0.5-second windows of radar data [2509.07995].

## 3. Learning architecture and pose parameterization

BodyWave uses a multi-view latent-temporal model composed of a windowed CNN front end, channel fusion, an LSTM, and linear regression layers. Each of the 8 channels with a \(60 \times 65\) feature map is split along one dimension into 6 windows. Each window is passed through the same CNN block, with weights shared among windows and channels. The CNN block uses convolutional layers with Batch Normalization, ReLU activation, and max-pooling.

After per-window feature extraction, all windows from all channels are concatenated and fed to an LSTM. The role of the LSTM is to capture latent temporal dependencies over the 0.5-second window, improving continuity and robustness under ambiguous instantaneous observations. The output is passed through fully connected linear layers to regress 3D coordinates.

The prediction target is 13 keypoints \(\times\) 3 coordinates, giving a 39-dimensional output vector. The coordinates are defined in a headset-centric frame. Training supervision comes from an exocentric front-facing Logitech RGB camera at \(1920 \times 1080@30\text{ fps}\), with MediaPipe Body API extracting 3D keypoints; no camera is used at runtime.

The principal metric is Mean Per-Joint Position Error (MPJPE):

$$
\text{MPJPE} = \frac{1}{N J}
\sum_{n=1}^{N} \sum_{j=1}^{J}
\left\| \hat{\mathbf{p}}_{n,j} - \mathbf{p}_{n,j} \right\|_2,
$$

where \(J = 13\). The paper states that MediaPipe outputs are normalized by body height and are rescaled using the average participant height for fair comparison with other works. No explicit bone-length or kinematic loss is described; this suggests that plausible kinematics are learned implicitly from the radar–pose training pairs rather than imposed through explicit skeletal constraints [2509.07995].

## 4. Dataset, calibration regimes, and quantitative performance

The dataset comprises approximately 525,000 frames of radar plus image ground truth, corresponding to approximately 292 minutes at 30 fps, from 14 participants. Four participants are used for pre-training and 10 for the user study, with the 4 pre-training participants also performing the user-study protocol. The motion protocol includes 19 gestures spanning arm, leg, and head categories, with repeated left/right execution, fast/slow variants, and returns to standing pose between repetitions. Data were collected in 4 rooms labeled L, M, S, and XS.

BodyWave evaluates three deployment regimes: user-independent, user-adaptive, and user-dependent.

| Setting | Definition | MPJPE |
|---|---|---:|
| User-independent | Leave-one-user-out; no per-user calibration | 9.85 cm |
| User-adaptive | Fine-tuning with a brief calibration session | 4.94 cm |
| User-dependent | Same user in training and testing | 3.86 cm |

In the user-dependent setting, performance degrades to 7.21 cm when tested on a different day with headset remounting and outfit changes. The reported gap from 3.86 cm to 7.21 cm is used in the paper to highlight sensitivity to hardware positioning and clothing.

The user-adaptive mode corresponds to approximately 10 minutes of calibration in the study and is presented as a practical compromise between accuracy and deployability. The user-independent result represents the zero-calibration case on unseen users. The reported quantitative summary in the abstract is a mean per-joint position error of 9.85 cm on unseen users, 4.94 cm with a few minutes of user calibration, and 3.86 cm in a fully-adapted user-dependent setting [2509.07995].

Training uses batch size 512, 50 epochs, an initial learning rate of 0.01, and ReduceLROnPlateau with factor 0.5, patience 5 epochs, and minimum learning rate \(10^{-5}\). Training is performed on an RTX 4080 GPU.

## 5. Comparative evaluation and ablation results

The paper reports architecture ablations in the user-adaptive setting. The latent-temporal model achieves 4.94 cm MPJPE. A cross-view transformer achieves 7.45 cm, and a vanilla CNN without explicit temporal modeling achieves 14.72 cm. The result is used to support the claim that explicit temporal modeling and multi-view latent features are critical for radar-based egocentric pose estimation.

The comparison to existing systems is framed carefully. On Human3.6M, exocentric RGB 3D pose methods are reported at 17.59–29.1 mm MPJPE, which the paper treats as a lower bound for egocentric methods operating under substantially heavier occlusion. EgoBody3M, an RGB egocentric headset-based system, is reported at approximately 5.18 cm overall MPJPE, with 12.2 cm error for out-of-view wrists. mmEgo, a head-mounted mmWave plus IMU system with radars placed 20–40 cm from the face, is reported at 4.287 cm MPJPE for upper body and 5.460 cm for lower body, but on only 3 test users and with hardware described as heavy and non-MR-compatible.

Against these baselines, BodyWave’s user-adaptive 4.94 cm MPJPE for 13 keypoints on 14 users is presented as comparable to state-of-the-art camera-based egocentric systems and mmEgo, while using a headset-compatible 4 cm form factor and no IMU fusion. Gesture-wise error analysis reports the worst cases for arm_stretch and arm_gaming, and somewhat higher error for head gestures. At the same time, lower-body and occluded motions such as leg_squat and leg_back are described as reasonably accurate, which the paper attributes to radar’s occlusion robustness [2509.07995].

Leave-one-gesture-out experiments show minimal performance drop for most gestures, with leg_squat showing relatively higher error. User–room-independent tests across L, M, S, and XS rooms show only minor performance differences, and room adaptation helps less than user adaptation. The paper therefore distinguishes two kinds of variation: environmental differences, which appear relatively manageable, and person-specific or mounting-specific variation, which remains more consequential.

## 6. Practical properties, limitations, and prospective directions

BodyWave is described as privacy-preserving because mmWave returns are signal energy patterns rather than photorealistic images. The system is also presented as lighting-robust, since radar is unaffected by illumination, shadows, and camera-specific visibility failures. Static clutter is mitigated by successive subtraction, and the paper argues that multi-path and non-line-of-sight behavior help detect otherwise occluded limbs.

The hardware and runtime profile are also central to its practical characterization. The model is reported as only approximately 209.5 KB, with an inference footprint of approximately 5.2 MB and measured performance of approximately 325 FPS on mobile-class SoCs, leaving substantial headroom within 30 Hz tracking. This suggests that BodyWave is not only a sensing concept but also a deployment-oriented system architecture under MR SWAP+C constraints.

The paper also identifies several limitations. Remounting the headset or changing outfits increases error by roughly 3–4 cm in the user-dependent case. Large-amplitude motions such as deep squats and rapid motions such as arm_gaming remain more difficult. Because no explicit kinematic constraints are imposed, predicted skeletons may violate anatomical constraints in some cases. Severe occlusion by solid objects, such as legs tucked under a thick table, can still degrade radar sensing.

The applications described include gesture control, full-body avatars or codec avatars, remote collaboration, and healthcare or fitness monitoring. For downstream animation, the predicted keypoints can be mapped through a skeleton and inverse kinematics to SMPL-X or codec-avatar parameters. Future directions listed in the paper include end-to-end codec avatars that bypass keypoint intermediates, sensor fusion with cameras, robustness studies in extreme occlusion settings, radar integration into future AR glasses, and self-supervised or unsupervised radar learning [2509.07995].

A plausible implication is that BodyWave marks a shift in egocentric body tracking from a camera-dominant formulation toward multimodal MR sensing stacks in which mmWave is treated as a first-class modality rather than a fallback. The evidence reported in the paper supports that interpretation specifically in the regime of lower-body occlusion, privacy-sensitive deployment, and headset-compatible hardware geometry.

Source: https://www.emergentmind.com/topics/bodywave