---
title: Acoustic Multi-person Pose (AMP) Dataset
url: https://www.emergentmind.com/topics/acoustic-multi-person-pose-amp-dataset
type: topic
---

# Acoustic Multi-person Pose (AMP) Dataset

The **Acoustic Multi-person Pose (AMP) Dataset** is a synchronized active-acoustic dataset for estimating the three-dimensional poses of multiple people from acoustic signals alone. It was introduced in *“Sound-based Multi-Person 3D Pose Estimation”* as a benchmark containing approximately six hours of acoustic recordings and 432,000 synchronized pose frames from one-, two-, and three-person scenes [2609.04902]. AMP addresses the superposition of motion-dependent acoustic signatures, inter-person reflections, propagation delays, reverberation, and background noise that complicate multi-person acoustic pose estimation. It should be distinguished from the unnamed single-person acoustic pose-estimation dataset constructed in *“Acoustic-based 3D Human Pose Estimation Robust to Human Position”*, which used five subjects and five subject-position offsets but did not report multi-person recordings or annotations [2411.07165].

## 1. Scope and research context

AMP was constructed for the previously unexplored task of estimating the 3D poses of several people using active acoustic sensing. In contrast to optical systems, active acoustic sensing does not require cameras, subject-worn devices, or subject-generated speech. The dataset is intended to support acoustic pose estimation in conditions where RGB-based methods may be limited by darkness, visual occlusion, or privacy concerns.

The central difficulty is that acoustic observations generated by multiple moving bodies are superposed. A variation in the received signal cannot necessarily be attributed to a particular individual. Reflections between bodies introduce additional propagation paths and delays, obscuring the temporal relationship between motion and acoustic features. These effects differ from the single-person setting, where signal changes can more directly be associated with one subject.

The name AMP is associated with the 2026 multi-person dataset. The earlier 2024 paper used an acoustic pose-estimation dataset containing recordings from multiple subject positions, but it did not formally name that dataset AMP. Its acquisition protocol was strongly indicative of single-person recording: five male subjects stood at five offsets from a loudspeaker–microphone line, and the paper did not report simultaneous multi-person recordings, multi-person ground truth, or identity-separated evaluation [2411.07165].

AMP is therefore a multi-person benchmark, whereas the earlier dataset is a position-robust single-person benchmark. Both use active acoustic sensing, repeated Time-Stretched Pulse (TSP) signals, a pair of Edifier ED-S880DB loudspeakers, a Zoom H3-VR ambisonics microphone, and motion-capture ground truth, but they serve different experimental purposes.

## 2. Dataset composition and acquisition

AMP contains approximately six hours of recordings and approximately 432,000 synchronized pose frames. Ground-truth pose data were recorded at 20 frames per second. The aggregate frame count is consistent with six hours at 20 frames per second:

$$
6 \times 60 \times 60 \times 20 = 432{,}000.
$$

The dataset contains 15 subjects: 12 male and three female participants, with reported heights ranging from 150 cm to 181 cm. The participants were divided into three groups, with five subjects per group implied by the group-disjoint evaluation protocol. Each group contributes 72 minutes of single-person data, 24 minutes of double-person data, and 24 minutes of triple-person data, for 120 minutes per group and 360 minutes in total across the three groups.

| Property | AMP dataset |
|---|---|
| Total duration | Approximately 6 hours |
| Synchronized pose frames | Approximately 432,000 |
| Subjects | 15: 12 male, 3 female |
| Scene cardinalities | Single-, double-, and triple-person |
| Pose frame rate | 20 frames per second |
| Maximum number of people per scene | Three |
| Skeleton | 21 joints |

The acoustic acquisition system consists of a pair of Edifier ED-S880DB loudspeakers and a Zoom H3-VR ambisonics microphone. The loudspeakers emit a periodic TSP, whose frequency changes within each cycle. The microphone records the resulting acoustic response after propagation, reflection, diffraction, and interaction with moving bodies.

The Zoom H3-VR records four ambisonics channels conventionally denoted $W$, $X$, $Y$, and $Z$. The $W$ channel is described as omnidirectional, while $X$, $Y$, and $Z$ are directional components. The raw acoustic representation therefore has four channels, $C=4$, intended to retain three-dimensional spatial information.

The recordings were made indoors in the presence of background noise and reverberation. The dataset is not described as an anechoic or noiseless benchmark. An additional experiment introduced multiple black partitions into the room to alter acoustic reflection characteristics. The supplied paper text does not establish whether those partition recordings are part of the released AMP training and test data, an additional evaluation subset, or a separate experiment.

Detailed acquisition information remains unspecified. The paper does not report the loudspeaker and microphone coordinates, loudspeaker spacing, sensor heights and orientations, subject-to-sensor distances, TSP frequency range or duration, recorder sampling rate, bit depth, gain, synchronization clock, room dimensions, reverberation time, or acoustic treatment.

## 3. Participants, activities, and annotations

Participants performed walking, twisting, and raising both hands. Motions occurred in random order and at random speeds. Multi-person recordings include single-person, double-person, and triple-person scenes, with subject pairings and positions randomized within each group.

The dataset contains concurrent multi-person motion. Qualitative examples include two people twisting and one person raising both arms while another person is walking. These examples represent the attribution problem created when different motions produce overlapping acoustic changes. The paper does not state whether participants touched, passed in front of one another, or performed explicitly coordinated actions.

Ground-truth 3D poses were recorded with a Motive motion-capture system from OptiTrack equipped with 16 cameras. The pose recordings are synchronized with the acoustic data and sampled at 20 frames per second. The skeleton contains 21 joints, including the head, neck, shoulders, arms, forearms, hands, waist, thighs, shins, feet, toes, hip, and spine.

A pose for subject $j$ is represented conceptually as a sequence

$$
\mathbf{p}_j=\{p_{j,t}\}_{t=1}^{T},
$$

where $t$ indexes pose frames and $p_{j,t}$ denotes the subject’s 3D pose at frame $t$. The paper does not formally specify the stored tensor shape, joint-index ordering, parent–child kinematic tree, left–right naming convention, marker layout, or the precise mapping between the verbal body-part list and the 21 tensor entries.

The dataset’s coordinate conventions are also not documented in detail. It is unspecified whether the stored coordinates are global OptiTrack coordinates or root-relative coordinates, what origin and axes are used, whether an acoustic-to-motion-capture rigid transform is applied, and whether poses are normalized for translation, rotation, or scale. Tracking accuracy, marker-occlusion rates, dropped-frame rates, and motion-capture reconstruction error are not reported.

Participant demographics beyond sex and height are not provided. Ages, ethnicities, body masses, clothing, footwear, handedness, health status, and identity names or identifiers are unspecified. The paper also does not state whether every participant appears in all three scene types.

## 4. Acoustic representation and temporal organization

The acoustic signal is segmented into intervals associated with pose frames. Let

$$
\mathbf{s}=\{s_t\}_{t=1}^{T\times L},
$$

where $T$ is the number of pose frames and $L$ is the number of acoustic samples associated with one pose-frame interval. For channel $c$, the corresponding segment is transformed by STFT and Mel filtering:

$$
a_{t,c} =
\log\left(
H_{\mathrm{mel}}\cdot
\mathcal{F}
\left(
\{s_{t',c}\}_{t'=(t-1)L+1}^{tL}
\right)
\right).
$$

Here, $H_{\mathrm{mel}}$ is the Mel-filter-bank matrix, $\mathcal{F}$ denotes the Fourier or STFT operation, and $c\in\{W,X,Y,Z\}$.

The resulting acoustic representation is

$$
\mathbf{a}=\{a_t\}_{t=1}^{T},
\qquad
a_t\in\mathbb{R}^{(C,B)},
$$

where $C=4$ and $B$ is the number of Mel filter banks. The implementation uses a base value of $B=128$.

SoundMHPE, the model evaluated on AMP, uses three temporal-frequency resolutions:

$$
a^{\mathrm{high\text{-}time}}
\in
\mathbb{R}^{(N_{\mathrm{in}},C,B)},
$$

$$
a^{\mathrm{mid}}
\in
\mathbb{R}^{(N_{\mathrm{in}}/2,C,2B)},
$$

and

$$
a^{\mathrm{high\text{-}freq}}
\in
\mathbb{R}^{(N_{\mathrm{in}}/4,C,4B)}.
$$

These representations use temporal windows $L$, $2L$, and $4L$, paired with $B$, $2B$, and $4B$ Mel filters. The shorter window preserves temporal detail, while longer windows provide finer frequency resolution. With $N_{\mathrm{in}}=24$, the nominal temporal dimensions are 24, 12, and six units, respectively, assuming no additional padding or overlap.

The model uses $N_{\mathrm{out}}=8$ output pose frames and $N_{\mathrm{prev}}=16$ preceding pose-frame intervals, giving

$$
N_{\mathrm{in}}=N_{\mathrm{out}}+N_{\mathrm{prev}}=24.
$$

These settings describe model input windows rather than native recording sequence lengths. The number of stored sequences, extracted training windows, audio samples, and frames after trimming or windowing is not reported.

STFT hop sizes, window functions, padding, magnitude or power conventions, logarithmic stabilization, frequency limits, and normalization procedures are unspecified. The paper also does not state whether AMP is distributed as raw waveforms, processed spectrograms, or both. File formats, directory structure, timestamp formats, calibration files, preprocessing scripts, and split files are not documented in the supplied text.

## 5. Evaluation protocol and benchmark tasks

The principal evaluation uses a cross-subject, group-disjoint protocol. In each fold, two groups containing 10 subjects are used for training, while the remaining group containing five unseen subjects is used for testing. The held-out group is rotated, producing three-fold cross-validation, and results are averaged across the three folds.

This protocol is subject-disjoint at the group level. However, the paper does not provide explicit training, validation, and test sequence counts, a conventional fixed validation set, the number of windows per fold, or a public split file. It also does not establish whether the partitions are scene-disjoint, whether randomized positions and pairings are disjoint across training and testing, or whether test subjects participate in calibration recordings.

SoundMHPE is configured for up to $M=15$ possible identities or queries, although AMP scenes contain at most three people. The Temporal Pose Decoder therefore produces

$$
M(N_{\mathrm{out}}+1)=15(8+1)=135
$$

queries. The additional query per person is used for an instance-level confidence score. This model capacity should not be interpreted as evidence that AMP contains 15-person scenes.

AMP supports several tasks:

- multi-person 3D pose estimation from active acoustic signals;
- disentanglement of overlapping acoustic signatures;
- simultaneous estimation of individual poses;
- modeling temporal motion and inter-person acoustic dependencies;
- evaluation under reverberation and background noise;
- comparison of single-, double-, and triple-person scenes;
- evaluation of modified reflection conditions using black partitions.

The dataset does not establish systematic variation in subject-to-sensor distance, orientation, azimuth, elevation, object occlusion, obstacle transmission, outdoor conditions, through-wall sensing, large-crowd activity, or unconstrained real-world deployment.

The reported metrics are MPJPE, PA-MPJPE, and PCKh@0.5. MPJPE is the mean Euclidean distance between predicted and ground-truth joint positions, reported in millimetres. PA-MPJPE applies Procrustes alignment before calculating MPJPE; the alignment removes global translation, rotation, and scale. PCKh@0.5 measures the fraction of joints whose Euclidean error is below one-half of the head–neck distance.

## 6. Models and quantitative results

SoundMHPE is an encoder–decoder framework with two principal components. The **Acoustic Multi-scale Encoder (AME)** extracts temporal and fine-grained frequency features from the multi-resolution log-Mel representations. The **Temporal Pose Decoder (TPD)** uses attention to disentangle multi-person information across successive frames and reconstruct frame-wise individual poses.

The AME ablation uses only the high-temporal-resolution spectrogram. The TPD ablation uses one query per person to predict the multi-frame sequence. Attention ablations compare standard attention with temporal self-attention and frequency self-attention in the encoder, together with multi-scale attention and instance self-attention in the decoder.

The principal comparison uses two adapted baselines. The first adapts Shibata et al.’s single-person active-acoustic estimator based on TSP signals by retaining its feature extractor and adding a multi-person regression head. The second adapts Person-in-WiFi 3D by replacing its input stem to accept acoustic log-Mel spectrograms while retaining its transformer architecture.

| Method | MPJPE (mm) | PA-MPJPE (mm) | PCKh@0.5 |
|---|---:|---:|---:|
| Adapted Shibata et al. | 121.7 | 71.5 | 0.36 |
| Adapted Yan et al. | 119.9 | 69.7 | 0.36 |
| SoundMHPE | **106.5** | **65.0** | **0.43** |

Performance degrades from single-person to triple-person scenes, but SoundMHPE remains better than both adapted baselines.

| Method | Single MPJPE | Single PA-MPJPE | Single PCKh | Triple MPJPE | Triple PA-MPJPE | Triple PCKh |
|---|---:|---:|---:|---:|---:|---:|
| Adapted Shibata et al. | 111.3 | 65.8 | 0.39 | 124.5 | 73.3 | 0.39 |
| Adapted Yan et al. | 108.7 | 64.1 | 0.40 | 122.4 | 71.2 | 0.38 |
| SoundMHPE | **95.0** | **58.9** | **0.47** | **111.2** | **68.1** | **0.44** |

The ablation results associate the multi-scale encoder and temporal pose decoder with improved benchmark performance.

| Model | MPJPE (mm) | PA-MPJPE (mm) | PCKh@0.5 |
|---|---:|---:|---:|
| SoundMHPE without AME | 115.2 | 67.5 | 0.38 |
| SoundMHPE without TPD | 116.5 | 69.0 | 0.38 |
| Full SoundMHPE | **106.5** | **65.0** | **0.43** |

The reported STFT-window ablation obtains its best result with the $L,2L,4L$ configuration.

| STFT windows | MPJPE | PA-MPJPE | PCKh |
|---|---:|---:|---:|
| $L/4,L/2,L$ | 118.5 | 69.6 | 0.37 |
| $L/2,L,2L$ | 117.0 | 68.9 | 0.37 |
| $L,2L$ | 114.5 | 67.9 | 0.38 |
| $L,2L,4L$ | **106.5** | **65.0** | **0.43** |

## 7. Limitations, access, and interpretation

AMP provides a controlled benchmark for studying overlapping acoustic signatures, inter-person reflections, reverberation, temporal ambiguity, and multi-person pose disentanglement. Its synchronized acoustic and motion-capture recordings enable evaluation of both individual pose reconstruction and the degradation caused by increasing the number of people in a scene.

Several limitations constrain interpretation. The dataset contains only 15 subjects and at most three people per scene. Recordings use a single indoor room and a single sensor arrangement, with limited explicitly listed motion types. Detailed acoustic geometry, calibration information, sampling parameters, synchronization procedures, tracking-error analyses, file schemas, and annotation formats are not reported. The evidence for generalization beyond the recording room is limited to an additional modified-reflection experiment using black partitions.

The paper provides the project page `https://oumi03.github.io/sound-mhpe/`, but the supplied text does not state a dataset download URL, license, terms of use, archival repository, DOI, participant-consent information, or whether raw acoustic recordings and motion-capture annotations are publicly released. The project page is therefore the only stated access point.

AMP should be interpreted as a controlled, synchronized research benchmark for acoustic multi-person 3D pose estimation rather than as a comprehensive corpus of unconstrained human activity. It supports investigation of multi-person acoustic disentanglement under indoor reverberation and background noise, but it does not by itself establish robustness to arbitrary environments, large crowds, unseen sensor geometries, outdoor settings, through-wall conditions, or broad real-world deployment.

Source: https://www.emergentmind.com/topics/acoustic-multi-person-pose-amp-dataset