CPEP: Contrastive Pose-EMG Pre-training

• The paper introduces a novel cross-modal pre-training strategy that aligns EMG signals with pose embeddings to enhance both linear probing and zero-shot gesture classification.
• The method employs Transformer-based unimodal encoders and MAE pre-training, using a fixed pose encoder as a high-SNR anchor for stable, structured alignment.
• Empirical results on the emg2pose dataset show significant improvements, with up to 72% accuracy gains on unseen gestures over traditional EMG-only approaches.

Searching arXiv for the specified papers and closely related work. Contrastive Pose-EMG Pre-training (CPEP) is a cross-modal alignment framework for learning surface electromyography (sEMG) representations that are explicitly informative of hand pose or skeleton, with the aim of improving gesture recognition and enabling zero-shot classification. The framework is introduced in "CPEP: Contrastive Pose-EMG Pre-training Enhances Gesture Generalization on EMG Signals" (Cui et al., 4 Sep 2025). Its central premise is that sEMG is a weak modality—highly variable, noisy, and low-level—whereas pose encodes structured kinematics in a compact, semantically meaningful manifold. By aligning EMG embeddings to a high-quality pose embedding space via a CLIP-style contrastive objective, CPEP induces pose-aware EMG features that generalize to unseen gestures and users.

1. Conceptual basis and problem setting

CPEP is situated in the problem of hand gesture classification from wearable biosignals. The motivating difficulty is that predicting hand gestures from sEMG is difficult due to low SNR, variability across users and sensor placements, and limited labeled corpora. In contrast, visual and pose modalities provide high-quality, structured signals with clear spatial semantics and large-scale datasets. Conventional self-supervised pretext tasks on EMG underperform, and supervised regression of pose from EMG, exemplified by emg2pose-style systems, struggles to generalize to unseen gestures and users (Cui et al., 4 Sep 2025).

The key idea is to pre-train unimodal encoders for EMG and pose, freeze the pose encoder, and then align EMG to pose through a symmetric InfoNCE loss. In this design, the pose encoder acts as a stable, high-SNR anchor. The EMG encoder is updated so that its embeddings are close to synchronized pose embeddings and far from mismatched pairs. The resulting representation is intended to inherit the structural semantics of pose, thereby supporting linear separability for in-distribution gestures and retrieval-based zero-shot classification of unseen gestures.

This setup distinguishes CPEP from direct EMG-to-pose regression. Rather than requiring the EMG branch to reconstruct or regress pose coordinates explicitly during the alignment stage, the method uses only MAE pre-training and symmetric contrastive alignment. No auxiliary reconstruction or regression is included during CPEP; the pose encoder remains fixed to provide stable targets. This suggests that the method treats pose primarily as a representation-space prior rather than as a supervised output target.

2. Architecture and pre-training design

CPEP uses Transformer-based unimodal encoders for both EMG and pose. Both branches employ a Transformer encoder backbone with a linear tokenizer over temporal patches, and a [CLS] token summarizes the sequence. For a sequence of length $T$, a temporal patch length $S$ yields $K = \lfloor T/S \rfloor$ non-overlapping tokens. At each patch, channels over time within the patch are flattened and projected with a linear tokenizer into $d$-dimensional embeddings. Each modality uses a 4-layer Transformer encoder and a 2-layer Transformer decoder with embedding dimension $d = 256$ during MAE pre-training (Cui et al., 4 Sep 2025).

The MAE objective reconstructs masked tokens from unmasked tokens. In the notation of the paper, with token embeddings $\mathbf{z}_i \in \mathbb{R}^d$, masked token set $\mathcal{M}$, and encoder-decoder pair $(\phi,\psi)$, the unimodal pre-training loss is

$$\mathcal{L}_{\mathrm{MAE}} = \frac{1}{|\mathcal{M}|} \sum_{i \in \mathcal{M}} \left\| \psi\!\bigl(\phi(\{\mathbf{z}_j\}_{j \notin \mathcal{M}})\bigr)_i - \mathbf{z}_i \right\|_2^2.$$

After MAE pre-training, CPEP appends a lightweight 1-layer projection head with hidden size 256 to the EMG encoder. This head maps the EMG [CLS] token into the pose embedding space. The pose encoder is frozen during contrastive alignment. For synchronized EMG-pose pairs $(x_i,p_i)$, the aligned representations are

$S$0

After $S$1 normalization and temperature scaling,

$S$2

The loss is a CLIP-style symmetric InfoNCE objective over a batch of $S$3 paired samples:

$S$4

During alignment, the pose encoder $S$5 is frozen, the EMG encoder $S$6 and projection head $S$7 are trained, and $S$8 is learnable and initialized to $S$9. The use of a fixed pose anchor is not incidental: the reported ablations state that training both pose and EMG encoders together fails to converge, underscoring the importance of a stable anchor in low-SNR cross-modal alignment (Cui et al., 4 Sep 2025).

3. Dataset, preprocessing, and training protocol

CPEP is evaluated on the public emg2pose dataset, described as containing 370 hours of synchronized sEMG and hand pose data from 193 users across 29 behavioral groups, with over 80 million pose labels (Cui et al., 4 Sep 2025). Each user performed four sessions per gesture category with different EMG-band placements; each session lasts 45–120 s and includes repeated gesture bouts or freeform movements. The training signal pipeline uses non-overlapping 2-second windows sampled at 2 kHz for both EMG and pose, so $K = \lfloor T/S \rfloor$0 time steps.

The EMG stream is instance-normalized, processed with a 2–250 Hz band-pass filter and a 60 Hz notch filter, and augmented with channel rotation following emg2pose. Pose consists of synchronized hand-skeleton joint angles, and the method uses the pose encoder as an anchor without additional pose-specific augmentation. For MAE pre-training, the token lengths are $K = \lfloor T/S \rfloor$1 and $K = \lfloor T/S \rfloor$2, with non-overlapping temporal tokens and masking ratio $K = \lfloor T/S \rfloor$3 for both modalities. Each EMG window $K = \lfloor T/S \rfloor$4 is paired with its synchronized pose sequence $K = \lfloor T/S \rfloor$5 from the same 2-second window; within-batch synchronized pairs define positives and non-matching pairs define negatives.

The training protocol is explicit. MAE pre-training for EMG and pose uses AdamW with learning rate $K = \lfloor T/S \rfloor$6, weight decay $K = \lfloor T/S \rfloor$7, cosine annealing with warm restarts, batch size 256, and 100 epochs per modality. Contrastive alignment uses the 1-layer MLP projection head on the EMG [CLS], $K = \lfloor T/S \rfloor$8 normalization for cosine similarity, learnable temperature initialized at $K = \lfloor T/S \rfloor$9, batch size 256, and 100 epochs. Negatives are in-batch only, with no memory queue, and the loss is bidirectional. All models are trained on 4× NVIDIA V100 GPUs, and end-to-end training per model is reported as approximately 4.5 hours.

Two practical clarifications follow directly from these details. First, CPEP is not a purely unpaired cross-modal method: it requires paired EMG-pose segments for contrastive pre-training. Second, its zero-shot behavior is not parameter-free at deployment time: pose embeddings can be precomputed for fast retrieval, but the quality and diversity of the pose corpus affect performance and scalability.

4. Evaluation protocols and empirical performance

CPEP is assessed using linear probing and zero-shot classification (Cui et al., 4 Sep 2025). In linear probing, the EMG encoder is frozen and a softmax linear classifier is trained on top of its embeddings using labeled EMG-only data; macro accuracy, defined as mean per-class accuracy, is reported on held-out sets. In zero-shot classification, inference is retrieval-based. Pose embeddings are precomputed over the relevant corpus, and for a test EMG sample $d$0 the method retrieves the top-$d$1 nearest poses by cosine similarity:

$d$2

Prediction is then obtained by majority vote over the labels of the retrieved poses:

$d$3

The split protocol pre-trains on the full public train split $d$4. For in-distribution linear probing, the linear head is trained on $d$5 and evaluated on $d$6. For unseen-gesture linear probing, the linear head is trained on $d$7 and evaluated on $d$8. Zero-shot evaluation is performed on $d$9, and all users in $d = 256$0 are unseen, explicitly testing user-level generalization.

The reported results state that CPEP surpasses self-supervised EMG-only MAE and supervised pose-regression baselines, including PoseT, emg2pose, Vemg2pose, and NeuroPose, in linear probing, and uniquely enables zero-shot gesture classification. Reported macro accuracy improvements reach up to 21% on in-distribution gesture classification and 72% on unseen gesture classification over emg2pose benchmark models, depending on the split and metric. The zero-shot setting is evaluated only for CPEP because the baselines do not produce aligned EMG-pose embeddings enabling retrieval.

The ablation results reported for CPEP are as follows:

Variant	LP in-distribution	ZS in-distribution	LP unseen	ZS unseen
CPEP	0.782	0.757	0.536	0.481
CPEP-AvgPool	0.761	0.711	0.518	0.454
CPEP-$d = 256$1 RandInit	0.748	0.701	0.479	0.454
CPEP-$d = 256$2 Frozen	0.372	0.344	0.326	0.298

The reported interpretation is that, in zero-shot evaluation, CPEP’s accuracy on unseen gestures approaches its own linear-probe performance and exceeds the linear-probe results of supervised EMG-to-pose baselines. This suggests that aligned representation geometry can substitute for task-specific classifier training when the target gestures are not seen during supervised fitting.

5. Ablation findings and design implications

The ablation studies identify several design choices as consequential (Cui et al., 4 Sep 2025). Pre-training the EMG encoder with MAE is described as crucial: random initialization degrades both linear probing and zero-shot performance and slows convergence. Freezing the EMG encoder and training only the projection head significantly hurts performance, indicating that encoder adaptation is necessary for successful alignment. Using the [CLS] token outperforms average pooling across tokens. Together, these results imply that the contrastive stage is not merely learning a shallow compatibility layer between pre-existing embeddings; it materially reshapes the EMG representation.

The behavior of the pose anchor is especially significant. Training both pose and EMG encoders together fails to converge, while freezing the pose branch yields stable optimization. This is presented as evidence that a high-SNR, pre-trained anchor modality is important when aligning a weak modality such as sEMG. A common misconception is that symmetric cross-modal training necessarily entails symmetric parameter updates; in CPEP, the loss is bidirectional but the optimization is intentionally asymmetric because only the EMG branch and its projection head are updated during alignment.

MAE hyperparameters also affect downstream performance. The reported pattern is that shorter pose patches and higher mask ratios improve downstream results, with $d = 256$3 and $d = 256$4 performing best among tested settings. Longer EMG patches consistently degrade performance, indicating the importance of fine temporal resolution for EMG tokenization. This suggests that temporal granularity is more fragile on the EMG side than on the pose side, plausibly because EMG carries informative dynamics in shorter-scale fluctuations.

The robustness analysis further notes that all evaluation users are unseen, demonstrating user-generalization. The dataset includes variable EMG-band placements across sessions, and the unseen gestures include occlusion-prone motions such as CountingWiggling and FingerPinches. The paper frames this as resilience to kinematic variability and visual occlusions, particularly relevant when extending the idea to vision-based pose anchors.

6. Interpretation, limitations, and relation to later cross-modal EMG work

The discussion in the original paper characterizes CPEP as a form of structured supervision for EMG representation learning (Cui et al., 4 Sep 2025). Pose or skeleton is treated as lying on a low-dimensional, semantically structured manifold of hand kinematics, rich in relational and geometric information that EMG alone does not easily expose. Aligning EMG to that manifold acts as a regularizer: it discourages spurious EMG-specific noise and encourages features that reflect joint-angle dynamics and gesture semantics. Because the symmetric InfoNCE objective uses in-batch positives and negatives, EMG samples are organized relative to pose similarity, which enhances linear separability and enables retrieval-based zero-shot inference.

Several limitations are stated explicitly. Despite alignment, EMG remains sensitive to electrode placement drift, skin impedance changes, and muscle fatigue. CPEP depends on high-quality synchronized pose labels during training, so noisy or poorly synchronized pose estimation can deteriorate alignment. Although alignment may reduce the amount of paired data needed compared with CLIP-scale training, the method still requires paired EMG-pose segments. Zero-shot inference depends on a pose exemplar corpus, and the coverage and diversity of that corpus affect both performance and scalability. These limitations clarify that zero-shot classification in CPEP refers to gesture labels at evaluation time, not to the absence of paired supervision during representation learning.

Potential extensions listed in the paper include application to other biosignals such as IMU or EEG by aligning them to pose or to other structured modalities, the introduction of temporal consistency objectives or cross-modal masked modeling, and lightweight deployment via distillation of the EMG encoder and projection head for wearable devices. Prototype-based zero-shot inference, using per-class pose prototypes rather than retrieval over a full corpus, is also proposed as a way to reduce retrieval cost and memory footprint.

A closely related development is "KinEMbed: Decoding Kinematics from Electromyography via Cross-Modal Contrastive Learning" (Gilardini et al., 6 Jul 2026), which explicitly positions itself as implementing a CPEP-style approach for continuous hand kinematic regression rather than discrete gesture classification. KinEMbed aligns EMG embeddings and kinematic embeddings on a shared unit hypersphere using cross-modal InfoNCE, then discards the projection head and trains a temporal decoder on frozen EMG embeddings. This later work suggests that the core CPEP principle—using a structured target modality to shape EMG representation geometry—extends beyond classification to continuous regression. At the same time, KinEMbed introduces issues that are more specific to regression, such as the false-negative problem in continuous target spaces, and reports that standard InfoNCE outperformed Soft-InfoNCE and VICReg under its evaluation regime (Gilardini et al., 6 Jul 2026).

Within the broader EMG literature represented by these two papers, CPEP’s contribution is therefore twofold. First, it defines a concrete pre-training strategy that aligns EMG with a frozen, high-quality pose encoder to produce pose-informed EMG embeddings. Second, it demonstrates that such embeddings support both improved linear probing and retrieval-based zero-shot gesture classification on the large-scale emg2pose dataset, with ablations validating the importance of MAE initialization, [CLS]-based summarization, a fixed pose anchor, and specific patch-length and mask-ratio choices.