Decoding High-Dimensional Finger Motion from EMG Using Riemannian Features and RNNs

Abstract: Continuous estimation of high-dimensional finger kinematics from forearm surface electromyography (EMG) could enable natural control for hand prostheses, AR/XR interfaces, and teleoperation. However, the complexity of human hand gestures and the entanglement of forearm muscles make accurate recognition intrinsically challenging. Existing approaches typically reduce task complexity by relying on classification-based machine learning, limiting the controllable degrees of freedom and compromising on natural interaction. We present an end-to-end framework for continuous EMG-to-kinematics regression using only consumer-grade hardware. The framework combines an 8-channel EMG armband, a single webcam, and an automatic synchronization procedure, enabling the collection of the EMG Finger-Kinematics dataset (EMG-FK), a 10-h dataset of synchronized EMG and 15 finger joint angles from 20 participants performing rich, unconstrained right-hand motions. We also introduce the Temporal Riemannian Regressor (TRR), a lightweight GRU-based model that uses sequences of multi-band Riemannian covariance features to decode finger motion. Across EMG-FK and the public emg2pose benchmark, TRR outperforms state-of-the-art methods in both intra- and cross-subject evaluation. On EMG-FK, it reaches an average absolute error of $9.79 °\pm 1.48$ in intra-subject and $16.71 °\pm 3.97$ in cross-subject. Finally, we demonstrate real-time deployment on a Raspberry Pi 5 and intuitive control of a robotic hand; TRR runs at nearly 10 predictions/s and is roughly an order of magnitude faster than state-of-the-art approaches. Together, these contributions lower the barrier to reproducible, real-time EMG-based decoding of high-dimensional finger motion, and pave the way toward more natural and intuitive control of embedded EMG-based systems.

• The paper introduces TRR, a lightweight GRU model that combines multi-band covariance-based Riemannian features with temporal context for continuous 15-joint finger-angle regression from sEMG.
• The paper reports 9.79° intra-subject error on EMG-FK and 16.71° cross-subject error, outperforming or matching strong baselines in low-data single-session tests and achieving 0.73 NMSE on emg2pose.
• The paper provides a 10-hour unconstrained EMG-FK dataset and shows that simplified TRR runs at nearly 10 predictions per second on a Raspberry Pi 5, supporting real-time prosthetic, robotic-hand, and XR applications.

This paper presents an end-to-end framework for continuous regression of high-dimensional finger kinematics from surface electromyography (sEMG), combining a low-cost data acquisition pipeline, a new public dataset (EMG-FK), and a lightweight recurrent model, the Temporal Riemannian Regressor (TRR), that outperforms state-of-the-art baselines on both the new dataset and the emg2pose benchmark while running in real time on embedded hardware (2604.22499).

Motivation and positioning

Continuous EMG-to-kinematics regression is motivated by applications in prosthetic control, AR/XR interfaces, teleoperation, and rehabilitation. The authors argue that most prior work either reduces the task to classification over discrete postures—restricting controllable degrees of freedom (DOF) and producing unnatural control that myoelectric prosthesis users have cited as a reason for device abandonment—or constrains data collection to guided postures and imposed trajectories, which yields overly optimistic performance estimates. Existing regression approaches also rely on a narrow set of feature representations: time-domain features (TDF) or CNN-based representations of raw EMG.

The paper's methodological contribution is to bring covariance-based Riemannian features into this regression setting. Covariance matrices projected from the Riemannian manifold to the Euclidean tangent space (CMTS) have proven effective for EEG classification and for finger decoding from ECoG, and Jaquier and Calinon demonstrated geometry-aware covariance modeling for wrist kinematics; the authors claim TRR is the first approach integrating multi-band CMTS features with a recurrent network for finger joint-angle regression from sEMG.

Data acquisition framework and the EMG-FK dataset

The acquisition setup deliberately uses consumer-grade hardware: an 8-channel MindRove dry-electrode armband sampling at 500 Hz, and a single laptop webcam running MediaPipe to estimate 15 finger joint angles, resampled to 500 Hz. Because no dedicated synchronization hardware is used, the authors introduce an automatic synchronization procedure that finds the temporal shift maximizing correlation between peak joint velocity and a Hilbert-transform-derived "move command" amplitude from the EMG. An appendix validates this procedure by showing that artificially introduced offsets degrade performance as expected, though the method does not model the physiological electromechanical delay; empirically, introducing such delay yielded minimal gains, so zero-latency prediction was preferred.

Twenty participants performed 30 minutes each of free, unconstrained right-hand gestures, yielding the 10-hour EMG-FK dataset. The protocol incorporates an active-learning element: after 15 minutes of visually guided free motion, an interim model drives a virtual hand, encouraging participants to explore configurations the model predicts poorly. PCA analysis shows that 90% of kinematic variance requires 7 components for EMG-FK versus 12 for emg2pose (9 when restricted to shared joints), indicating slightly lower but comparable gesture diversity and confirming that both datasets genuinely require regression rather than classification. A limitation inherent to the sensor is that its single-reference design induces strong cross-channel contamination; common average referencing improves visualization but is not used in the learning pipelines.

The TRR model

TRR feeds sequences of CMTS feature vectors into a shallow GRU-based network: a dense layer of 256 units, two GRU layers (256 and 128 units, tanh activations, 10% dropout), and a final dense layer of 64 units, trained with Adam and Huber loss. For each sample, ten overlapping 300 ms windows (100 ms step) are processed, with covariances computed in three frequency bands (5–40, 40–80, 80–150 Hz); a simplified variant uses a single full band (5–150 Hz). The design reflects two premises: CMTS features reduce the need for deep representation learning, and the context dependence of EMG (activity reflecting movement dynamics rather than absolute posture) motivates recurrence.

The ablation study supports each design choice. Multi-band CMTS significantly outperforms TDF and convolutional features extracted from raw EMG or envelopes; the recurrent architecture outperforms MLPs, gradient boosting with regressor chains, and ridge regression on identical inputs; and performance converges at sequences of ten windows, with both shorter sequences and single longer windows degrading accuracy. Notably, an appendix shows intra-subject error only converges when nearly all available training data are used, tempering any expectation of strong performance from small calibration sets even within a subject.

Benchmark results

Evaluation covers intra-subject single-session (10-fold blocked cross-validation per subject) and cross-subject single-session (leave-one-subject-out) settings, using average normalized mean square error (NMSE) across joints, on both EMG-FK and a subsampled emg2pose benchmark (30 subjects, one session each). Baselines include vemg2pose (Meta's CNN-LSTM state-of-the-art), CRNNs on raw EMG and envelopes, MLPs on TDF, and MLPs on CMTS.

On EMG-FK, TRR achieves an intra-subject NMSE of $0.43 \pm 0.08$, corresponding to an average absolute error of $9.79\degree \pm 1.48$, versus $11.04\degree \pm 1.51$ for vemg2pose; in cross-subject evaluation TRR reaches $16.71\degree \pm 3.97$ versus $16.76\degree \pm 3.31$ for vemg2pose, where rankings vary across subjects and differences are less pronounced. On emg2pose, TRR attains $0.73 \pm 0.10$ intra-subject NMSE and again ranks best. A notable and somewhat counterintuitive result is that vemg2pose performs worst among all methods on its own benchmark under these evaluation conditions; the authors attribute this to the mismatch between their light training regimes (under 30 minutes intra-subject, roughly 15 hours cross-subject) and the 300+ hour training corpus used in the original work, conceding that vemg2pose would likely prevail with very large training quantities. This is an important boundary condition on the headline claim: TRR's superiority holds in low-data, single-session regimes, not necessarily in large-scale multi-session training.

Per-joint analysis shows thumb joints are decoded least accurately, attributed to the smaller range of motion captured by the vision system and the anatomical complexity of thumb movement. Qualitative comparisons indicate cross-subject predictions preserve overall gesture shape despite higher error, suggesting TRR captures some subject-invariant structure.

Embedded deployment

TRR's practical advantage is computational. On a Raspberry Pi 5, the simplified TRR runs at nearly 10 predictions/s (about 1.5 ms feature extraction plus 107 ms inference per sample), whereas vemg2pose requires $919 \pm 42$ ms per sample—an order-of-magnitude difference—and drives the CPU to its throttling temperature in under 10 minutes, which TRR avoids. A proof-of-concept demonstration shows real-time intuitive control of a multi-DOF robotic hand by one subject, though the authors correctly note this is not a statistical evaluation.

Limitations and open questions

The paper is explicit about several constraints. First, the acquisition protocol deliberately excludes grip-force generation, object manipulation, and concurrent arm or body movements, so generalization to real-world tasks with variable external loads and motion artifacts remains unvalidated—a significant gap for prosthesis control specifically. Second, quantization was not explored and could further improve edge efficiency or enable deeper models on constrained hardware. Third, all results derive from short single-session recordings; sEMG distributions drift over time due to electrode displacement, impedance variation, and fatigue, implying that sustained performance will require periodic recalibration or adaptive strategies such as domain adaptation or continuous learning, none of which are addressed here. Finally, the cross-subject gap ($9.79\degree$ to $16.71\degree$ on EMG-FK) remains substantial, and reducing it via domain adaptation is identified as an open direction, along with studying the model under the noise and context effects of genuine robotic hand control.

Conclusion

This work delivers a reproducible, low-cost pipeline for synchronized sEMG–kinematics acquisition, a public 10-hour dataset of unconstrained multi-DOF finger motion, and a Riemannian-feature recurrent regressor that sets the best reported results in intra- and cross-subject joint-angle estimation on both EMG-FK and emg2pose under matched low-data conditions, while being fast enough for thermal-constrained embedded inference. Its main claims are strongest for single-session, moderate-data regimes; extending robustness across sessions, subjects, and ecologically valid manipulation tasks remains open.