---
title: 'MS-MANO: Musculoskeletal Hand Model'
url: https://www.emergentmind.com/topics/ms-mano
type: topic
---

# MS-MANO: Musculoskeletal Hand Model

Searching arXiv for the primary MS-MANO paper and closely related hand-model references.
MS-MANO, short for **Musculoskeletal MANO**, is a musculoskeletal extension of the MANO parametric hand model for visual hand dynamics analysis and tracking. It is designed to address the unnatural, robot-like artifacts that arise when hand motion is modeled as direct joint actuation by instead introducing explicit muscle- and tendon-driven actuation, physiologically constrained torque trajectories, and simulation-in-the-loop learning. The framework has two central components: the **MS-MANO musculoskeletal hand model** and **BioPR**, a biomechanical pose refinement pipeline that can refine the output of any MANO-based estimator [2404.10227].

## 1. Definition and motivation

MS-MANO is motivated by a specific deficiency in conventional hand tracking systems: joint-actuated formulations generate torques directly at joints without modeling how human hand motion is actually produced by muscles and tendons. In the formulation presented in "MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints" [2404.10227], this simplification is identified as a source of unnatural or infeasible motions, especially in visually ambiguous settings such as occlusion or motion blur.

The proposed remedy is not to replace MANO, but to augment it. MANO retains its role as a learnable parametric hand representation with pose and shape adaptivity, while musculoskeletal structure is added to impose physiological realism. The paper identifies three principal contributions: muscle-/tendon-driven dynamics over **31 hand muscles** plus **8 wrist muscles**, physiologically constrained torque trajectories via a **Hill-type muscle model and routing**, and **BioPR**, an MLP-based refinement framework that uses musculoskeletal simulation rather than only image-based temporal features [2404.10227].

This architecture places MS-MANO in a particular position within hand modeling research. It remains compatible with standard MANO-based prediction pipelines, yet seeks to constrain those pipelines with biomechanical actuation. A plausible implication is that the framework is intended as an intermediate layer between purely image-driven hand pose estimation and more anatomically grounded simulation systems.

## 2. Musculoskeletal augmentation of MANO

The core technical move in MS-MANO is the integration of MANO with muscle-tendon routing, anatomical origin and insertion points, and muscle-driven torque generation. Muscles are modeled as **Hill-type actuators** with a **contractile element (CE)**, **parallel elastic element (PEE)**, and **serial elastic element (SEE)**. The anatomical routing is transferred from **MyoSuite’s MyoHand**, which itself is built atop OpenSim’s **MoBL** and **2nd-Hand anatomical models** [2404.10227].

Each muscle segment originates from an origin point $n_{\text{origin}}$ and ends at an insertion point $n_{\text{insertion}}$. Torque production from a muscle segment at a joint is defined as

$$
\bm\tau_\text{m} = f(F, x)\left\lVert(\bm q - \bm j)\times \frac{\bm s_c}{\lVert\bm s_c\rVert}\right\rVert,
$$

where $F$ is the contractile force, $x$ is the muscle state, $\bm q$ is the attachment point on bone, $\bm j$ is the joint location, and $\bm s_c$ is the muscle segment direction [2404.10227]. The paper treats $f(F,x)$ as an abstract mapping from muscle force and state to torque magnitude and does not provide an explicit analytic form for Hill force-length or force-velocity behavior.

A central representational problem is that MyoHand routing is **bone-centric**, whereas MANO is **joint-centric** and shape-adaptive. MS-MANO therefore introduces a joint-centric remapping. With MANO joints $\mathcal{M} = \{m_i\}_{i=1}^n$ and MyoHand bone subgroups $\mathcal{O} = \{O_i=\{b_{i,j}\}_{j=1}^k\}_{i=1}^n$, the mapping

$$
f_\text{mapping}: O_i \mapsto m_j
$$

associates each bone subgroup with a MANO joint, and muscle attachment displacements are remapped accordingly so that muscle routing follows changes in MANO joint locations under shape variation [2404.10227]. This remapping is essential to preserving MANO’s adaptivity while making muscle geometry anatomically meaningful.

The paper also notes an anatomical mismatch between MyoHand and MANO because they originate from different human sources. For that reason, **minor manual revisions to insertion points**, performed with anatomical expert input, are required to avoid tendon-skin intersections and maintain plausibility [2404.10227]. This is not a peripheral detail: the paper later identifies insertion-point sensitivity as a major determinant of trajectory quality and learning stability.

## 3. Dynamics, torque constraints, and inverse dynamics learning

The forward dynamics of MS-MANO are implemented in a physics engine using **RFUniverse + Unity/Kinesis**, with muscle excitation signals driving the articulated system [2404.10227]. The emphasis is not on a closed-form multibody derivation. The paper explicitly does **not** provide a full rigid-body dynamics equation, explicit activation dynamics, tendon compliance equations, pennation parameterizations, or maximum voluntary torque curves.

Instead, the formalism centers on inverse and forward mappings between kinematic states and muscle excitation. Given a pose sequence $\bm p_i \in \mathbb{R}^{n_{\text{joint}}\times 3}$ in axis-angle form and angular velocity $\bm v_i \in \mathbb{R}^{n_{\text{joint}}\times 3}$, inverse dynamics is defined as

$$
f_\text{inv}(\bm p_{i}, \bm v_{i}, \bm p_{i+1}, \bm v_{i+1}) = \bm a_i,
$$

and forward dynamics as

$$
f_\text{fwd}(\bm p_i, \bm v_i, \bm a_i) = (\bm p_{i+1}, \bm v_{i+1}).
$$

Here, $\bm a_i$ denotes the muscle excitation inferred for the transition from frame $i$ to frame $i+1$ [2404.10227].

The inverse mapping is learned by **IDNet**, which predicts muscle excitation despite the absence of ground-truth excitation labels. Supervision is therefore indirect. During training, IDNet is guided by torque matching against a PD controller with inverse dynamics compensation:

$$
\bm\tau_\text{pd} = k_p ({\bm p_d - \bm p})+ k_d ( \dot{\bm p}_d - \dot{\bm p} ),
$$

with reward

$$
r = \exp\left( \omega_{\tau} \cdot \lVert \bm\tau_\text{pd} - \bm\tau_\text{m} \rVert\right).
$$

The PD controller is used only during training; at test time, motion is driven by muscle actuators alone [2404.10227].

This training strategy makes the physiological constraint operational. The simulator enforces anatomical joint ranges and collision behavior through anatomically aligned colliders with small skin clearance, but the paper does not formalize these as explicit inequality constraints. The practical effect is that torque trajectories are shaped by musculotendon routing and simulation constraints rather than by post hoc kinematic regularization alone.

## 4. BioPR: simulation-in-the-loop pose refinement

BioPR is the second major component of the framework. Its role is to refine an initial pose estimate from any MANO-based estimator by using the musculoskeletal simulator as a source of biomechanically plausible reference motion [2404.10227].

The pipeline proceeds in three stages. First, a base MANO estimator outputs per-frame predicted pose $\bm p_i^\text{pred}$ and shape parameters; angular velocities $\bm v_i^\text{pred}$ are then computed by finite differences. For refinement, wrist-joint velocities are ignored, so the effective refinement velocity input is $\bm v^\text{pred}\in \mathbb{R}^{15\times 3}$ [2404.10227].

Second, for consecutive frame pairs $(\bm p_i,\bm v_i), (\bm p_{i+1},\bm v_{i+1})$, IDNet infers muscle excitation $\bm a_i$. Starting from $(\bm p_i^\text{pred}, \bm v_i^\text{pred})$, the simulator applies forward dynamics under $\bm a_i$ and produces a biomechanically valid reference state $(\bm p_{i+1}^\text{ref}, \bm v_{i+1}^\text{ref})$ [2404.10227].

Third, a lightweight MLP combines the predicted and reference poses:

$$
\bm p^\text{refined} = \mathcal{M}(\bm p^\text{pred}_{t+1}, \bm p^\text{ref}_{t+1}),
$$

with supervised loss

$$
\mathcal L_\text{refine} = \left\lVert \bm p^\text{gt} - \bm p^\text{refined} \right\rVert.
$$

BioPR therefore does not directly replace the base estimator’s prediction with the simulator output. Instead, it uses the simulated pose as a biomechanical prior for learned correction [2404.10227]. This distinction is significant: the framework is designed to preserve the image-conditioned estimate while biasing it toward physically and anatomically plausible configurations.

## 5. Architecture, training regime, and implementation details

The implementation uses a relatively small model footprint. **IDNet** has input size **16×3×4**, output size **31** for the **31 hand-muscle excitations**, and **two 256-dimensional hidden layers**. It is trained with **PPO** using **Stable-Baselines3** on an **NVIDIA A40 GPU**, with learning rate $3\times 10^{-4}$ and adaptive scheduling. Training injects Gaussian noise $N(0, 0.1)$ degrees into joint rotations, uses **128 distributed processes × 64 agents each**, a batch size of **16384 per step**, approximately **10K FPS** simulation throughput, and takes roughly **1 hour** [2404.10227].

The simulator uses **RFUniverse (Unity/Kinesis)** and includes **31 muscles for the hand** and **8 muscles for wrist control**. Muscle maximum isometric force and contractile element length are sourced from MyoHand/MyoSuite data. The implementation also includes forearm and wrist structure via **SMPLX integration** to capture muscles originating in the forearm [2404.10227].

The **Refine Net** is an MLP with input size **93**—reported as **48 for $\bm p^\text{pred}$ and 45 for wrist-ignored inputs**—a single **64-dimensional hidden layer**, and output size **48**. It is trained for **4,500 iterations** with batch size **10,240** and learning rate $1\times 10^{-3}$ on an **A40 GPU**, taking roughly **5 minutes** [2404.10227]. Parameter counts remain small: approximately **0.1M** for IDNet and **0.01M** for Refine Net, while BioPR adds roughly **9 ms** inference overhead.

The paper also compares MS-MANO against heuristic priors. Temporal smoothing is formulated as

$$
\hat{\bm p}_i = \arg\min\limits_{\bm m}\sum_{j=i-d/2}^{i+d/2}{\left\lVert\bm p_j - \bm m\right\rVert_2^2}.
$$

According to the reported ablation, temporal smoothing and TOCH yielded **<0.8 mm** improvement in MPJPE on DexYCB when added to gSDF, whereas PCA increased MPJPE by **7.91 mm** [2404.10227]. In context, this is used to argue that biomechanical simulation contributes a stronger regularizing effect than handcrafted temporal heuristics.

## 6. Empirical evaluation, observed behavior, and scope

The evaluation is split into two parts. First, the musculoskeletal model itself is compared with **MyoSuite** by stimulating specific muscles and examining joint trajectories. For ring-finger bending under **FDS4_R** excitation, normalized time-position plots of three ring-finger joints show similar movement patterns between MS-MANO and MyoHand, while MS-MANO exhibits a quicker response to muscle stimulation [2404.10227]. The paper interprets this as suggesting fast activation behavior consistent with human physiology, but does not provide explicit numeric metrics for this comparison.

Second, **BioPR** is benchmarked on **DexYCB** and **OakInk** using two recent MANO-based baselines, **gSDF** and **Deformer**. On DexYCB, the reported metrics are **MPJPE (mm)**, **AUC**, and **AE (mm/s\(^2\))** [2404.10227].

| Method | DexYCB results |
|---|---|
| gSDF | MPJPE 14.4, AUC 89.1, AE 30.3 |
| gSDF + BioPR | MPJPE 12.81, AUC 89.7, AE 29.9 |
| Deformer | MPJPE 13.64, AUC 89.6, AE 31.7 |
| Deformer + BioPR | MPJPE 12.92, AUC 90.4, AE 30.7 |

These results show consistent improvements in both accuracy and temporal smoothness for the two baselines [2404.10227]. OakInk is used to assess robustness in dynamic interactions, but the provided text does not include explicit numerical results.

Qualitative analysis emphasizes reduction of failure modes that are familiar in image-based hand tracking: more anatomically aligned finger tightness when grasping a mustard bottle, avoidance of odd thumb twisting, and fewer implausible penetrations under severe occlusion [2404.10227]. The stated interpretation is that biomechanical constraints stabilize predictions in visually degraded conditions.

The paper also delineates the framework’s limitations. It remains dependent on the base estimator, so errors induced by occlusion or lighting can persist; it is sensitive to incorrect annotations; and extremely slow motions are difficult because weak excitations generate small torques and therefore minimal frame-to-frame offsets [2404.10227]. The system likewise omits explicit joint coupling formulas, activation equations, tendon compliance equations, and object-contact force modeling. Future work is suggested in differentiable simulation, personalized muscle parameters, explicit Hill-type parameterizations, and contact-aware interaction modeling.

## 7. Relation to adjacent work and terminological distinctions

Within hand modeling, MS-MANO is positioned against both **joint-actuated MANO-only approaches** and **standalone musculoskeletal simulators**. Relative to MANO-only systems, its defining feature is the integration of a comprehensive musculoskeletal system adapted from MyoSuite into MANO through joint-centric mapping, while preserving MANO’s learnability and shape adaptivity [2404.10227]. Relative to MyoSuite’s **MyoHand**, its novelty lies in embedding musculoskeletal actuation into a parametric MANO framework and coupling it to a learning-based refinement pipeline, rather than operating solely as a simulator [2404.10227].

The paper further distinguishes itself from full-body musculoskeletal integrations such as **BASH** and **RCare World** by focusing on detailed hand muscles and forearm/wrist routing for hand pose tracking tasks [2404.10227]. Practical application areas identified in the paper include **AR/VR avatar hands**, **teleoperation**, **human-robot interaction**, **dexterous manipulation**, and **biomechanics**, where physiological realism is linked to plausibility, safety, and control [2404.10227].

The name **MS-MANO** also requires terminological disambiguation. In the literature provided here, **MeMaHand** uses MANO in a different sense, namely a token-level interaction between dense mesh prediction and MANO parameter prediction for two-hand reconstruction from a single RGB image; that work does not define MS-MANO as a musculoskeletal hand simulator [2303.15718]. In a separate domain, **MANO** denotes **Management and Orchestration** in 5G/NFV systems, as in open-source MANO frameworks, benchmarking studies, and optimization-enhanced MANO for network slicing and energy efficiency [2010.03849]; [1904.10697]; [1907.10669]. Another distinct use is **MaNo**, standing for **Matrix Norm**, for unsupervised accuracy estimation under distribution shift; that paper explicitly states that the term **MS-MANO** does not appear there [2405.18979]. In the present context, therefore, MS-MANO refers specifically to a **muscle- and tendon-driven extension of MANO for physiologically constrained hand pose tracking** [2404.10227].

Source: https://www.emergentmind.com/topics/ms-mano