OpenMoCap: Robust Optical Motion Capture

Updated 4 July 2026

OpenMoCap is a robust optical motion capture model designed to accurately reconstruct human motion amidst sustained marker occlusion in real-world scenarios.
It employs a decoupled position and rotation solver with marker–joint chain inference, leveraging self-attention to capture long-range dependencies.
The model achieves significant reductions in joint position error on CMU and SFU datasets, validating its effectiveness over existing methods.

OpenMoCap is a motion-solving model for optical motion capture under large-scale real-world marker occlusion. Introduced in "OpenMoCap: Rethinking Optical Motion Capture under Real-world Occlusion" (Qian et al., 18 Aug 2025), it addresses two limitations identified in existing optical mocap pipelines: the lack of training datasets that accurately reflect realistic marker occlusion patterns, and the absence of training strategies designed to capture long-range dependencies among markers. The work couples a new dataset, CMU-Occlu, with a decoupled position-and-rotation inference architecture and reports integration into the MoSen MoCap system for practical deployment.

1. Problem setting and failure modes

Optical motion capture in studio and production settings typically relies on dozens of infrared cameras surrounding an actor wearing reflective markers. Reconstruction by triangulation requires each marker to be seen by at least two cameras simultaneously. In this setting, OpenMoCap targets the regime in which many markers, especially those on the abdomen, chest, or forehead, become blocked by the actor’s own body, props, or set pieces for long intervals (Qian et al., 18 Aug 2025).

The paper isolates three failure modes. First, there is co-visibility failure: if fewer than two cameras see a marker, its 3D position cannot be triangulated. Second, real occlusions exhibit structured, sustained occlusion patterns rather than the frame-wise random dropout commonly used in synthetic datasets; clusters of neighboring markers can be affected for tens or hundreds of consecutive frames. Third, there is a breakdown of local-only inference: methods that fill in a missing marker by looking only at spatial neighbors fail when entire regions go dark.

Within this framing, OpenMoCap is not presented as a generic open-source mocap label but as a model specialized for robust motion solving under highly structured occlusion. Its central premise is that realistic occlusion simulation and explicit propagation of information across long spatial chains are both necessary when conventional triangulation collapses over extended temporal intervals.

2. CMU-Occlu and physics-based occlusion synthesis

CMU-Occlu is constructed by injecting a physics-based occlusion simulator into the CMU synthetic corpus. The simulator represents each camera ray as

$\mathbf{R}(t)=\mathbf{r}_0+t\,\mathbf{d},$

where $\mathbf{r}_0$ is the camera center and $\mathbf{d}$ its viewing direction. The body mesh is modeled as triangles with vertices $\mathbf{v}_1,\mathbf{v}_2,\mathbf{v}_3$ , and a marker point $\mathbf{p}$ on a triangle satisfies

$\mathbf{p}=(1-\beta-\gamma)\,\mathbf{v}_1+\beta\,\mathbf{v}_2+\gamma\,\mathbf{v}_3,\quad \beta,\gamma\ge0,\;\beta+\gamma\le1.$

Occlusion is determined by solving the Möller–Trumbore intersection

$\mathbf{r}_0+t\,\mathbf{d} =(1-\beta-\gamma)\mathbf{v}_1+\beta\,\mathbf{v}_2+\gamma\,\mathbf{v}_3$

for $t,\beta,\gamma$ . A marker is occluded on a ray if the closest intersection $t_{\rm hit}$ satisfies

$t_{\rm hit}<\|\mathbf{p}-\mathbf{r}_0\|.$

A marker is declared lost if fewer than two rays reach it (Qian et al., 18 Aug 2025).

The construction pipeline then samples dozens of virtual camera layouts whose occlusion statistics best match those of SFU-Mocap by minimizing KL divergence, oversamples frames with heavy occlusion, and adds random shifts to the remaining visible markers. The reported result is a dataset whose marker-wise occlusion probability and duration histograms closely mirror real-world data. This suggests that CMU-Occlu is intended not merely as augmentation, but as a training distribution aligned with the temporal and spatial structure of practical optical occlusion.

3. Architecture and marker–joint chain inference

OpenMoCap decouples motion solving into a Position Solver and a Rotation Solver (Qian et al., 18 Aug 2025). The Position Solver takes as input the raw 3D positions of all visible markers, while occluded markers are represented by a small set of shared, learnable embeddings. Feature extraction uses a mixture of 1D convolutions to capture local marker co-occurrence and self-attention to enable global context.

Its distinctive mechanism is marker–joint chain inference. The model forms a bipartite directed graph $\mathbf{r}_0$ 0 of markers and joints. At layer $\mathbf{r}_0$ 1 it computes a learned propagation matrix $\mathbf{r}_0$ 2. After $\mathbf{r}_0$ 3 steps, each node representation is

$\mathbf{r}_0$ 4

The paper characterizes this as a bidirectional scheme in which joint estimates inform distant marker completion and vice versa, establishing chains such as foot $\mathbf{r}_0$ 5 ankle $\mathbf{r}_0$ 6 knee $\mathbf{r}_0$ 7 hip $\mathbf{r}_0$ 8 opposite foot. The Position Solver outputs reconstructed 3D positions $\mathbf{r}_0$ 9 for all markers and $\mathbf{d}$ 0 for all joints.

The Rotation Solver first aligns the reconstructed markers to a canonical T-pose by Procrustes alignment on eight pelvis-region reference markers, removing global translation and rotation. A stack of multihead self-attention blocks then maps aligned marker positions to joint rotations $\mathbf{d}$ 1 in continuous 6D form. The decoupled design separates recovery of marker geometry from articulation inference, while the chain-inference stage provides the long-range dependency propagation that local neighborhood methods lack under region-level marker dropout.

4. Supervision, constraints, and optimization regime

The supervision strategy uses Euclidean and angular objectives. For the Position Solver, the loss combines terms for accuracy on truly occluded markers, refinement of originally visible but perturbed markers, and joint-position accuracy. In practice the weights are set to $\mathbf{d}$ 2 (Qian et al., 18 Aug 2025). The occluded-marker term is defined over $\mathbf{d}$ 3, the visible-but-shifted term over $\mathbf{d}$ 4, and the joint term over all joints $\mathbf{d}$ 5.

For the Rotation Solver, the loss is

$\mathbf{d}$ 6

where $\mathbf{d}$ 7 and $\mathbf{d}$ 8 are predicted and ground-truth 6D rotation matrices, and $\mathbf{d}$ 9 maps a rotation matrix to its axis-angle vector.

Training uses several explicit stabilizers. Data augmentation applies random marker occlusions up to 20% and small Gaussian shifts on visible markers. Curriculum learning gradually increases the fraction of heavily occluded frames from $\mathbf{v}_1,\mathbf{v}_2,\mathbf{v}_3$ 0 during early epochs to stabilize convergence. Occluded markers and joints both start from learned codebooks that quickly adapt to global patterns of missingness. The reported regimen uses batch size 256 on an RTX-4090 and trains end-to-end for 80 epochs (Qian et al., 18 Aug 2025).

These design choices collectively suggest a training setup tuned for prolonged missingness rather than isolated dropout. In particular, the curriculum on heavily occluded frames indicates that optimization difficulty is treated as a function of occlusion severity, not only of sample count.

5. Quantitative evaluation and system deployment

Evaluation uses Joint Position Error (JPE, cm) and Joint Orientation Error (JOE, degrees) on CMU, CMU-Occlu, and SFU (Qian et al., 18 Aug 2025).

Method	CMU JPE/JOE	CMU-Occlu JPE/JOE
MoSh++	2.58 cm / 9.40°	2.72 cm / 9.68°
MoCap-Solver	2.56 / 6.51	2.95 / 6.83
LocalMoCap	0.94 / 3.59	1.23 / 3.80
RoMo	0.89 / 3.43	1.16 / 3.54
OpenMoCap	0.41 / 2.52	0.46 / 2.60

The paper states that this represents over a 27% reduction in JPE relative to RoMo on CMU, from 0.89 to 0.41 cm, with similarly large gains under heavy occlusion.

On the real-world SFU test, the paper reports the following when models are trained on CMU versus CMU-Occlu:

Method	SFU JPE/JOE (trained on CMU)	SFU JPE/JOE (trained on CMU-Occlu)
MoCap-Solver	5.50 / 10.03	5.73 / 10.21
LocalMoCap	1.93 / 4.86	1.41 / 4.25
RoMo	1.46 / 4.78	1.38 / 4.22
OpenMoCap	0.40 / 4.47	0.39 / 4.10

The reported transfer from CMU-Occlu to SFU indicates that realistic occlusion synthesis improves real-data robustness, especially for OpenMoCap and the stronger learned baselines.

OpenMoCap is embedded in MoSen, described as a low-cost mocap rig using off-the-shelf IR cameras. Marker tracking runs in real time on a GPU and feeds 3D point clouds to the OpenMoCap network. The Position Solver fills in any untriangulated markers online, and the Rotation Solver immediately outputs a full-body skeleton. No manual post-cleanup or fitting is required. In a Russian-twist live demo, OpenMoCap achieves a JPE of 1.01 cm versus 4.73 cm for RoMo and 7.68 cm for MoCap-Solver (Qian et al., 18 Aug 2025). The code is released at https://github.com/qianchen214/OpenMoCap.

6. Terminology, neighboring systems, and scope

A recurring source of ambiguity is that OpenMoCap can denote the specific optical motion-solving model of Qian Chen and collaborators, whereas adjacent literature uses closely related language for broader open or open-environment motion-capture systems. In the present usage, OpenMoCap refers specifically to the optical model designed for environments with significant marker occlusions (Qian et al., 18 Aug 2025).

By contrast, "Mesquite MoCap: Democratizing Real-Time Motion Capture with Affordable, Bodyworn IoT Sensors and WebXR SLAM" describes an open-source inertial system combining a body-worn network of 15 IMU sensor nodes with a hip-worn Android smartphone for position tracking, operating cross-platform entirely in the browser (Vanani et al., 27 Dec 2025). "Ubiquitous Robot Control Through Multimodal Motion Capture Using Smartwatch and Smartphone Data" presents WearMoCap, an open-source library with Watch Only, Upper Arm, and Pocket modes for arm-pose estimation and robot control using consumer wearables (Weigend et al., 2024). "FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments" introduces a calibration-free hybrid method that combines a single LiDAR with expandable moving cameras for global multi-person motion capture in open environments (Xue et al., 2024). "An Open Source, Fiducial Based, Visual-Inertial Motion Capture System" instead uses a monocular camera, IMU, and AprilTag fiducials within an EKF-SLAM formulation to provide pose and velocity estimates without precalibration (Neunert et al., 2015).

This broader context suggests that contemporary open motion-capture research spans heterogeneous sensing stacks: optical marker systems, body-worn IMUs, hybrid LiDAR-camera systems, and visual-inertial fiducial pipelines. OpenMoCap occupies the optical-marker branch of that landscape and is distinguished there by its emphasis on realistic occlusion synthesis, marker–joint chain inference, and decoupled position/rotation solving under sustained co-visibility failure.