CMU-Occlu Dataset: Realistic Marker Occlusion in MoCap

• The dataset introduces a synthetic benchmark that combines CMU MoCap's scale with ray-traced occlusion to mimic real marker dropouts.
• It employs a parallelized Möller–Trumbore ray–triangle intersection test to determine marker visibility using camera subsets aligned with real-world statistics.
• The benchmark offers detailed annotations and evaluation protocols based on joint-space error metrics, bridging the gap between synthetic and practical occlusion scenarios.

Searching arXiv for the specified paper to ground the article and citation. arxiv_search(query="(Qian et al., 18 Aug 2025)", max_results=5) CMU-Occlu is a motion-capture dataset introduced in "OpenMoCap: Rethinking Optical Motion Capture under Real-world Occlusion" (Qian et al., 18 Aug 2025). It is built on top of a SMPL-driven version of the original CMU MoCap corpus and is designed to model realistic marker occlusion patterns rather than only "random" marker dropouts. In the paper’s framing, the dataset addresses a central failure mode of optical motion capture under large-scale marker occlusions common in real-world applications: existing training data often do not accurately reflect practical occlusion statistics, and models trained on such data fail under prolonged, structured occlusions. CMU-Occlu therefore combines the scale and motion diversity of synthetic CMU-derived data with a ray-tracing pipeline intended to reproduce occlusion patterns closer to those observed in the real SFU dataset (Qian et al., 18 Aug 2025).

1. Origin, definition, and intended role

CMU-Occlu is presented as a synthetic benchmark for robust motion solving under substantial occlusion. The dataset inherits the SMPL-driven CMU set of 5,000 sequences, approximately 8 million frames, and uses a visibility-generation procedure based on camera rays and body-mesh intersections rather than uniform random deletion of markers (Qian et al., 18 Aug 2025). This design is explicitly motivated by two limitations identified in current models: the lack of training datasets accurately reflecting realistic marker occlusion patterns, and the absence of training strategies designed to capture long-range dependencies among markers.

The dataset is introduced together with OpenMoCap, a motion-solving model that uses a marker-joint chain inference mechanism for simultaneous optimization and construction of deep constraints between markers and joints. Within that larger framework, CMU-Occlu functions as the training and evaluation substrate for studying solver behavior when markers are persistently out of view. A plausible implication is that the dataset is intended not merely as augmentation for existing synthetic corpora, but as a benchmark for methods whose robustness depends on structured occlusion statistics rather than isolated corruption events.

2. Ray-traced occlusion generation

The defining methodological feature of CMU-Occlu is its ray-tracing pipeline. Each virtual infrared camera casts rays

$r(t) = r_0 + t \cdot d$

toward each marker point $p$ on the body mesh. A triangular mesh face is defined by vertices $v_1, v_2, v_3$, and a point $p$ on that triangle satisfies

$$p = (1-\beta-\gamma)\cdot v_1 + \beta \cdot v_2 + \gamma \cdot v_3,\qquad 0 \le \beta,\gamma,\beta+\gamma \le 1.$$

To test occlusion, the procedure solves for $(t,\beta,\gamma)$ from

$$r_0 + t \cdot d = (1-\beta-\gamma)\cdot v_1 + \beta \cdot v_2 + \gamma \cdot v_3$$

via the standard linear system inversion of $[-d, v_2-v_1, v_3-v_1]$ (Qian et al., 18 Aug 2025).

The implementation is a parallelized variant of the Möller–Trumbore ray–triangle intersection test. If the nearest intersection parameter $t_n$ satisfies

$t_n < \|p-r_0\|,$

the ray hits the body before reaching the marker, and that camera does not see the marker. A marker is deemed visible in a given frame only if at least two of the selected cameras register no occlusion.

Camera selection is also data-driven. Four-camera subsets are drawn from a pool of typical real-world MoCap layouts, and the authors choose the four-camera combinations whose occlusion-statistics most closely match those from the real SFU dataset by minimizing the Kullback–Leibler divergence of per-marker occlusion distributions. To increase the prevalence of hard cases, an oversampling strategy is applied to frames where more than $p$0 of the markers are occluded; the details of $p$1 are left to the reader, but the stated purpose is to oversample "rare" heavy-occlusion frames. On top of simulated occlusions, classical random dropouts and small random spatial shifts are still applied so that downstream solvers remain robust to both long-term and short-term marker corruption.

3. Dataset composition and occlusion statistics

CMU-Occlu inherits the motion diversity of the SMPL-driven CMU corpus. The data cover walking, running, dancing, jumping, sports, and a broad set of daily living and athletic motions drawn from the original CMU MoCap archive (Qian et al., 18 Aug 2025). The body-shape space is driven by shape parameters from the CAESAR anthropometry set and includes approximately 1,700 distinct bodies. The marker-layout space includes approximately 5,100 different marker-to-mesh mappings, with each layout placing between 40 and 70 markers, reflecting typical optical MoCap rigs.

The virtual environment is deliberately constrained: it is a plain studio with static obstacles limited to the subject’s own body mesh, with no furniture or scene geometry beyond the actor. Camera layouts use four cameras at varying azimuth and elevation selected to emulate real studios. This makes the dataset specifically a benchmark for body-mesh-induced marker occlusion under studio-style camera arrangements. A common misunderstanding would be to treat it as a benchmark for arbitrary scene occlusion; the stated environment description does not support that interpretation.

The dataset’s central empirical claim concerns its statistical relation to real occlusion. Relative to vanilla CMU synthetic data, whose per-marker occlusion probability is described as highly uniform, low variance, and dominated by short bursts, CMU-Occlu is reported to match SFU much more closely. The paper states that per-marker occlusion duration in CMU-Occlu follows the heavy-tail of SFU, meaning that many markers remain out of view for tens to hundreds of consecutive frames. Its plots in Fig. 2 and Fig. 5 show that the KL divergence between CMU-Occlu and SFU in both occlusion probability and occlusion length is an order of magnitude smaller than between CMU and SFU. Grouping test sequences by percentage of markers occluded per frame at 5%, 10%, 15%, and 20% further shows that CMU-Occlu contains a significant fraction of frames in the 15–20% range, which the paper describes as closely reproducing real-world conditions (Qian et al., 18 Aug 2025).

4. Annotation schema, representation, and metadata

The dataset distinguishes raw and processed representations. In the raw representation, each frame contains a list of $p$2 marker entries $p$3 plus a boolean visibility flag. Invisible markers may be omitted or set to a special value such as NaN (Qian et al., 18 Aug 2025). In the processed representation, the data contain solved 3D joint positions and joint rotations in 6D or axis-angle form, aligned to a canonical T-pose.

The coordinate convention used for solver input is explicit: the centroid of visible markers is subtracted per frame before feeding the data into the solver. Marker identities are tied to the underlying SMPL mesh location; the reference gives examples such as marker_01=head and marker_02,left_shoulder, and notes that a JSON or text file lists the mapping from marker ID to mesh triangle. Joint definitions follow the SMPL kinematic tree of 24 body joints, and in processed data each joint has $p$4 and a 6D rotation vector.

The paper does not specify file extensions. It notes only that typical practice would be C3D or per-frame text/CSV. That omission is relevant for practitioners because it means the logical content of the dataset is defined more clearly than its serialized packaging. A plausible implication is that interoperability is expected to rely on the released code, data converters, and starter scripts rather than on a formally standardized container format.

5. Partitions and evaluation protocol

The dataset uses a three-way split over the 5,000 SMPL-CMU sequences. Training consists of 80% of the sequences with full simulated occlusions and random noise. Validation consists of 10% held out and sampled uniformly across occlusion severities. Test consists of 10% held out and further subdivided into occlusion buckets of 5%, 10%, 15%, and 20% of markers occluded per frame (Qian et al., 18 Aug 2025). In addition, the real SFU dataset, comprising eight actors and 44 sequences, is used only for cross-domain evaluation.

The evaluation protocol centers on joint-space error metrics. Joint Position Error (JPE, in cm) is defined as the mean Euclidean distance between predicted and ground-truth joint positions. Joint Orientation Error (JOE, in degrees) is the mean angular error between predicted and ground-truth rotations, using the continuous 6D representation for stability. The protocol also includes occlusion-sensitivity curves, which report JPE and JOE as a function of the fraction of occluded markers.

These choices make the benchmark sensitive to both geometric recovery and rotational fidelity under varying corruption regimes. Because the test set is explicitly bucketed by occlusion severity and paired with an external real-data evaluation on SFU, the benchmark is structured to separate performance on synthetic in-domain corruption from cross-domain robustness. This suggests that absolute in-domain accuracy and resilience to realistic occlusion statistics are treated as distinct evaluation targets rather than interchangeable outcomes.

6. Relation to existing benchmarks, usage conventions, and significance

The paper positions CMU-Occlu between two established reference points. Vanilla synthetic CMU data are described as having large scale and varied motions, but only random short-term occlusions, with the consequence that models trained on them fail under prolonged, structured occlusions. SFU is described as having authentic occlusions and noise, but as small, with eight actors and 44 sequences, and without ground-truth marker visibility flags. CMU-Occlu is therefore characterized as bridging the gap by combining CMU’s scale with SFU’s occlusion statistics while providing per-marker visibility labels and remaining synthetic enough to generate millions of frames (Qian et al., 18 Aug 2025).

The usage guidance is correspondingly specific. Training should use both the simulated ray-traced occlusions and a small amount of random marker dropouts, because this dual strategy yields the best downstream robustness. Training batches should be balanced so that heavy-occlusion frames, defined as at least 15% of markers missing, are not vanishingly rare; the authors used oversampling. Missing markers should be masked using the provided visibility flags rather than zero-imputed. Each frame should be centralized by its visible-marker centroid before network input. For fair benchmarking, the recommended procedure is to follow the published splits and report JPE and JOE on the four occlusion-severity buckets and on the SFU hold-out set.

The broader significance of CMU-Occlu lies in its attempt to make marker visibility itself a modeled variable rather than an afterthought of synthetic data generation. The dataset opens the door for future studies in robust motion solving, and the associated OpenMoCap system is integrated into the MoSen MoCap system for practical deployment. At the same time, the dataset’s virtual environment excludes furniture and other scene geometry, so its realism is targeted rather than universal. The most accurate characterization is therefore not that it reproduces all real-world occlusion, but that it reproduces a specific and practically important class of real-world occlusion statistics under optical studio capture conditions.