EMFANT: Infant Sensorimotor Simulation
- EMFANT is a musculoskeletal, infant-specific virtual embodiment that simulates first-person sensorimotor experience with realistic proprioceptive, tactile, and visual signals.
- The system retargets infant motion using biomechanically constrained inverse kinematics and PD-controlled forward dynamics to produce high-frequency, synchronized multimodal data.
- EMFANT prioritizes biological plausibility over pure geometric accuracy, offering deeper insights into infant development and potential early detection of neurodevelopmental disorders.
Searching arXiv for the main paper and cited components to ground citations. EMFANT is a musculoskeletal, infant-specific virtual embodiment used to simulate first-person sensorimotor experience from retargeted infant motions. It was introduced as part of a framework for reconstructing infant movement from video, mapping that movement onto humanoid embodiments, and replaying it to generate synchronized multimodal streams comprising proprioception, touch, and binocular vision. Within that framework, EMFANT is distinguished less by minimum kinematic error than by biological plausibility: it is designed to represent infant anatomy and biomechanics closely enough to generate realistic proprioceptive signals, including muscles, while enforcing infant-appropriate joint couplings and ranges of motion (López et al., 30 Apr 2026).
1. Conceptual role and design objective
EMFANT is the musculoskeletal virtual embodiment in a broader pipeline for simulating infant first-person sensorimotor experience. Its stated goal is biological plausibility rather than purely geometric replay. In practice, this means that the embodiment is intended to produce realistic proprioceptive signals, tactile responses, and binocular vision while constraining motion according to infant anatomy and biomechanics (López et al., 30 Apr 2026).
The system addresses a limitation identified in conventional human-to-humanoid motion retargeting: many methods reproduce kinematics while ignoring the associated sensorimotor consequences of movement. EMFANT is therefore positioned as an embodiment for studying how retargeted infant motion gives rise not only to joint trajectories but also to muscle-based proprioception, self-contact, and first-person visual experience. This makes it relevant to developmental science, robotics, and work on early detection of neurodevelopmental disorders.
A central design implication is that EMFANT is not merely a visualization target. It is a physically simulated developmental platform in which the replayed motion generates multimodal data streams that can be analyzed as time-synchronized correlates of infant movement. This suggests a shift from retargeting as pose transfer to retargeting as sensorimotor reconstruction.
2. Anatomical model, simulation engine, and sensor modalities
Architecturally, EMFANT comprises 17 rigid body segments and 37 degrees of freedom. Its morphology is derived from infant CT/MRI via segmentation to produce anatomically realistic skin, bones, and muscle routing. The physics and rendering engine is MuJoCo, contact is handled with MuJoCo’s polygonal contact model, and cameras are rendered binocularly (López et al., 30 Apr 2026).
The sensory stack is specified as follows.
| Modality | Representation | Specification |
|---|---|---|
| Proprioception | Joint angles and muscle-based signals | 37 DOF; 162 muscle–tendon lengths |
| Touch | Skin mesh tactile sensing | 5,000 tactile sensors across the whole body |
| Vision | Binocular cameras | Two cameras, 90° field of view, 1000×1000 pixels per eye |
All sensory streams are available at 1 kHz, matching the simulation rate, so the simulator yields synchronized, high-frequency multimodal time series. Contact forces from MuJoCo’s collision resolution are distributed to nearby skin sensors to generate tactile activations, following the procedure cited in the source description. Because the cameras are head-mounted, with one per eye, and the head belongs to the same rigid-body system undergoing forward dynamics, the first-person viewpoint remains automatically synchronized with head and eye motion (López et al., 30 Apr 2026).
The proprioceptive representation is especially notable. EMFANT exposes both joint-level and muscle-level signals: joint angles over 37 DOF and muscle-based signals from 162 muscle–tendon lengths. The latter are described as the biologically richer channel and are used to support mechanistic analysis of musculoskeletal coordination. Relative to simpler geometric embodiments, this expands the notion of proprioception beyond configuration variables alone.
3. From infant video to EMFANT replay
The pipeline from recorded infant behavior to EMFANT replay begins with skeletal extraction and full-body 3D pose estimation. For the infant recording used in the study, consisting of 2,900 frames over 1:56 with multi-camera input, 2D keypoints are detected independently per view using ViTPose. When multi-view input is available, corresponding 2D keypoints are triangulated to obtain 3D coordinates across frames. If only monocular input is available, the source description states that 3D pose and shape can alternatively be recovered by fitting a parametric body model to 2D projections, for instance with SMPLify-X and an infant body model such as SMIL (López et al., 30 Apr 2026).
In the reported implementation, ViTPose and multi-view correspondence are used to reconstruct framewise 3D infant keypoints, which then serve as retargeting targets for EMFANT. Before inverse kinematics, EMFANT is scaled to the recorded infant’s body size. This scaling step is central to the embodiment: calibration to the infant is achieved by resizing EMFANT prior to retargeting, which supports lower geometric error and preserves first-person relevance checks defined relative to the infant-sized body.
Motion retargeting onto EMFANT is formulated as biomechanically constrained inverse kinematics plus forward dynamics tracking. Inverse kinematics is solved in OpenSim while respecting anatomical constraints such as joint couplings, including cervical interactions and scapulohumeral rhythm, as well as infant-appropriate range-of-motion limits. The OpenSim solver produces joint-angle trajectories that satisfy biomechanical constraints while best matching the reconstructed infant keypoints. Those target trajectories are then replayed in MuJoCo by EMFANT using joint-space PD control (López et al., 30 Apr 2026).
The source description explicitly notes several omissions. It does not provide an explicit optimization objective in LaTeX for the inverse kinematics, and it does not specify additional camera intrinsics or logging file formats. Temporal smoothing is described as implicit in the inverse-kinematics and controller tracking over time rather than as a separately defined post-processing stage.
4. Dynamics, control, and coordinate conventions
Replay in EMFANT uses position tracking via joint-space PD control rather than torque-level control. At each time step, the controller commands target positions for all joints, MuJoCo computes forward dynamics, integrates the system, and resolves contacts through its polygonal contact model. This arrangement is intended to robustly follow inverse-kinematics targets while maintaining stability under contact (López et al., 30 Apr 2026).
Actuator constraints are realized implicitly through two mechanisms: infant-appropriate range-of-motion limits and joint coupling embedded in the EMFANT model, and PD gains tuned to stably follow the time-varying targets under contact. The paper does not specify the exact controller gains. The result of this control architecture is physically consistent replay that simultaneously produces proprioceptive, tactile, and visual streams at 1 kHz.
Tactile signals are generated when replayed motion induces contact events, including self-contacts such as hand-to-torso interactions. MuJoCo resolves the contact forces, and these forces are mapped to nearby skin sensors to produce spatially distributed tactile activations. This enables analyses such as self-touch detection and localization from the infant’s perspective. Because tactile sensing is defined over a whole-body skin mesh, the representation is not limited to sparse event detection.
For evaluations involving first-person relevance, the framework defines “relative orientations” using vectors from a body-center anchor at the MidHip keypoint to end effectors. This coordinate convention supports checks such as whether the hand falls in the camera frustum. “Relative velocities” are computed as the mean absolute error of changes in those orientation vectors in degrees per step. These definitions matter because EMFANT is used not only to reproduce motion but also to assess whether reconstructed motion remains meaningful relative to the embodied visual field.
5. Retargeting accuracy and cross-embodiment analysis
Evaluation involving EMFANT focuses on retargeting accuracy and cross-embodiment sensorimotor consistency. Geometric fidelity is quantified through “relative distances,” defined as the mean absolute error between infant 3D keypoints and the humanoid’s retargeted keypoints, scaled to infant body dimensions when necessary. Because EMFANT’s morphology can be resized to match the infant, it achieves among the lowest mean absolute error values in the reported comparison, with all platforms under 10 cm for the session shown. However, MIMo performs best in raw kinematic fidelity, reaching 4.6 mm MAE in the reported session (López et al., 30 Apr 2026).
For first-person relevance, “relative orientations” and “relative velocities” are evaluated using the MidHip-centered vector definitions. MIMo again performs best overall, with average orientation MAE of 1.96° and velocity MAE of 0.15°/step. EMFANT and iCub/pyCub are higher, though the source description characterizes them as within acceptable ranges, with velocities below 1°/step and orientation errors above 5° for iCub/pyCub. EMFANT’s stated strength therefore lies less in sub-millimeter kinematic accuracy than in biomechanical realism and proprioceptive richness.
Cross-embodiment invariance is evaluated by fusing tactile, proprioception, and binocular vision streams from iCub, EMFANT, and MIMo into windowed features, then z-scoring and Frobenius-normalizing modality blocks, concatenating them, and performing PCA followed by Generalized Procrustes Analysis alignment to obtain latent spaces . Pairwise Spearman correlations are reported as around 0.4 for all pairs. An invariance index based on the minimum bidirectional mean transfer accuracy across the three embodiment pairs increases with latent dimensionality and plateaus around at 0.19%, which is interpreted in the source as indicating a shared, compact multimodal manifold across embodiments (López et al., 30 Apr 2026).
A plausible implication is that EMFANT’s value is maximized when the research objective is not only pose fidelity but also latent alignment across heterogeneous sensory embodiments. In that setting, musculoskeletal realism can be a meaningful axis of comparison even when another embodiment attains lower pointwise kinematic error.
6. Relation to MIMo, pyCub, and iCub
EMFANT is compared directly with pyCub, iCub, and MIMo along the axes of fidelity, ease of use, and realism. MIMo is described as MuJoCo-based and composed of simple geometric primitives such as capsules and a sphere head. It can be “grown” and resized per session and uses MuJoCo’s inverse kinematics with mocap bodies to directly minimize keypoint MAE, which yields the best kinematic fidelity in the comparison. pyCub and the physical iCub provide realistic eye cameras and large-area tactile skins, but retargeting is geometric and constrained by stricter joint limits and fixed anthropometry corresponding to a 4-year-old body (López et al., 30 Apr 2026).
EMFANT differs from both classes of embodiment by emphasizing infant-specific musculoskeletal realism. Its distinguishing properties include infant-appropriate joint couplings and range-of-motion limits, a muscle–tendon model producing 162 proprioceptive signals, and physically consistent forward dynamics under PD control. The source description therefore identifies EMFANT as the embodiment of choice for studying biologically grounded proprioception and muscle coordination during early development.
The trade-off is explicit. MIMo is preferred for ultra-accurate pose replay and fast multimodal simulation. pyCub and iCub are preferred when alignment with a physical robot and real-world constraints is central. EMFANT is preferred when the priority is mechanistic analysis of proprioception and anatomically plausible constraints. This comparison also clarifies a common misconception: lower retargeting MAE does not by itself imply greater biological realism or greater usefulness for developmental interpretation.
7. Demonstrated uses, implementation availability, and open questions
Applications demonstrated with EMFANT include multimodal analysis and enhanced automated behavior annotation. By replaying infant motion in EMFANT, the framework visualizes and records muscle–tendon length dynamics together with touch and binocular vision, enabling analysis of how muscle-based proprioception complements vision and touch in emergent whole-body coordination. In the proof-of-concept automated annotation study, distributions of self-touches detected as collisions are computed over six regions: head, torso, left arm, right arm, left leg, and right leg (López et al., 30 Apr 2026).
In the same proof of concept, the simulated iCub is reported to capture the infant’s lateralization best, including 13% left-hand to left leg and 33% right-hand to right leg versus human-annotated 9% and 17%. EMFANT nonetheless produces usable self-touch distributions and, because it supplies richer proprioceptive signals, is described as potentially supporting semi-automated coding workflows that require only expert validation. Its tactile mapping and binocular camera streams further support multimodal annotation tasks such as hand regard and self-touch localization from the infant’s perspective.
The full pipeline, including EMFANT retargeting and multimodal logging, is available in the motion-retargeting repository identified in the source. The documented workflow consists of running ViTPose on each camera view, triangulating 2D detections to 3D keypoints, scaling EMFANT’s morphology to infant size, solving inverse kinematics in OpenSim, and replaying the motion in MuJoCo with joint-space PD control while recording joint angles, muscle–tendon lengths, tactile skin activations, and binocular vision at 1 kHz. Configuration parameters exposed include camera resolution and field of view, specified as 1000×1000 pixels and 90°, and sensor rates at 1 kHz (López et al., 30 Apr 2026).
Several limitations are stated directly. EMFANT’s kinematic retargeting accuracy does not reach MIMo’s sub-centimeter MAE in the reported session. Its tactile skin resolution, at 5,000 sensors, is coarser than MIMo’s 13,833, which may limit fine-grained touch localization. The paper does not detail the exact inverse-kinematics objective, controller gains, or logging formats, and generalization to different infants and diverse movement contexts, including object interactions, remains to be tested at scale. Future work includes deployment on larger infant datasets, study of atypical developmental trajectories, extension to children and adults, and incorporation of more complex whole-body and object interactions. In the robotics direction, the source suggests that EMFANT’s biologically plausible proprioception and constraints could contribute to foundation models for humanoids enriched with touch and muscle-level signals, thereby helping narrow the embodiment gap (López et al., 30 Apr 2026).