Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences (1810.07538v1)

Abstract: Statistical models of the human body surface are generally learned from thousands of high-quality 3D scans in predefined poses to cover the wide variety of human body shapes and articulations. Acquisition of such data requires expensive equipment, calibration procedures, and is limited to cooperative subjects who can understand and follow instructions, such as adults. We present a method for learning a statistical 3D Skinned Multi-Infant Linear body model (SMIL) from incomplete, low-quality RGB-D sequences of freely moving infants. Quantitative experiments show that SMIL faithfully represents the RGB-D data and properly factorizes the shape and pose of the infants. To demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of freely moving infants and show, with a case study, that our method captures enough motion detail for General Movements Assessment (GMA), a method used in clinical practice for early detection of neurodevelopmental disorders in infants. SMIL provides a new tool for analyzing infant shape and movement and is a step towards an automated system for GMA.

• The paper introduces the Skinned Multi-Infant Linear (SMIL) model, learned from low-quality RGB-D sequences, to accurately track the 3D body shape of freely moving infants.
• A novel methodology adapts adult body models, fuses data across multiple frames, and learns an infant-specific shape space for robust tracking despite occlusions and movement.
• The SMIL model achieves high accuracy (2.51 mm error) and shows significant potential for automating General Movements Assessment in clinical applications for early diagnosis.

Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D Sequences

The paper presented by Hesse et al., titled "Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences," addresses a unique challenge in the field of computer vision and medical imaging. It proposes a methodology for capturing and modeling the 3D body shape of infants using RGB-D cameras, overcoming the limitations of traditional 3D scanning methods which are not feasible with the infant population. This work introduces the Skinned Multi-Infant Linear model (SMIL), a statistical body model learned from low-quality, incomplete RGB-D sequences of infants, and demonstrates its utility in the context of medical assessments.

Methodology and Architecture

The authors present a comprehensive approach that begins with adapting the adult Skinned Multi-Person Linear model (SMPL) to create an initial parameterized mesh model of infants. This adaptation involves not only altering the shape to reflect infant body proportions but also scaling the pose blend shapes to accommodate smaller body sizes. The significant innovation in this work lies in its ability to derive accurate 3D models from low-quality data that include incomplete views due to occlusions and the infants' inability to hold poses or follow instructions.

The process consists of several critical phases:

1. Data Preprocessing: RGB-D data is captured in a clinical setting, where infants are not subjected to specific scanning protocols, allowing freedom of movement that is essential for natural data collection. The acquired data undergoes noise reduction and segmentation to distinguish the infant from the background and clothing from skin.
2. Initialization and Registration: The model is initialized based on selected frames where body parts are optimally visible. Registration of the initial model to the point cloud data is achieved through optimization techniques that consider factors such as landmark matching and proximity to the table plane.
3. Fusion and Personalization: The paper introduces the concept of a fusion cloud, which aggregates information across multiple frames to overcome the challenge of incomplete data from a single frame. This approach allows the generation of personalized infant shapes that capture specific characteristics not represented in the original SMPL shape space.
4. Learning the Shape Space and Pose Prior: With personalized shapes from each sequence, a weighted principal component analysis (WPCA) is performed to create a new shape space specific to infants. This new shape space, combined with a learned pose prior, forms the backbone of the SMIL model, suitable for analyzing infant motion.

Results and Contributions

The SMIL model is evaluated for accuracy and practicality. Quantitative assessments show that SMIL achieves an impressive average scan-to-mesh distance error of 2.51 mm. The model provides significant potential in the application of General Movements Assessment (GMA), a clinical tool for early detection of neurodevelopmental disorders. By automating parts of this assessment, the proposed system can reduce reliance on human judgment, which is subject to variability.

Furthermore, the authors demonstrate the model's generalizability by testing it on older infants, capturing detailed and challenging movements, such as self-contact motions. This indicates the robustness of the SMIL model in various infant postures and motions.

Implications and Future Work

The introduction of SMIL opens new avenues for automating the analysis of infant motions, with potential implications in early diagnosis and monitoring of developmental disorders. The model facilitates a non-intrusive and low-cost method for gathering motion data, making it attractive for clinical use.

Future developments could focus on enhancing the model to capture finer details, like facial expressions and minute hand movements, which are currently limited by the resolution of the data. Additionally, expanding the dataset by integrating data from different demographics and environments could further strengthen SMIL's applicability and reliability.

In conclusion, this paper makes a substantial contribution to the field by providing a framework that bridges the gap between advanced computer vision techniques and practical, real-world medical applications. The SMIL model sets a foundational step towards more comprehensive and automated assessments of infant movement, promising improvements in early medical interventions.