SpaER: Learning Spatio-temporal Equivariant Representations for Fetal Brain Motion Tracking

Abstract: In this paper, we introduce SpaER, a pioneering method for fetal motion tracking that leverages equivariant filters and self-attention mechanisms to effectively learn spatio-temporal representations. Different from conventional approaches that statically estimate fetal brain motions from pairs of images, our method dynamically tracks the rigid movement patterns of the fetal head across temporal and spatial dimensions. Specifically, we first develop an equivariant neural network that efficiently learns rigid motion sequences through low-dimensional spatial representations of images. Subsequently, we learn spatio-temporal representations by incorporating time encoding and self-attention neural network layers. This approach allows for the capture of long-term dependencies of fetal brain motion and addresses alignment errors due to contrast changes and severe motion artifacts. Our model also provides a geometric deformation estimation that properly addresses image distortions among all time frames. To the best of our knowledge, our approach is the first to learn spatial-temporal representations via deep neural networks for fetal motion tracking without data augmentation. We validated our model using real fetal echo-planar images with simulated and real motions. Our method carries significant potential value in accurately measuring, tracking, and correcting fetal motion in fetal MRI sequences.

• The paper presents SpaER, which leverages spatio-temporal equivariant representations to dynamically track fetal brain motion and improve tracking precision.
• It employs 3D steerable CNNs with self-attention mechanisms to integrate temporal encoding and correct geometric distortions in MRI data.
• Experimental results demonstrate a significant reduction in translation error (approximately 3.8 mm) compared to traditional methods, enhancing clinical MRI evaluations.

Overview of "SpaER: Learning Spatio-temporal Equivariant Representations for Fetal Brain Motion Tracking"

The paper introduces SpaER, a method designed to enhance fetal motion tracking, specifically addressing challenges associated with fetal brain MRI. The method innovates by utilizing spatio-temporal representations through equivariant filters and self-attention mechanisms, achieving a dynamic tracking of the fetal head's movement over time, differing from traditional static estimate methodologies. The approach is significant due to its ability to process long-term dependencies and compensate for misalignment caused by contrast variations and severe motions without relying on data augmentation techniques.

Methodological Advances

The methodology is rooted in the development of an equivariant neural network capable of learning rigid motion sequences in a reduced spatial representation of images. Key methodological components include:

• Equivariant Neural Networks: These networks utilize 3D steerable CNNs to capture rotationally equivariant features. The framework effectively learns low-dimensional spatial means that are essential for tracking rigid motion accurately.
• Spatio-temporal Encoding: SpaER leverages temporal encoding akin to the one used in Transformers to integrate time-series data into the spatial image representations, facilitating the capture of complex motion patterns.
• Self-attention Mechanisms: By incorporating a multi-layer self-attention network, the approach ensures that spatial and temporal features are comprehensively utilized to improve the precision of motion tracking.
• Geometric Deformation Estimation: The model integrates a geometric correction module that addresses local image distortions and assists in refining motion tracking accuracy. This is achieved through the application of diffeomorphic transformations ensuring the physiological plausibility of the deformations.

Experimental Validation

The proposed approach was validated on a dataset comprising 240 sequences of fetal MRI scans. The results, presented in comparison to established approaches such as DeepPose, KeyMorph, and Equivariant Filters, demonstrated SpaER's superior performance in reducing translation and angular errors. For example, SpaER achieved a translation error of approximately 3.8 mm, showcasing a marked improvement over other models.

Implications and Future Work

The SpaER framework has significant implications for clinical practices, particularly in improving the reliability and accuracy of fetal MRI examinations by offering an advanced motion correction tool. Beyond immediate practical applications, the study contributes a theoretical framework that can generalize across diverse image modalities without requiring extensive retraining. Future research directions might explore the adaptation of this methodology to other medical imaging domains where motion artifacts present challenges, as well as the integration of the proposed model into real-time MRI scanning frameworks.

By addressing both the theoretical and practical aspects of fetal brain motion tracking, the paper makes a meaningful contribution to medical imaging, specifically in scenarios where conventional methods face limitations due to the dynamic and unpredictable nature of fetal movements.