Personalization of Wearable Sensor-Based Joint Kinematic Estimation Using Computer Vision for Hip Exoskeleton Applications (2411.15366v1)

Abstract: Accurate lower-limb joint kinematic estimation is critical for applications such as patient monitoring, rehabilitation, and exoskeleton control. While previous studies have employed wearable sensor-based deep learning (DL) models for estimating joint kinematics, these methods often require extensive new datasets to adapt to unseen gait patterns. Meanwhile, researchers in computer vision have advanced human pose estimation models, which are easy to deploy and capable of real-time inference. However, such models are infeasible in scenarios where cameras cannot be used. To address these limitations, we propose a computer vision-based DL adaptation framework for real-time joint kinematic estimation. This framework requires only a small dataset (i.e., 1-2 gait cycles) and does not depend on professional motion capture setups. Using transfer learning, we adapted our temporal convolutional network (TCN) to stiff knee gait data, allowing the model to further reduce root mean square error by 9.7% and 19.9% compared to a TCN trained on only able-bodied and stiff knee datasets, respectively. Our framework demonstrates a potential for smartphone camera-trained DL models to estimate real-time joint kinematics across novel users in clinical populations with applications in wearable robots.

• The paper presents an adaptive framework that integrates wearable sensors and computer vision to reduce RMSE by up to 19.9% using transfer learning.
• The methodology combines temporal convolutional networks and human pose estimation to achieve real-time joint kinematic estimation from limited gait cycles.
• The approach offers personalized, cost-effective solutions for hip exoskeleton control, enhancing rehabilitation and robotic assistance in irregular gait patterns.

Personalization of Wearable Sensor-Based Joint Kinematic Estimation Using Computer Vision for Hip Exoskeleton Applications

This paper explores an innovative framework for estimating lower-limb joint kinematics utilizing wearable sensors and computer vision technologies. It specifically focuses on improving kinematic estimation accuracy for hip exoskeleton applications, which is of critical importance in rehabilitation, patient monitoring, and exoskeleton control. The presented framework highlights the integration of temporal convolutional networks (TCNs) and computer vision, aiming to address existing limitations in wearable sensor-based systems for joint kinematic estimation, particularly in cases of irregular gait patterns such as stiff knee (SK) gait.

Contributions and Methodology

The authors propose an adaptation framework that leverages computer vision-based deep learning models to provide accurate joint kinematic estimations using significantly smaller datasets than traditionally required. The research comprises several key components:

1. Wearable Sensing Suit: The paper employs a wearable sensing suit equipped with inertial measurement units (IMUs) placed on the pelvis and thighs, providing data for real-time inference.
2. Computer Vision Integration: The framework integrates a computer vision-based human pose estimation (HPE) pipeline utilizing the MMpose library, ViTPose for keypoint estimation, and Video-Pose3D for reconstructing 3D kinematics from monocular video inputs. This method bypasses the need for expensive and cumbersome motion capture systems.
3. Machine Learning Model: A Temporal Convolutional Network (TCN) is employed for processing time-series data from the wearable sensors, enabling efficient model training and real-time kinematic estimation.
4. Adaptation and Transfer Learning: The framework uses transfer learning to adapt a TCN model, initially trained on able-bodied (AB) data, to new gait patterns (such as SK gait) by incorporating limited vision-extracted kinematic data.

Results and Implications

The authors report that their adaptive framework significantly reduces error rates. The adapted model achieved a reduction of 9.7% and 19.9% in root mean square error (RMSE) compared to models trained solely on AB and SK datasets, respectively. This reduction was achieved using only 1-2 gait cycles worth of new training data from the SK pattern, highlighting the efficiency and adaptability of the proposed method.

These findings have substantial implications, particularly in scenarios requiring real-time and accurate kinematic data such as robotic exoskeleton control. The successful integration of computer vision methods to facilitate wearable sensor systems enables more personalized and precise assistance strategies without assuming access to high-cost professional motion capture facilities. The efficiency in adapting to new gait patterns makes this framework especially valuable in personalized healthcare and rehabilitation contexts.

Future Directions

The paper lays the groundwork for future research in several areas:

• Enhanced Data Collection Setup: Future iterations could involve multiple camera systems to address occlusion issues and further improve the accuracy of markerless motion capture options, potentially incorporating novel models like Sapiens or WHAM.
• Smartphone-Based Systems: Transitioning the framework from high-end motion capture environments to accessible smartphone platforms for wider applicability in diverse settings, including home and remote clinical environments.
• Broader Dataset Expansion: Expanding AB datasets to improve the robustness and generalizability of baseline models across diverse populations and gait patterns.
• Robotic Exoskeleton Applications: Implementing the framework for enhanced real-time control of robotic exoskeletons, facilitating better synchronization with user gait and personalized assistance.

In summary, the paper presents a compelling approach to improving joint kinematic estimation through the integration of wearable sensor technology and computer vision, providing a pathway toward more adaptive and personalized systems in biomechanical applications.