Health Guardian: Using Multi-modal Data to Understand Individual Health

Abstract: AI has shown great promise in revolutionizing the field of digital health by improving disease diagnosis, treatment, and prevention. This paper describes the Health Guardian platform, a non-commercial, scientific research-based platform developed by the IBM Digital Health team to rapidly translate AI research into cloud-based microservices. The platform can collect health-related data from various digital devices, including wearables and mobile applications. Its flexible architecture supports microservices that accept diverse data types such as text, audio, and video, expanding the range of digital health assessments and enabling holistic health evaluations by capturing voice, facial, and motion bio-signals. These microservices can be deployed to a clinical cohort specified through the Clinical Task Manager (CTM). The CTM then collects multi-modal, clinical data that can iteratively improve the accuracy of AI predictive models, discover new disease mechanisms, or identify novel biomarkers. This paper highlights three microservices with different input data types, including a text-based microservice for depression assessment, a video-based microservice for sit-to-stand mobility assessment, and a wearable-based microservice for functional mobility assessment. The CTM is also discussed as a tool to help design and set up clinical studies to unlock the full potential of the platform. Today, the Health Guardian platform is being leveraged in collaboration with research partners to optimize the development of AI models by utilizing a multitude of input sources. This approach streamlines research efforts, enhances efficiency, and facilitates the development and validation of digital health applications.

• The paper presents the Health Guardian platform as an innovative framework using AI and multi-modal data to accurately assess clinical conditions.
• It details the integration of text, video, and wearable microservices for advanced evaluation of depression, Parkinson’s disease, and mobility.
• The study leverages cloud-based microservices and dynamic cohort management to optimize clinical study designs and research workflows.

Health Guardian: Using Multi-modal Data to Understand Individual Health

Introduction

The field of Digital Health has significantly evolved, leveraging advanced technologies like IoT devices, mobile sensors, and cloud computing to improve personalized healthcare delivery. Digital biomarkers acquired through various technological means offer a robust methodology to create a healthcare "digital twin," and AI-powered models assist in generating health insights for proactive interventions (Figure 1). This paper presents the Health Guardian platform, a non-commercial scientific research framework that leverages AI to enhance disease assessment and management through cloud-based microservices. The platform's flexible architecture supports diverse data types, enabling comprehensive health evaluations.

Figure 1: Envisioned information flow diagram to create a virtual replica of a person, utilizing IoT technologies for biosignal acquisition.

Health Guardian Platform

The Health Guardian (HG) platform is designed for efficient AI/ML model deployment through microservices, enabling the assessment and management of individual health using multi-modal data inputs. The platform supports creating customized data pipelines for processing text, audio, and video, with results feedback enabling model accuracy improvement. The HG architecture rests on a compliance services layer and is built on OpenShift and multi-cloud infrastructure (Figure 2).

Figure 2: Overview of the Health Guardian Platform, depicting the diverse services and microservice integration for AI-based analytics.

The platform facilitates the processing of over 70 analytics capabilities across domains such as neurodegenerative diseases, mental health, and general wellness. Critical to this is the Clinical Task Manager (CTM), which facilitates clinical study setups by enabling cohort management and microservice deployment. The CTM enhances study design efficiency and supports dynamic assignment of microservices to study cohorts, ensuring streamlined research.

Integration of Text-based Microservice for Depression Assessment

Clinical Background

The prevalence of depression necessitates effective evaluation tools like the PHQ-8 questionnaire. Traditional methods lack frequency and remote access, thus the text-based PHQ-8 microservice within HG enhances depression tracking flexibility and accuracy, providing structured data (Figure 3).

Digital Health Solution

The mobile-based PHQ-8 microservice enables regular monitoring through structured text response submission. Upon completion, responses are processed to determine depression severity, with results viewable via mobile applications interfaced with clinician dashboards, enhancing intervention decision-making.

Figure 3: PHQ-8 depression assessment microservice workflow illustrating user interaction and data processing.

Integration of Video-based Microservice for Sit-to-Stand Mobility Assessment

Clinical Background

Parkinson's Disease management relies on the accurate assessment of motor symptoms like bradykinesia and PIGD. By utilizing computer vision techniques, the sit-to-stand microservice predicts UPDRS subscores from short video clips, bypassing limitations of self-reports and wearable compliance issues.

Digital Health Solution

The video-based microservice processes sit-to-stand motions using pose estimation models, providing UPDRS score predictions and real-time motion graphs to assist clinicians (Figure 4).

Figure 4: Sit-to-stand microservice processing demonstrating the video-based prediction and analysis of UPDRS scores.

Integration of Wearable-based Microservice for Functional Mobility Assessment

Clinical Background

The Timed Up and Go Test (TUG) is a standard for evaluating gait and balance, now integrated as a wearable-based microservice. The automated prediction model utilizes accelerometer data from wrist-worn devices, enabling continuous monitoring of functional mobility beyond traditional snapshot assessments.

Digital Health Solution

This microservice captures accelerometer data from wrist devices, processing it to predict TUG scores indicative of fall risk and mobility impairment, facilitating proactive management (Figure 5).

Figure 5: Implementation of TUG assessment microservice demonstrating wearable data processing and score prediction.

Discussion

The HG microservices allow for expanded clinical condition assessment through multi-modal data. The CTM portal significantly enhances study design and execution efficiency by managing subject cohorts, defining tasks, and dynamically assigning microservices. This integration facilitates comprehensive data collection, processing, and insightful analysis, thus fostering advanced real-world AI research in digital health.

Conclusions

The Health Guardian platform exemplifies an end-to-end solution for digital health research, supporting multi-modal data acquisition, robust analytics, and streamlined study designs. Through the Clinical Task Manager, researchers can efficiently deploy microservices, optimizing AI model development and clinical application validation. The scalable architecture facilitates novel insights into individual health, accelerating digital health innovation.

---

This essay demonstrates the comprehensive capabilities of the Health Guardian platform in enhancing digital health through meticulous AI-driven analytics and multi-modal data integration.