Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions

Abstract: Wearable devices record physiological and behavioral signals that can improve health predictions. While foundation models are increasingly used for such predictions, they have been primarily applied to low-level sensor data, despite behavioral data often being more informative due to their alignment with physiologically relevant timescales and quantities. We develop foundation models of such behavioral signals using over 2.5B hours of wearable data from 162K individuals, systematically optimizing architectures and tokenization strategies for this unique dataset. Evaluated on 57 health-related tasks, our model shows strong performance across diverse real-world applications including individual-level classification and time-varying health state prediction. The model excels in behavior-driven tasks like sleep prediction, and improves further when combined with representations of raw sensor data. These results underscore the importance of tailoring foundation model design to wearables and demonstrate the potential to enable new health applications.

• The paper introduces a foundation model (WBM) that integrates behavioral data from wearables to improve both static and time-varying health predictions.
• It utilizes optimized tokenization strategies like TST and advanced architectures such as Mamba-2 to effectively process irregular wearable data.
• The study demonstrates that incorporating behavioral data complements sensor data, offering actionable insights for personalized healthcare and preventive interventions.

"Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions"

Introduction

The paper "Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions" explores the use of foundation models to enhance health predictions by leveraging behavioral signals derived from wearable devices. The authors propose a foundation model trained on an extensive dataset from the Apple Heart and Movement Study (AHMS), consisting of over 2.5 billion hours of data from 162,000 participants. This foundation model is optimized for health detection tasks and demonstrates strong performance, particularly in behavior-driven tasks such as sleep prediction.

Methodology

The foundation model relies on optimizing tokenization strategies and neural network architectures for the unique characteristics of wearable data, which often faces irregular sampling and missing data. The authors compare different tokenization approaches, such as TST, mTAN, and Tuple, to handle these challenges. They also explore various sequence-to-sequence architectures, including Transformers with self-attention, Rotary Position Embeddings, and state-space models like Mamba-2. Through empirical evaluation, the paper identifies the TST tokenization and Mamba-2 architecture as the most effective combination.

Figure 1: An overview of the approach to solving health detection tasks, emphasizing the complementary nature of behavioral and PPG data.

The model development process uses a regularized contrastive self-supervised learning (SSL) objective, leveraging both InfoNCE and KoLeo regularization to enhance representation learning. The authors conduct extensive hyperparameter tuning across different combinations of tokenization and architectural paradigms.

Results and Evaluation

The foundation model, called the Wearable Behavior Model (WBM), is subjected to rigorous evaluation across various health prediction tasks. Notably, WBM demonstrates superior performance in sleep-related and behavior-driven health tasks, while also providing complementary insights when combined with a PPG foundation model.

Figure 2: The final WBM embedding pipeline that converts irregularly sampled data into a dense matrix for input to a neural network.

In addition to static health state predictions, WBM excels in time-varying health conditions, outperforming traditional sensor-based models in tasks such as pregnancy detection and sleep efficiency prediction. The model's embeddings capture valuable information from daily behavioral patterns, highlighting their utility beyond conventional sensor data.

Figure 3: WBM achieves strong predictive performance for baseline medical history and medications, outperforming a baseline model.

Discussion and Implications

The study underscores the importance of incorporating higher-level behavioral metrics into wearables foundation models. The research affirms that behavioral data offers valuable health insights that complement existing sensor data, thus broadening the scope and applicability of health detection models.

The successful integration of contextual and behavioral information into foundation models opens avenues for developing more comprehensive health monitoring systems. By improving the detection of both static and dynamic health states, these models promise significant advancements in personalized healthcare and preventive interventions.

Conclusion

The paper presents compelling evidence for the utility of behavioral data in enhancing health predictions through foundation models. The proposed WBM not only captures critical health indicators from behavioral patterns but also complements existing sensor-based models, paving the way for more holistic health detection systems.

Figure 4: Tuple tokenization framework, which efficiently aggregates wearable activity data into token representations.