Towards a General Intelligence and Interface for Wearable Health Data

Abstract: While ubiquitous wearable sensors capture a wealth of behavioral and physiological information, effectively transforming these signals into personalized health insights is challenging. Specifically, converting low-level sensor data into representations capable of characterizing higher-level states is difficult due to high phenotypic diversity and variation in individual baseline health, physiology, and lifestyle factors. Moreover, collecting wearable data paired with health outcome annotations is laborious and expensive, and retrospective annotation remains practically unfeasible, contributing to a scarcity of data with high-quality labels. To overcome these limitations, we propose a foundation model for wearable health that is pretrained on more than one trillion minutes of unlabeled sensor signals drawn from a large cohort of five million participants. We demonstrate that the joint scaling of model capacity and pretraining data volume leads to systematic improvements in performance, as evaluated on a diverse set of 35 health prediction tasks, spanning cardiovascular, metabolic, sleep, and mental health, as well as lifestyle choices and demographic factors. We find that this population scale representation unlocks label-efficient few-shot learning and generative capabilities for robust daily metric estimation. To further leverage this learned representation, we deploy a classroom of LLM agents to autonomously search the space of downstream predictive heads built on the model embeddings, showing broad performance improvements that increase with LLM model capacity. Finally, we show how integrating these downstream predictors into a Personal Health Agent can support model responses that are more relevant, contextually aware, and safe, and we validate this via 1,860 ratings from a cohort of clinicians.

• The paper introduces SensorFM, a population-scale foundation model trained on over one trillion minutes of wearable data to achieve robust health task performance.
• It employs a ViT-1D backbone with adaptive masking and agentic LLM frameworks to enhance imputation and downstream predictive accuracy across diverse health tasks.
• The model exhibits near-linear scaling across 35 health measures, offering clinically safe, personalized integration with AI health agents.

SensorFM: A Population-Scale Foundation Model and General AI Interface for Wearable Health

Introduction

The development of models capable of extracting useful, generalizable health representations from large-scale wearable sensor data presents unique challenges due to high phenotypic diversity, sparse annotated labels, and inherent data fragmentation. Addressing these, "Towards a General Intelligence and Interface for Wearable Health Data" (SensorFM) (2605.22759) introduces a population-scale sensor foundation model trained on over one trillion minutes of multimodal wearable data from five million participants. The study systematically establishes the empirical scaling laws that govern representation learning in wearable health, robustly evaluates generative and discriminative transfer across 35 distinct health tasks, harnesses agentic LLM frameworks for downstream model adaptation, and demonstrates the benefit of grounding health agents with these representations in clinical evaluation.

Figure 1: The SensorFM pretraining and evaluation pipeline at population scale, with generalizability across generative and discriminative health tasks, and strong scaling properties under joint increases in data and model capacity.

Model Architecture and Data Curation

SensorFM leverages a ViT-1D backbone with AIM (Adaptive and Inherited Masking) for robust learning on patchified, highly fragmented, minute-level sensor streams. Pretraining employs masked autoencoder objectives, reconstructing artificially masked segments while respecting naturally missing data arising from device usage variability.

Figure 2: SensorFM ingests 34 one-minute aggregated features from five sensor modalities, automatically handling diverse missingness patterns and artifacts essential to free-living wearable data.

Figure 3: Pretraining pipeline showing normalization, diverse masking schemes (temporal, sensor, random), and masked patch reconstruction with MSE loss.

Diversity and scale are a distinctive strength: the model is pretrained on globally sampled, device-heterogeneous data capturing demographic, geographic, and behavioral variation, vastly exceeding prior datasets both in size and resolution.

Figure 4: Global and US state-level demographic coverage of training and downstream study participants, ensuring broad generalization.

Scaling Laws and Predictive Performance

SensorFM empirically demonstrates that joint scaling of data and model capacity yields monotonic, near-linear improvement in both pretraining loss and downstream discriminative and generative performance—paralleling scaling phenomena seen in NLP and vision foundation models.

Figure 5: Discriminative performance across 35 health tasks as a function of model scale, illustrating that scale reliably boosts transfer across demographic, lifestyle, metabolic, cardiovascular, mental health, and sleep tasks.

Across 35 real-world tasks, including both clinical outcomes (e.g., hypertension, diabetes, ASCVD risk, depression/anxiety, sleep disturbance) and behavioral/demographic variables, SensorFM achieves win rates exceeding 90% versus supervised classifiers on feature-engineered summaries. The B (largest) variant notably outperforms demographic-only or manually engineered baselines, especially when abundant pretraining data is available.

Generative Capabilities and Data Imputation

SensorFM's reconstructive pretraining imparts strong generative capabilities for imputation and forecasting in the high-missingness regime ubiquitous to wearables. The model far surpasses traditional numerics (mean fill, nearest-neighbor, linear interpolation) in imputing missing sensor streams—achieving relative improvements up to 75–84% MSE reduction for random and blockwise missing patterns.

Figure 6: Multimodal day-long heatmap reconstructions; SensorFM fills fragmented intervals with physiologically plausible, non-linear inferences.

Figure 7: High-resolution generative gap-filling of fragmented signals such as HRV and REM with strong local contextual fidelity.

Figure 8: Robust imputation of large multi-hour blocks, leveraging global circadian and physiological priors despite lack of immediate local context.

Pragmatically, the imputed reconstructions yield daily activity, sleep, and cardiorespiratory summaries that nearly eliminate the underestimation biases seen in naive aggregation, retaining >99% of true value despite artificial loss of one hour per day.

Representation Analysis: Latent Structure and Attribution

Analysis of SensorFM's learned representation reveals that model scale governs both the dispersion and the compressibility of the latent space. Larger models create higher-entropy embeddings with broad physiological coverage, supporting improved downstream discrimination without dimensional collapse.

Figure 9: UMAP visualizations show that task-relevant clustering emerges in the latent space across a variety of physiological and behavioral health outcomes.

Figure 10: Larger models yield broader embedding dispersions and slower explained-variance saturation, indicating preservation of a long tail of discriminative features beyond dominant "super-features".

Attribution via SHAP confirms a decrease in reliance on explicit demographic features as pretraining scale increases: the model implicitly encodes demographic and physiological priors at scale.

Figure 11: (a) Cosine similarity chord diagram of SHAP attributions: correlated tasks (e.g., ASCVD and Framingham risk) share embedding dimensions, while others diverge, mapping underlying physiological association structures. (b) Feature attribution shifts toward embeddings and away from demographics as model/data scale increase.

Agentic Model Head Discovery

Addressing the bottleneck of selecting and tailoring downstream heads, the authors deploy a multi-agent LLM "classroom" architecture, in which collaborative/competitive agents iteratively generate code solutions for each target task. This automation, inspired by agentic code-generation advances, enables large-scale, efficient search over modeling strategies (e.g., architecture, regularization, preprocessing).

Figure 12: LLM-driven classroom architecture for algorithmic search, with progress curves for multiple agents and diverse downstream tasks.

Classroom-derived solutions (often linear with dimensionality reduction) frequently surpass static linear probes and demonstrate further headroom for performance gains as model/agent intelligence increases or as agents collaborate.

Figure 13: Classification and regression task improvements from agent-classroom discovered heads over linear probes.

Figure 14: Agent collaboration smooths stratification and elevates weak agents, with meta-analysis indicating a preference for lower-dimensional, often linear or ensemble heads.

Integration Into Health Agents and Clinical Utility

SensorFM serves as an AI physiological interface: its outputs can be provided as context to a personal health agent (LLM) to answer patient queries with clinical relevance and safety. Compared to agents acting directly on raw or engineered features, SensorFM-grounded responses are rated by physicians as significantly more relevant, personalized, context-appropriate, and safe, with no statistically significant inferiority to agents provided actual ground-truth labels.

Figure 15: SensorFM-augmented health agent workflow, with improved physician Likert ratings for context-aware and safe responses. Ratings by clinical rubric dimensions show substantial shift toward optimal scores with SensorFM context.

Figure 16: Distributions of physician ratings show the pronounced shift toward optimal scores for closeness to ground truth, contextualization, relevance, and safety with SensorFM integration.

Implications, Limitations, and Future Directions

SensorFM demonstrates that population-scale, domain-tailored foundation models can subsume the value of extensive feature engineering and demographic priors in wearable health, and are broadly label-efficient—even for tasks where annotation is scarce, noisy, or subjective (e.g., mental health). The integration of agentic model search/optimization with these representations further narrows the gap between generalized pretraining and clinical deployment.

However, limitations remain: generalization beyond the covered device population and data distribution is not guaranteed, especially across wearables with differing hardware or signal characteristics. Granularity is restricted to minute-level features for scalability, which omits some high-frequency phenomena. Downstream evaluations are limited by available label types and cross-sectional, single-turn agentic deployments rather than real-world, multi-turn interactions.

Further research will explore transfer and adaptation to new device platforms and geographies, streaming and real-time application, integration with EHRs, longitudinal personalized modeling, and advanced clinical endpoints.

Conclusion

SensorFM establishes a new population-scale benchmark for foundation modeling in wearable health. Joint increases in model and data scale predictably improve generative and discriminative health transfer, and the learned representation serves as a powerful substrate for downstream model search, imputation, and clinical integration. As both a signal-processing front-end and an AI interface, SensorFM accelerates the realization of general, label-efficient, and safe AI for continuous, personalized health monitoring and guidance.

---

Reference:

"Towards a General Intelligence and Interface for Wearable Health Data" (2605.22759)