---
title: 'SensorFM: Wearable Sensing Foundation Model'
url: https://www.emergentmind.com/topics/sensorfm
type: topic
---

# SensorFM: Wearable Sensing Foundation Model

Searching arXiv for SensorFM and closely related sensing foundation-model papers to ground the article.
SensorFM is a foundation model for wearable health that is pretrained on more than one trillion minutes of unlabeled sensor signals drawn from a large cohort of five million participants and is designed to transform low-level wearable streams into a general-purpose latent representation for downstream health inference [2605.22759]. In the supplied literature, the term also appears as a broader conceptual framing for sensing-native foundation models trained once and adapted across tasks, including WiFi ambient intelligence, smart-home event analysis, and RF human activity recognition [2602.11200][2602.01910][2410.19766]. The most specific and technically developed use, however, is the wearable-health model evaluated on 35 health prediction tasks, generative reconstruction benchmarks, and clinician-rated conversational assistance [2605.22759].

## 1. Problem formulation and conceptual scope

SensorFM addresses a recurrent problem in wearable sensing: low-level sensor streams are abundant, but labels for clinically meaningful or behaviorally meaningful states are scarce, expensive, and often impossible to obtain retrospectively [2605.22759]. The paper motivates the model by emphasizing high phenotypic diversity and variation in individual baseline health, physiology, and lifestyle factors, which make bespoke supervised models brittle and difficult to scale. Rather than training a separate predictor for each outcome, SensorFM is introduced as a self-supervised representation learner whose embedding can support linear probes, few-shot adaptation, generative reconstruction, and agentic downstream systems.

Within that framing, SensorFM is not merely a classifier. It is a reusable backbone intended to characterize higher-level states from long, noisy, and partially missing wearable signals. This distinguishes it from pipelines built around engineered wearable summary statistics, and also from downstream systems that rely primarily on demographic covariates. The paper’s empirical comparisons are structured to test precisely whether the learned embedding itself carries useful physiological information across heterogeneous tasks [2605.22759].

A common misreading is to conflate SensorFM with an LLM-centric system. The core model is instead a masked reconstructive foundation model over sensor sequences; LLMs appear later as a downstream search mechanism for predictive heads and as part of the Personal Health Agent, not as the representation-learning backbone [2605.22759].

## 2. Wearable data substrate and pretraining corpus

SensorFM is pretrained on minute-resolution, multimodal wearable features extracted from five sensors on Fitbit Sense 2 / Pixel Watch 2–style devices: PPG, accelerometer, electrodermal activity, skin temperature, and altimeter [2605.22759]. These raw inputs are converted into 34 one-minute aggregate features spanning heart rate / HRV, SpO\(_2\), sleep stages, motion, and skin-surface physiology. The representation problem is therefore posed at the level of structured minute-scale sensor summaries rather than raw waveforms.

The scale of pretraining is central to the system definition. The paper reports over 1 trillion minutes of sensor data from 5 million participants, spanning five sensor modalities and collected over roughly 10 weeks per person for pretraining sampling, yielding 175,062,146 day-long samples in the largest pretraining set [2605.22759]. Devices span many generations from 2012–2025. Independent validation and test sets of 10,000 subjects each are held out.

This corpus construction matters because SensorFM’s main claim is not only that self-supervision is useful, but that joint scaling of model capacity and data volume produces systematic downstream gains. The underlying data regime is therefore part of the model’s identity, not merely a training detail. A plausible implication is that SensorFM should be understood as a population-scale representation learner rather than a narrow benchmark model.

## 3. Architecture, masking strategy, and objective

SensorFM uses a masked autoencoder–style objective with a ViT-1D encoder–decoder backbone [2605.22759]. The encoder operates over sequences of wearable-feature tokens, while the decoder reconstructs masked portions of the input. The model uses 2D additive positional encodings over time and feature dimension, with sinusoidal encodings for temporal position and learned embeddings for feature position and cyclic time metadata.

Self-supervision is implemented through an Adaptive and Inherited Masking (AIM) scheme. True missingness from the device or user is combined with synthetic masking for training, and the masking patterns include random patch masking, contiguous temporal block masking, and modality block masking [2605.22759]. The training objective is reconstruction under realistic missingness rather than purely synthetic corruption. In the formulation given in the paper, the pretraining loss is

$$
\mathcal{L}_{\text{pretrain}}=\text{MSE}(\text{reconstructed masked tokens}, \text{original masked tokens}).
$$

The MSE is computed only on artificially masked tokens that were originally observed [2605.22759]. This restriction is important because it separates the learning target from true device-side missingness.

The architectural design implies that SensorFM is optimized to learn temporal regularities, cross-feature dependencies, and missingness-robust latent structure simultaneously. Because the model is trained reconstructively rather than contrastively, its embedding is explicitly tied to infilling and interpolation behavior as well as discriminative transfer.

## 4. Scaling behavior and discriminative transfer

A major contribution of the paper is the claim that model capacity and pretraining data volume scale predictably with performance [2605.22759]. The study sweeps four model sizes—XXS, XS, S, and B—corresponding to about \(10^5\), \(10^6\), \(10^7\), and \(10^8\) parameters, and four data volumes—5K, 50K, 500K, and 5M participants—corresponding to roughly \(2\times10^6\), \(2\times10^7\), \(2\times10^8\), and \(2\times10^9\) hours. The strongest results come from joint scaling of both dimensions. The large model trained on the largest data volume achieves about 31% lower reconstruction validation loss than the smallest variant, about 28% lower average MSE on generative tasks, about \(\Delta AUC = 0.09\) average improvement on classification tasks, and about \(\Delta r = 0.21\) average improvement on regression tasks. On the 35-task benchmark, the largest model ranks best on 33/35 tasks.

The downstream evaluation covers 35 person-level tasks spanning cardiovascular health, metabolic health, mental health, sleep, lifestyle, and demographics [2605.22759]. The downstream protocol is deliberately simple: freeze the SensorFM encoder, aggregate embeddings per person, reduce to 50 PCA components, and train a light linear head. Under this setup, SensorFM embeddings outperform supervised baselines trained on engineered wearable features on 34/35 discriminative tasks, and outperform demographics-only baselines on 24/30 discriminative tasks. With demographics included, SensorFM-B shows the smallest marginal gain from demographics on 33/35 tasks, which the paper interprets as evidence that large-scale pretraining learns physiologically relevant priors implicitly.

Few-shot behavior is presented as a second manifestation of transfer. With very little labeled data, demographics can be competitive; as more labeled data becomes available, SensorFM overtakes demographic-only models, and the larger variants outperform smaller variants throughout [2605.22759]. The paper therefore frames the model as label-efficient rather than label-free. That distinction is important: SensorFM reduces the amount of supervision needed, but it is not presented as a zero-shot clinical predictor.

The paper also analyzes the latent space. UMAP is reported to show physiologically meaningful structure, embeddings reflect demographic shifts such as BMI, age, and gender, and SHAP analysis suggests that tasks share latent dimensions when they are clinically related [2605.22759]. Larger models are described as having more expressive embeddings with a broad long tail of information, while smaller models show signs of dimensional collapse.

## 5. Generative reconstruction, automated head search, and conversational grounding

Because SensorFM is pretrained as a masked reconstructive model, it supports generative infilling, interpolation, extrapolation, and sensor signal imputation [2605.22759]. Reported gains over the best baselines are 74.8% better on random imputation, 38.8% better on temporal interpolation, 39.6% better on temporal extrapolation, and 83.7% better on sensor signal imputation. The paper links these generative capabilities to robust daily metric estimation under missing data. When a 60-minute contiguous data gap is simulated, the reconstructed interval preserves 99.7% accuracy for daily step count, 99.9% accuracy for deep sleep prediction, and 99.2% accuracy for light exercise tracking.

A further layer is the LLM-driven “classroom” for downstream predictive head search. The framework takes frozen SensorFM embeddings plus demographics and labels, uses parallel LLM agents to propose executable Python solutions, and iteratively generates, tests, critiques, and refines models over 30,000 experiments [2605.22759]. Compared with a plain linear probe, classroom-found solutions improve performance on 16/20 classification tasks and 12/15 regression tasks; across all tasks, the search improves on 29/35 tasks. The best solutions often reduce embeddings to about 50–100 dimensions, use mostly linear models, and sometimes use ensembles. The paper reports that stronger LLMs produce better downstream solutions and that improvement scales with model capability.

SensorFM is also used as a tool within a Personal Health Agent (PHA). Three conditions are compared: demographics + daily wearable metrics + SensorFM predictions; demographics + daily wearable metrics + ground-truth labels; and demographics + daily wearable metrics only [2605.22759]. The system is evaluated on real patient profiles by four board-certified clinicians using 1,860 ratings. Relative to the baseline without extra context, both SensorFM predictions and ground-truth labels significantly improve context, personalization, justifiability, relevance, and harm. Crucially, SensorFM predictions are reported as not statistically inferior to ground truth in clinician ratings, with \(p = 0.396\). In the paper’s framing, this shows that SensorFM is useful not only as a predictive representation but also as a grounding layer for downstream health dialogue.

## 6. Broader SensorFM-style literature, terminological ambiguity, and limitations

In the wider sensing literature supplied here, SensorFM also functions as a broader conceptual label for pretrained, sensor-native backbones that are adapted across tasks rather than rebuilt from scratch [2602.11200]. AM-FM extends that logic to WiFi sensing through Channel State Information, pretraining on 9,206,316 unlabeled CSI samples collected over 439 days from 20 commercial device types and evaluating on nine downstream tasks including fall detection, human activity recognition, localization, occupancy detection, and WiFi imaging [2602.11200]. DomusFM applies a comparable foundation-model strategy to smart-home binary sensor events through self-supervised dual contrastive learning, with leave-one-dataset-out evaluation across seven public datasets and strong performance even with only 5% labeled training data available for fine-tuning [2602.01910]. FM-Fi, by contrast, uses cross-modal contrastive knowledge distillation from CLIP to RF point-cloud sensing, achieving 72.5% zero-shot accuracy and 94.4% 3-shot accuracy on a 10-class HAR task [2410.19766].

These systems share a family resemblance, but they operate on fundamentally different sensing substrates: minute-resolution wearable aggregates in SensorFM, CSI tensors in AM-FM, sparse event tuples in DomusFM, and mmWave point clouds in FM-Fi [2605.22759][2602.11200][2602.01910][2410.19766]. In that sense, “SensorFM” is both a particular wearable-health model and, in parts of the recent literature, a more general research direction centered on pretrained sensing representations.

The term is not fully standardized. In a different line of work on integrated sensing and communication, “SensorFM” is used in a waveform-centric sense: a matched, FM-like sensing waveform framework for 6G ISAC employing dedicated sensing reference signals, time-frequency matched windows, and DNLFM to achieve constant modulus and low sidelobes [2309.09415]. This suggests that the label currently spans at least two meanings: a foundation-model family in sensor representation learning and an FM-like sensing waveform framework in ISAC.

Several boundaries follow directly from the supplied evidence. SensorFM is not a task-specific supervised wearable predictor; it is a reusable latent model whose downstream heads may remain simple [2605.22759]. It is also not a purely LLM-based wearable analytics system, since its representation is learned from sensor masking and reconstruction rather than prompting. Finally, the broader SensorFM-style literature shows that foundation-model transfer is domain-specific: smart-home events, wearable summaries, CSI, and RF point clouds require distinct inductive biases, preprocessing pipelines, and adaptation mechanisms [2602.01910][2602.11200][2410.19766]. A plausible implication is that “SensorFM” names a methodological orientation toward sensor-native pretraining more than a single universally applicable architecture.

Source: https://www.emergentmind.com/topics/sensorfm