Nutrition PPG Language Model (NPLM)

• Nutrition Photoplethysmography Language Model (NPLM) is a multimodal framework that fuses PPG data and meal descriptions to estimate caloric intake and satiety.
• It employs a modular pipeline with a pre-trained 1-D CNN for PPG encoding followed by a frozen GPT-2 for text fusion, ensuring precise physiological-text alignment.
• Performance evaluations reveal that NPLM significantly outperforms text-only baselines, maintaining high accuracy even with missing meal description tokens.

The Nutrition Photoplethysmography LLM (NPLM) is a multimodal architecture for noninvasive, scalable estimation of caloric intake and satiety by integrating continuous photoplethysmography (PPG) measurements from consumer wearables and natural language meal descriptions. NPLM leverages PPG-derived physiological embeddings as input to a frozen LLM (GPT-2), enabling joint modeling of an individual’s real-time physiological state and contextual meal information. Trained on an extensive corpus of meal-PPG pairs, NPLM demonstrates robust performance in free-living and controlled environments, outperforming unimodal text-based baselines, particularly under missing or sparse meal description conditions. The model provides new methodological routes for large-scale dietary monitoring and research at the intersection of wearable sensing and computational nutrition science (Verrier et al., 24 Nov 2025).

1. Model Architecture

NPLM is structured around a modular encoding–fusion pipeline:

• PPG Encoder: Input comprises 60-second green-light PPG waveform segments sampled at either 64 Hz or 256 Hz from wrist-worn Apple Watches during low-motion periods. These segments are processed by a pre-trained 1-D convolutional neural network (as described by Abbaspourazad et al., 2024), generating a 256-dimensional PPG embedding $z = E_{PPG}(x_{PPG}) \in \mathbb{R}^{256}$.
• Linear Adapter (Embedding Projection): The PPG embedding $z$ is mapped into $K$ “prefix” token embeddings compatible with GPT-2’s token space, using

$$f_\theta: \mathbb{R}^{256} \rightarrow \mathbb{R}^{K \times d},$$

where $d$ is the GPT-2 token dimension (e.g., 768). Each prefix token is calculated as $[f_\theta(z)]_i = W_i z + b_i$, $i = 1 \ldots K$ with learnable parameters $W_i \in \mathbb{R}^{d \times 256}$ and $b_i \in \mathbb{R}^d$. All other parameters in GPT-2 remain frozen.

• Text Encoder and Fusion: The K prefix tokens are prepended to the tokenized meal description $x_{text}$, forming a combined sequence that traverses all GPT-2 transformer blocks.
• Satiety/Intake Head: The output hidden states $H \in \mathbb{R}^{(K+L) \times d}$ undergo max pooling across the time axis, yielding $h^* \in \mathbb{R}^d$. For downstream classification (e.g., above-personal-mean caloric intake or decreased fullness), a linear/logistic head maps $h^*$ to $\hat{y} = \sigma(w^\top h^* + b)$.

The architecture freeze strategy anchors general linguistic knowledge while enabling the multimodal adapter to specialize over physiological–text alignment.

2. Mathematical Formalism

Core mathematical definitions and learning objectives:

• Modality Embeddings
  • $z = E_{PPG}(x_{PPG})$, $h = E_{text}(x_{text})$
• Conditional Likelihood and Alignment Objective
  • The alignment objective models the likelihood of text given PPG:

  $$p_\theta(x_{text} \mid x_{PPG}) = \prod_{i=1}^L q(x_{text}^{[i]} \mid [f_\theta(z), x_{text}^{[<i]}])$$ - Training maximizes log-likelihood across pairs $(x_{PPG}, x_{text})$ in dataset $D$:

  $$\theta^* = \arg \max_{\theta} \sum_{(x_{PPG}, x_{text}) \in D} \log p_\theta(x_{text} \mid x_{PPG})$$

  Equivalently, the alignment loss is:

  $$\mathcal{L}_{alignment} = -\log p_\theta(x_{text} \mid x_{PPG})$$

• Downstream Classification Loss

  • For supervised tasks (intake, satiety): binary cross-entropy

  $$\mathcal{L}_{satiety} = -[y \log \hat{y} + (1-y)\log(1-\hat{y})]$$

• Alignment Win Rate (for analysis rather than training)

  • WinRate($w$) quantifies the probability that $p(x_{text}|x_{PPG,w})$ exceeds the cohort-shuffled likelihood, tracking how meal–PPG temporal correspondence degrades with time shift.

3. Datasets and Training Procedures

NPLM’s empirical foundation draws on two principal cohorts:

Dataset	Size	Features
AHMS Cohort	19,340 participants	1,122,834 self-reported meal logs (free-text, calories, macronutrients)
Validation Study	140 participants	720 controlled lunch meals with app-logged descriptions and post-meal satiety ratings

• Preprocessing: Meals <50 kcal and >2,500 kcal are excluded, along with non-informative entries (“Skipped”, “Nothing”) and missing-calorie or duplicate records. Inclusion requires logging ≥5 days with at least one meal per day.
• PPG/Meal Pairing: For any meal at $t_{meal}$, the nearest PPG window within $w_{0-4h} = [t_{meal}-4h, t_{meal}+4h]$ is selected.
• Training: Only parameters in $f_\theta$ are updated. Adam optimizer is used for 10,000 steps (learning rate $3 \times 10^{-4}$, batch size 64), with early stopping on a held-out fraction (10%) of the training data. GPT-2 and PPG encoder weights are frozen.
• Splitting: 80%/20% participant-level split for training/testing in AHMS; 10% of training held out for validation. The controlled Validation Study is reserved for out-of-sample evaluation.

4. Performance and Robustness

NPLM exhibits substantial improvements relative to unimodal architectures in diverse task settings:

• Daily Energy Intake Prediction (AHMS)
  • NPLM: AUC = 0.82 (95% CI: 0.814–0.826)
  • Text-only baseline: AUC = 0.74 (95% CI: 0.732–0.748)
• Postprandial Satiety Prediction (Validation Study)
  • NPLM: AUC = 0.71 (95% CI: 0.70–0.72)
  • Text-only: AUC = 0.64 (95% CI: 0.63–0.65)
  • Relative improvement ≈11%.
• Missing Text Robustness
  • When 25%, 50%, 75%, or 100% of meal description tokens are dropped at random, text-only models deteriorate sharply, whereas NPLM retains high discriminative accuracy.
  • With 50% of tokens missing, NPLM still surpasses the text-only full-baseline. Even when LLM-generated single-word summaries (~22% of the original token length) are used, NPLM attains AUC = 0.75 (95% CI: 0.74–0.76), remaining above the comprehensive-text baseline.
• Alignment Analysis
  • Alignment win rate for PPG–meal pairs in the $[−4, +4]$ h window is 0.79 (95% CI: 0.78–0.80).
  • Cohort-shuffled (negative control): 0.43; within-subject shuffle: 0.69.
  • Alignment performance declines monotonically with increasing PPG–meal separation, consistent with physiological time-course.

5. Physiological Mechanisms and Embedding Interpretation

PPG is sensitive to microvascular blood-volume pulsatility and pulse wave dynamics, which vary in response to meal ingestion and macronutrient load. PPG-derived embeddings encode aspects of:

• Cardiovascular reactivity to feeding events
• Endogenous metabolic and circadian signals
• Behavioral state context (activity, sleep), subtly present in the waveform morphology

A surrogate regression of NPLM satiety scores on meal macronutrient content reveals positive coefficients for fat, protein, fiber, and carbohydrates—matching established determinants of satiety. This suggests the multimodal embedding synergistically models individual variation in postprandial vascular responses and their modulation by meal type, supporting personalized, context-aware hunger/satiety estimation.

6. Limitations, Biases, and Future Development

Key limitations of NPLM and its evaluation pipeline include:

• Data Quality: Reliance on self-reported meal logs and subjective fullness assessments introduces measurement error and recall bias.
• Cohort Generalizability: AHMS data is constrained to U.S. Apple Watch users, and the controlled Validation Study comprises midday weekday diners, limiting extrapolation across geographic, socioeconomic, and circadian contexts.
• Pairing Precision: The wide ±4 h PPG/meal window admits possible confounds, such as unlogged snacks, physical activity, or psychological stress.
• Unmeasured Variables: No direct incorporation of non-caloric intake, specific medications (including GLP-1 agonists), or CGM (continuous glucose monitoring) data.
• Causal Inference: The observational design precludes formal causal analysis or assessment of real-time dietary feedback.

Potential biases arise from the likelihood that participants logging meals have increased health consciousness, and disparities between cohorts preclude subgroup or intersectional inference.

Anticipated extensions for NPLM include: randomized interventional trials utilizing real-time PPG-derived feedback, integration with additional noninvasive biosensors (e.g., accelerometry, skin temperature), fine-grained temporal alignment (anchored to postprandial biomarker timecourses), and deployment studies focusing on usability, user compliance, and real-world health impact.

7. Implications for Digital Nutrition Monitoring

NPLM presents evidence that wrist-worn PPG, in combination with even minimal meal text, suffices for reliable day-to-day caloric and satiety monitoring in unrestricted living environments. It tolerates missing or abbreviated meal descriptions, facilitating scalable, low-burden dietary tracking suitable for digital health applications. The model exemplifies the translation of physiological sensing into LLMs for real-world, person-level nutritional monitoring, without recourse to invasive procedures or comprehensive self-report (Verrier et al., 24 Nov 2025).