QualityFM: Multimodal Signal Model

• QualityFM is a multimodal foundation model that uses paired high- and low-quality ECG and PPG signals to learn robust representations through dual-track encoders and self-distillation.
• It employs a windowed sparse self-attention mechanism and composite spectral-domain losses to efficiently capture both local waveform morphology and global rhythmic structures.
• Pre-trained on over 21 million waveforms, QualityFM enhances clinical tasks such as false alarm reduction, arrhythmia detection, and non-invasive blood pressure estimation.

QualityFM refers to a multimodal foundation model architecture designed for the physiological signal domain, targeting signal quality challenges in critically ill patient settings. Unlike prior approaches reliant on extensive labeling or single-modal learning, QualityFM leverages paired electrocardiogram (ECG) and photoplethysmogram (PPG) waveform data from large-scale hospital records. The model adopts a dual-track encoder scheme with self-distillation and composite spectral-domain supervision, enabling robust representation learning across variable signal quality scenarios. It is evaluated on substantial data (21 million waveforms, totaling ~180,000 hours), demonstrating effective transfer learning to clinically vital tasks such as false alarm reduction, arrhythmia identification, and blood pressure estimation.

1. Dual-Track Architecture and Input Construction

QualityFM processes independently curated paired physiological signals of differing quality—one high-quality, one low-quality—through parallel encoder tracks. Formally, for a pair {(Xᵢ, Lᵢ), (Xⱼ, Lⱼ)} where L denotes a quality score and Lᵢ > Lⱼ, encoders with parameters θₜ (teacher for high-quality) and θₛ (student for low-quality) generate feature representations:

• High-quality encoder output: Uₜ = E₍θₜ₎(Xᵢ)
• Low-quality encoder output: Uₛ = E₍θₛ₎(Xⱼ)

A decoder (parameter-tying with encoder) reconstructs frequency-based spectral features (amplitude and phase) from Uₛ. This paired approach is essential as it operationalizes supervision for signal quality—rarely available at scale—by mapping high-quality features onto noisy, artifact-laden low-quality contexts.

2. Self-Distillation Mechanism

QualityFM employs a self-distillation paradigm in which the high-quality encoder (“teacher”) guides the low-quality encoder (“student”). Distillation loss aligns their output distributions. For embedding dimension m, student and teacher outputs are converted to probability distributions using softmax at temperature τₛ or τₜ:

$$P_s(x)^{(m)} = \frac{\exp(E_{θ_s}(X_j)^{(m)} / \tau_s)}{\sum_m \exp(E_{θ_s}(X_j)^{(m)} / \tau_s)}$$

$$P_t(x)^{(m)} = \frac{\exp(E_{θ_t}(X_i)^{(m)} / \tau_t)}{\sum_m \exp(E_{θ_t}(X_i)^{(m)} / \tau_t)}$$

Direct distillation loss:

$$\mathcal{L}_{dis} = -\sum_{(i,j)} \sum_m P_t(X_i)^{(m)}\log(P_s(X_j)^{(m)})$$

Critically, while θₛ is updated through backpropagation, θₜ is a slow exponential moving average with rate λ:

$θ_t \gets λθ_t + (1 - λ)θ_s$

This ensures that the teacher remains a denoised, temporally-stable supervision signal, as opposed to simply copying the student’s weights, which would nullify the distillation effect.

3. Windowed Sparse Attention for Long Sequential Signals

To efficiently process long, quasi-periodic physiological waveforms, QualityFM integrates a windowed sparse self-attention mechanism within its Transformer backbone. Rather than global attention (quadratic cost, O(n²)), attention weights are computed locally within a sliding window of fixed width w, resulting in O(n·w) complexity where n is the sequence length.

Layer normalization (LN) is applied to queries and keys in the attention computation:

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{\text{LN}(Q)\text{LN}(K)^T}{\sqrt{d_{head}}}\right)V$$

This design prevents uncontrolled growth of attention logits and ensures stable training. Early layers have narrow receptive fields, capturing local morphology, while stacked layers expand context, permitting learning of global rhythmic structure endemic in monitoring signals.

4. Composite Spectral-Domain Loss Formulation

QualityFM’s loss combines direct distillation (as above) with indirect spectral reconstruction losses. For signal xᵢ(n), the DFT yields Xᵢ[k]:

$X_i[k] = \sum_{n=0}^{N-1} x_i(n) e^{-j2\pi kn/N}$

Amplitude and phase are extracted:

$$A_i[k] = \sqrt{\text{Re}(X_i[k])^2 + \text{Im}(X_i[k])^2}$$

$$\Phi_i[k] = \arctan(\text{Im}(X_i[k]), \text{Re}(X_i[k]))$$

A feedforward decoder reconstructs amplitude (ˆAⱼ) and phase (ˆΦⱼ) from Uₛ. MSE losses are computed per batch:

• Amplitude: ||ˆAⱼ – Aᵢ||²
• Phase: ||ˆΦⱼ – Φᵢ||²

Full pre-training loss aggregates:

$$\mathcal{L}_{pre} = \mathcal{L}_{dis} + \lambda_{Amp}\sum_{(i,j)} ||\hat{A}_j - A_i||^2 + \lambda_{Pha}\sum_{(i,j)} ||\hat{\Phi}_j - \Phi_i||^2$$

The spectral losses enforce preservation of essential cardiac and vascular waveform characteristics critical for downstream biomedical inference.

5. Large-Scale Pre-training and Transfer Learning Efficacy

QualityFM is pre-trained on 21,287,295 waveforms (each of 30 seconds) from multi-hospital clinical repositories, covering 179,757 hours and encompassing diverse artifact presence, morphology, and patient states. Three scale variants are trained: base (9.6 M params), large (70 M), and huge (319 M). Post pre-training, the model is transferred to three clinical tasks:

• Ventricular tachycardia false alarm detection
• Atrial fibrillation identification
• Arterial blood pressure estimation from PPG/ECG

In each task, initializing with QualityFM’s pre-trained weights yields substantial improvements in classification/regression accuracy, raising the model’s practical value for ICU/OR deployment scenarios plagued by persistent signal quality variability.

6. Clinical Impact and Signal Quality Handling

QualityFM directly addresses pervasive issues in biomedical signal monitoring:

• Reduces false alarms (e.g., in ventricular tachycardia detection) by generating robust, quality-aware representations
• Improves detection of complex arrhythmias (AF), capturing waveform irregularities via sparse attention
• Enhances non-invasive blood pressure estimation, leveraging frequency-domain constraints for physiologically consistent measurement

The combination of self-distillation, frequency-aware reconstruction, and local/global attention allows QualityFM to correct for, or be resilient to, missing data portions, noise, and inconsistent acquisition conditions—a primary bottleneck in real-world critical care data.

7. Research Significance and Future Directions

QualityFM integrates architectural innovations (dual-track encoders, sparse attention), self-supervised learning (self-distillation), and a composite spectral-domain objective, yielding a versatile multimodal backbone for physiological signal quality representation. The approach’s scalability, cross-task generalizability, and demonstrated real-world performance establish a foundation for subsequent methods in cross-sensor, cross-modal, and cross-population signal quality modeling. Further research directions include adaptation to additional modalities (e.g., capnography, EEG), refinement of attention mechanisms for extreme sequence lengths, and exploration of task-specific fine-tuning strategies tailored for resource-constrained clinical hardware environments.