---
title: 'TolerantECG: Imperfect ECG Pretraining'
url: https://www.emergentmind.com/topics/tolerantecg
type: topic
---

# TolerantECG: Imperfect ECG Pretraining

Searching arXiv for the specified papers to ground the article in current records.
{"query":"id:2507.09887 OR \"TolerantECG: A Foundation Model for Imperfect Electrocardiogram\"","max_results":5,"sort_by":"submittedDate","sort_order":"descending"}
{"query":"id:2405.00724 OR \"Baseline Drift Tolerant Signal Encoding for ECG Classification with Deep Learning\"","max_results":5,"sort_by":"submittedDate","sort_order":"descending"}
{"query":"id:2506.16494 OR \"Manifold Learning for Personalized and Label-Free Detection of Cardiac Arrhythmias\"","max_results":5,"sort_by":"submittedDate","sort_order":"descending"}
TolerantECG is a foundation model for electrocardiogram representation learning designed for imperfect ECG acquisition, specifically settings in which recordings are corrupted by noise or contain arbitrary subsets of the standard 12 leads. It was introduced as a multimodal and self-supervised pretraining framework that jointly learns ECG representations from waveform data, knowledge-retrieval-enriched text report descriptions, and corrupted or lead-missing signal views, with the stated goal of maintaining utility under realistic degradation rather than assuming clean, fully observed 12-lead input [2507.09887].

## 1. Concept and problem setting

TolerantECG addresses two practical failure modes of automated ECG analysis. The first is noise corruption, including body movement, poor electrode contact, respiration, baseline wander, muscle artifacts, and electrode motion artifacts. The second is lead unavailability: many deployed devices, such as smartwatches and Holter monitors, do not provide all 12 standard leads. In that setting, a model optimized only for complete and clean ECGs can become brittle, either because waveform morphology is distorted or because cardiac information is only partially observed [2507.09887].

The model is therefore framed as a foundation model for imperfect electrocardiograms. In the source formulation, “imperfect” refers specifically to noisy signals, lead-missing signals, and combinations of the two. The central thesis is that ECG pretraining should encode tolerance to these conditions directly, rather than delegating robustness to a separate denoising stage or restricting deployment to complete 12-lead acquisitions [2507.09887].

A useful interpretive distinction is between tolerance at the representation level and tolerance at the preprocessing level. TolerantECG belongs to the former category: it aims to make the learned embedding stable under clinically common degradations. This suggests a deployment model in which robustness is built into feature learning itself rather than appended as an external correction module.

## 2. Pretraining formulation and architectural components

TolerantECG combines two pretraining components. The first is **Report Alignment (ReportAlign)**, a contrastive multimodal objective aligning ECG signals with detailed report descriptions. The second is **DuoDistill**, a self-supervised distillation module specialized for imperfect ECGs through separate treatment of lead-missing and noise-corrupted inputs [2507.09887].

These components are combined through the total objective
$$
\mathcal{L}_{\text{Total}}=\alpha \mathcal{L}_{\text{ReportAlign}}+\beta \mathcal{L}_{\text{DuoDistill}},
$$
with
$$
\alpha=\beta=1.
$$

In ReportAlign, the ECG encoder is a modified **1D ConvNeXt V2** in which 2D convolutions are replaced by 1D convolutions, and the final feature map is average-pooled over time to obtain the ECG embedding. The text encoder is **BioLinkBERT**, initialized from pretrained biomedical language weights, with the final-layer **[CLS]** token used as the text embedding. The ECG encoder is randomly initialized, whereas the text encoder is pretrained and updated during pretraining [2507.09887].

The ReportAlign objective is CLIP-style and symmetric. For batch size \(B\), projected ECG embeddings \(S_i\), projected text embeddings \(T_i\), and temperature \(\tau\), the loss is
$$
\mathcal{L}_{\text{ReportAlign}}
=
-\frac{1}{2}
\left(
\sum_{i=1}^{B}
\log
\frac{\exp(S_i^\top T_i/\tau)}
{\sum_{j=1}^{B}\exp(S_i^\top T_j/\tau)}
+
\sum_{i=1}^{B}
\log
\frac{\exp(T_i^\top S_i/\tau)}
{\sum_{j=1}^{B}\exp(T_i^\top S_j/\tau)}
\right).
$$

The intended effect is bidirectional alignment: each ECG embedding should be closer to its paired report than to other reports in the batch, and each report embedding should be closer to its paired ECG than to other ECGs. The resulting representation is described as encoding waveform structure together with clinical semantics [2507.09887].

## 3. Knowledge-retrieval text supervision

A distinctive element of TolerantECG is **Cardio Feature Retrieval (CFR)**, which enriches ECG-text supervision without using a generic LLM-based retrieval-augmented generation pipeline. CFR is a lightweight knowledge-retrieval system that converts terse diagnostic text into more detailed report descriptions by appending waveform criteria and diagnostic feature descriptions [2507.09887].

The CFR database is organized as a dictionary-like structure in which keys are cardiac diagnoses and values are associated waveform criteria or diagnostic feature descriptions. The diagnosis-feature knowledge is scraped from **Life in the Fastlane (LITFL)**. Diagnosis names are embedded using **all-MiniLM-L6-v2**, and the resulting embeddings are stored in a vector database implemented with **Chroma** [2507.09887].

For each ECG case, the pipeline expands SCP-style diagnostic labels or statements into full terms, embeds each expanded diagnosis, computes cosine similarity against the database keys, selects the most similar diagnosis above a threshold, and appends the associated waveform description to form a detailed report description. The stated motivation is that raw ECG reports can be too terse or too semantically similar across patients, which weakens contrastive learning. CFR is intended to make the text target richer in morphology-level and clinical information [2507.09887].

The paper reports that **CFR improves ReportAlign**, especially for lead-missing settings. A plausible implication is that text supervision becomes more useful when waveform evidence is incomplete, because the enriched description supplies structure that the partially observed signal does not fully expose.

## 4. DuoDistill and corruption-tolerant representation learning

DuoDistill is the self-supervised component responsible for explicit corruption tolerance. It is inspired by DINO-style self-distillation, but adapted to ECG-specific imperfections and implemented with two teacher branches rather than one: a lead-missing teacher and a noise teacher. The student receives more degraded views, while the teacher receives less degraded views, and the student is trained to match the teacher’s representations [2507.09887].

For lead-missing training, the signal is corrupted by masking leads. The formulation defines \(N\) major masks and \(K\) minor masks, with \(N=8\) and \(K=2\) in the experiments. A **Major Mask** keeps only 1 to 6 leads; a **Minor Mask** keeps 6 to 12 leads; masked leads are zeroed out. The student receives all \(N+K\) masked variants, whereas the teacher receives only the \(K\) minor-mask versions, i.e. the less severely damaged signals [2507.09887].

For noise robustness, noise is taken from the **MIT-BIH Noise Stress Test Database**, specifically baseline wander, muscle artifact, and electrode motion artifact. Noise is resampled to **500 Hz**, a random **10-second segment** is extracted, each noise type is applied with probability **0.7**, noise is added to each lead with probability **0.5**, and the signal-to-noise ratio is sampled uniformly from **-10 dB to 0 dB**. The student sees all major and minor noisy versions, while the teacher sees only the minor noisy versions. Minor variants include the original signal, filtered variants using high-pass and low-pass filters, and, if additional minor variants are needed, high-SNR noisy examples [2507.09887].

The DuoDistill objective is given as
$$
\mathcal{L}_{\text{DuoDistill}}
=
\mathbb{E}
\left[
\sum_{i=1}^{K}
\sum_{\substack{j=1 \\ j\neq i}}^{K+N}
\sum_{d=1}^{D}
-
\operatorname{softmax}\left(\frac{\mathbf{q}_d^i-\mathbf{c}_d}{\tau_t}\right)
\cdot
\log
\left(
\operatorname{softmax}\left(\frac{\mathbf{p}_d^j}{\tau_s}\right)
\right)
\right],
$$
where \(\mathbf{q}_d^i\) denotes teacher logits, \(\mathbf{p}_d^j\) student logits, \(\mathbf{c}_d\) the teacher centering term, \(\tau_t\) and \(\tau_s\) the teacher and student temperatures, \(K\) the number of minor views, \(N\) the number of major views, and \(D\) the logit dimension [2507.09887].

A key methodological detail is the alternating “ping-pong” training schedule. On even steps, the model trains on lead-missing corrupted inputs, computes masked-branch distillation loss plus contrastive loss, updates the student, and updates the lead-missing teacher. On odd steps, it trains on noise-corrupted inputs, computes noisy-branch distillation loss plus contrastive loss, updates the student, and updates the noise teacher. The rationale given is that simultaneous EMA-style updating could make the two teachers too similar and reduce specialization; the alternating schedule is intended to preserve distinct expertise for the two imperfection modes [2507.09887].

## 5. Pretraining corpus, benchmarks, and empirical behavior

TolerantECG is pretrained on **MIMIC-IV-ECG**, comprising about **800,000** 12-lead ECGs from nearly **160,000** unique patients. Each record is **10 seconds** long, sampled at **500 Hz**, and accompanied by demographic information and machine-generated reports. All-zero and NaN-containing signals are removed. Training uses **AdamW**, a learning rate of \(3\times10^{-5}\), **10** epochs, **4 NVIDIA A100 80GB GPUs**, and batch size **32 per GPU** [2507.09887].

For downstream evaluation, the pretrained ECG encoder is transferred by attaching a linear classifier head and fine-tuning the whole network. The main benchmark is **PTB-XL**, which contains **21,837** records from **18,885** patients and provides 10-second 12-lead ECGs with multi-label clinical annotations. Evaluation is carried out at six class levels: **Super-Diag**, **Sub-Diag**, **Diag**, **Rhythm**, **Form**, and **All**, corresponding respectively to 5, 23, 44, 12, 19, and 71 labels or statements [2507.09887].

Because no public imperfect-ECG benchmark is stated to exist, the work constructs four PTB-XL conditions: **Original**; **Lead-missing**, where each lead is zeroed with probability \(p_{\text{mask}}=0.5\) and at least one lead remains; **Noisy**, where noise is added with probability \(p_{\text{noise}}=0.5\), noise types come from MIT-BIH NSTDB, and SNR ranges from \(-10\) dB to \(0\) dB; and **Lead-missing + Noisy**, where noise is applied before lead masking [2507.09887].

A second benchmark is the **MIT-BIH Arrhythmia Database**, used to test robustness in a limited-lead setting. It consists of **48** half-hour recordings with only **two leads** from a subset of possible leads. Recordings are segmented into **10-second clips**, resampled to **500 Hz**, and the beat-based labels are mapped into **5 classes**: Normal, LBBB, RBBB, PAC, and PVC. The train/validation/test split is **70% / 10% / 20%** [2507.09887].

The model is compared against **SimCLR**, **BYOL**, **CPC**, **METS**, **MERL**, and **ECG-FM**. Metrics are **AP** and **AUC**. The reported empirical summary is that TolerantECG consistently ranks as the **best or second-best performer across various ECG signal conditions and class levels in the PTB-XL dataset**, and achieves the **highest performance on the MIT-BIH Arrhythmia Database**. More specifically, it is described as strongest overall on **Noisy** PTB-XL conditions, closely competitive with **ECG-FM** on **Lead-missing**, and among the best, often the best, on **Lead-missing + Noisy**. On MIT-BIH, it is reported to achieve the best AP and AUC in both the original two-lead setting and the two-lead-plus-noise setting [2507.09887].

The ablation analyses identify several trends. As the number of visible leads increases, all models improve, but TolerantECG remains strongest across lead counts, especially with only **1–4 leads**. As SNR decreases, all models degrade, but TolerantECG degrades the least and maintains the best AUC across noise levels. The component studies report that **DuoDistill** contributes more to robustness than **ReportAlign** alone, that **CFR** improves **ReportAlign**, especially for lead-missing settings, that **DuoDistill** outperforms **UniDistill**, and that the full TolerantECG model is best overall under all conditions [2507.09887].

## 6. Relation to robust ECG research, scope, and interpretation

TolerantECG belongs to a broader line of ECG research concerned with robustness to nuisance variation, but its strategy differs from both handcrafted signal encoding and purely unsupervised manifold analysis. A useful comparison is **Derived Peak (DP) encoding**, which was proposed as a non-parametric event-driven representation based on zero crossings of ECG derivatives. That method was designed to be invariant to shift and positive rescaling, and it maintained strong AUC under drift, shift, and rescaling on PTB-XL, with the core argument that derivative zero crossings encode waveform geometry rather than absolute voltage [2405.00724]. TolerantECG addresses a partially overlapping problem space—imperfect signals—but does so at the level of large-scale representation learning rather than encoding-specific invariance. This suggests a distinction between signal-level robustness mechanisms and representation-level robustness mechanisms.

A second adjacent direction is personalized and label-free ECG analysis via nonlinear dimensionality reduction. In that work, t-SNE and UMAP were used on MIT-BIH heartbeat waveforms and were reported to discriminate individual recordings in mixed populations with at least **90%** accuracy while achieving a median personalized arrhythmia classification accuracy of **98.96%** and a median F1-score of **91.02%** without requiring labels during embedding construction [2506.16494]. That line of work emphasizes tolerance to inter-patient variation, lead variability, and label scarcity. TolerantECG addresses a different axis—noise and lead missingness—but the two are conceptually aligned in treating robustness as a primary modeling objective rather than a secondary refinement.

Within its own stated scope, TolerantECG should not be reduced to a denoiser, a lead-imputation system, or a conventional supervised classifier. Its defining contribution is joint pretraining from ECG waveforms, enriched text descriptions, and corruption-specific self-distillation. The model is also not restricted to the assumption of full 12-lead acquisition, since arbitrary subsets of the standard 12-lead ECG are part of the training design [2507.09887].

The practical significance claimed for the model concerns **wearables**, **Holter monitors**, and **limited-lead devices**, together with scenarios in which completeness and quality cannot be guaranteed. A plausible implication is that TolerantECG is best understood as a clinically oriented foundation encoder optimized for imperfect acquisition regimes, rather than as a generic ECG pretraining method whose robustness emerges only incidentally.

Source: https://www.emergentmind.com/topics/tolerantecg