Electrocardiogram-Language Models (ELMs)

• Electrocardiogram-Language Models (ELMs) are a research framework that couples ECG signal tokenization and continuous embeddings with language-processing techniques.
• They encompass diverse paradigms including multimodal generation, domain-adapted LLMs, and self-supervised ECG tokenization to handle clinical signals.
• ELMs leverage varied methodologies—from vector quantization and segmentation to retrieval-augmented generation—ensuring precise, interpretable clinical reasoning.

Electrocardiogram-LLMs (ELMs) are a heterogeneous class of models that couple electrocardiographic signals, electrocardiographic knowledge, or both with language modeling, بحيث ECG data can be queried, summarized, classified, retrieved, or forecast in natural language. The term is used across at least three partially overlapping paradigms: multimodal systems that condition text generation on ECG signals, domain-specific LLMs adapted to electrocardiography, and self-supervised models that treat cardiac waveforms or rhythms as language-like token sequences (Tang et al., 2024, Ahrens et al., 21 Oct 2025, Wang et al., 26 Feb 2026). This terminological breadth is not incidental: it reflects a research area in which “language” may denote textual clinical reasoning, symbolic ECG tokenization, or joint ECG–text representation learning.

1. Conceptual scope and historical formation

The intellectual basis for ELMs predates current multimodal LLM practice. “ECG Language Processing (ELP)” explicitly framed an ECG record as a sequence of heartbeats analogous to sentences, with P-wave, QRS complex, T-wave, and U-wave morphologies serving as word-like units; that work clustered wave snippets into a vocabulary, embedded token sequences, and modeled them with CNNs or Bi-LSTM architectures for heartbeat classification and atrial fibrillation detection (Mousavi et al., 2020). Subsequent work extended the language analogy from symbolic sequence modeling to multimodal learning between ECGs and reports, including retrieval systems that aligned ECG images with clinical text through ViT–BERT-style architectures (Qiu et al., 2023).

A later generation of work consolidated the modern ELM label but did not converge on a single canonical architecture. Some papers define ELMs as multimodal foundation models that jointly represent raw ECG waveforms and generate natural-language interpretations or reports, as in few-shot ECG question answering and long-context forecasting (Tang et al., 2024, Velingker et al., 17 Feb 2026). Others use the term for domain-adapted LLMs specialized for electrocardiography literature and clinical knowledge rather than direct signal ingestion (Ahrens et al., 21 Oct 2025). A parallel line treats ECG as a structured language for self-supervised learning, with rhythm-aware tokenization and masked prediction objectives (Wang et al., 26 Feb 2026).

Family	Representative papers	Core formulation
Signal-as-language SSL	(Mousavi et al., 2020, Jin et al., 15 Feb 2025, Wang et al., 26 Feb 2026)	ECG waves, beats, or rhythms are tokenized and modeled with language-style objectives
Multimodal ECG-conditioned generation	(Tang et al., 2024, Wan et al., 2024, Velingker et al., 17 Feb 2026)	ECG embeddings or ECG tokens are fused with an LLM to answer questions or generate reports
Domain-adapted and retrieval-grounded ECG LLMs	(Ahrens et al., 21 Oct 2025, Yu et al., 30 Apr 2025, Song et al., 30 Sep 2025)	Textual ECG knowledge, rules, or retrieved documents ground ECG-oriented reasoning

This diversity suggests that ELM is best understood as a research program rather than a single model family. A plausible implication is that comparisons across papers require attention to what is being aligned: waveform-to-language, report-to-language, or cardiology text-to-language.

2. Representation strategies

Representation design is the defining architectural fault line in ELM research. One branch uses explicit symbolic tokenization. ELP clustered normalized wave snippets into a vocabulary with $K=20$ total clusters and converted ECGs into sequences of integer tokens (Mousavi et al., 2020). HeartLang advanced this idea by introducing the QRS-Tokenizer, which centers segmentation on detected QRS complexes, forms heartbeat “words,” pads or truncates to a sentence length $l=256$, and learns a vector-quantized vocabulary $\mathcal V$ with $k=8\,192$ entries of dimension $d=128$, of which $5\,394$ were used in validation (Jin et al., 15 Feb 2025). RhythmBERT further imposed physiological structure by delineating P, QRS, and T segments, learning separate convolutional autoencoders with $d_P=d_T=12$ and $d_{QRS}=24$, clustering latents into $k_P=13$, $k_{QRS}=11$, and $l=256$0, and combining these discrete rhythm tokens with continuous morphology embeddings projected to $l=256$1 (Wang et al., 26 Feb 2026).

A second branch relies on learned ECG encoders that produce continuous embeddings for an LLM. In the few-shot ECG question-answering ELM, a self-supervised ECG encoder outputs a sequence $l=256$2, which is mapped by the trainable “Meta Mapper” into a prefix $l=256$3 prepended to textual token embeddings for a frozen decoder-only LLM; the default mapper uses 4 attention layers, 8 heads, and dropout $l=256$4 (Tang et al., 2024). MEIT likewise converts a $l=256$5-lead, $l=256$6 s ECG $l=256$7 into ECG embeddings $l=256$8 and injects them as keys and values in every causal-attention layer of the LLM, rather than adding a separate cross-modal head (Wan et al., 2024). CLIC uses a different multimodal design: a ResNet18-based ECG encoder produces a $l=256$9-dimensional vector $\mathcal V$0, a frozen ClinicalBERT text encoder produces a $\mathcal V$1-dimensional $\mathcal V$2, and the concatenated $\mathcal V$3-dimensional representation is passed to an MLP classifier (Lucafo et al., 18 May 2026).

A third branch minimizes modality-specific machinery. ECG-Byte replaces a pretrained ECG encoder with an adapted byte-pair encoding tokenizer. One implementation quantizes ECGs into an alphabet of size $\mathcal V$4, applies a default $\mathcal V$5, obtains a final vocabulary of approximately $\mathcal V$6, and reports a median encoded length of approximately $\mathcal V$7 tokens per $\mathcal V$8 s segment with a $\mathcal V$9 compression ratio (Han et al., 2024). A later unified benchmark used $k=8\,192$0 merges on $k=8\,192$1 full preprocessed ECGs and reported a final vocabulary of size approximately $k=8\,192$2 (Han et al., 24 May 2025). ELF goes further by removing the ECG encoder entirely: a raw $k=8\,192$3-lead, $k=8\,192$4 s, $k=8\,192$5 Hz signal $k=8\,192$6 is flattened and mapped by a single linear projection $k=8\,192$7, and the resulting vector is inserted as a “<signal>” token embedding in the LLM input sequence (Han et al., 5 Jan 2026).

Image-based representations remain a viable but separate lineage. ECG retrieval work encoded denoised PTB-XL ECGs as Markov Transition Field, Gramian Angular Field, Recurrence Plot, or All-Grid images, then aligned them with reports using ViT and BERT projections into a $k=8\,192$8-dimensional normalized joint space (Qiu et al., 2023). The later unified representation benchmark found that symbolic representations achieved the greatest number of statistically significant wins over both signal and image inputs across six public datasets and five evaluation metrics (Han et al., 24 May 2025).

3. Learning objectives and adaptation regimes

Training strategies in ELMs range from classic masked modeling to instruction tuning, meta-learning, retrieval conditioning, and curriculum learning. In domain-specific textual adaptation, ECG-LLM fine-tuned open-weight Llama 3.1 $k=8\,192$9B and $d=128$0B models on approximately $d=128$1 synthetic question–answer pairs generated from $d=128$2 ECG-focused journal articles and textbooks. Fine-tuning used LoRA adapters on attention, feed-forward, and output projections, covering approximately $d=128$3 of model parameters, with AdamW and paged-32 quantization, cosine decay from $d=128$4 to $d=128$5, batch size $d=128$6 examples per H100 GPU, $d=128$7 epochs, weight decay $d=128$8, global norm clipping $d=128$9, and cross-entropy on answer tokens only: $5\,394$0 The reported best LoRA setting was $5\,394$1 (Ahrens et al., 21 Oct 2025).

Few-shot multimodal ELMs use a different regime. The meta-learning QA model freezes the LLM backbone, trains only the fusion module, and optimizes under MAML with inner-loop task adaptation

$5\,394$2

and outer-loop meta-optimization

$5\,394$3

This setup is explicitly designed for tasks with limited labeled ECG-question-answer triples and unseen attribute–answer classes (Tang et al., 2024).

Instruction tuning dominates report-generation ELMs. MEIT converts each ECG–report pair into a chat-style template and computes autoregressive cross-entropy only on the target report tokens, masking the prompt and ECG positions. The paper states that no separate contrastive or additional alignment loss is used; alignment is induced through shared cross-attention fusion and the autoregressive objective. Fine-tuning employs LoRA with rank $5\,394$4, alpha $5\,394$5, dropout $5\,394$6, batch size $5\,394$7 with gradient accumulation $5\,394$8, sequence length $5\,394$9, $d_P=d_T=12$0 epochs, and a linear learning-rate schedule with $d_P=d_T=12$1 warm-up (Wan et al., 2024).

Self-supervised ECG-language pretraining has increasingly become multi-objective. ESI pairs ECGs with LLM-generated, textbook-anchored text and combines a symmetric InfoNCE contrastive loss with a captioning loss over approximately $d_P=d_T=12$2k pairs from PTB-XL, Chapman-Shaoxing, and MIMIC-IV-ECG (Yu et al., 2024). MELP extends this to three scales—token, beat, and rhythm—by combining captioning, local contrastive alignment, and global contrastive alignment in

$d_P=d_T=12$3

with default $d_P=d_T=12$4 and $d_P=d_T=12$5, trained on $d_P=d_T=12$6k high-quality ECG–text pairs from MIMIC-IV-ECG (Wang et al., 11 Jun 2025). RhythmBERT, by contrast, uses a standard BERT-style masked language modeling objective on $d_P=d_T=12$7 of discrete rhythm tokens after pretraining the tokenizer and transformer on $d_P=d_T=12$8 MIMIC-IV-ECG recordings (Wang et al., 26 Feb 2026). HeartLang separates form-level pretraining from rhythm-level pretraining through vector-quantized heartbeat reconstruction followed by masked ECG sentence prediction (Jin et al., 15 Feb 2025).

The most elaborate curriculum appears in CAMEL. Its frozen MedGemma-4B backbone is adapted with LoRA modules of rank $d_P=d_T=12$9 through five stages: ECG autoencoding on over $d_{QRS}=24$0 billion $d_{QRS}=24$1 s segments, multiple choice and short answer instruction tuning on $d_{QRS}=24$2 M examples, statistics questions, $d_{QRS}=24$3K multi-turn dialogues, and a forecasting stage with $d_{QRS}=24$4K examples from Icentia11k (Velingker et al., 17 Feb 2026). This suggests that, in the ELM literature, language-style supervision is increasingly used not only to map ECGs to text but also to scaffold clinical reasoning behaviors.

4. Tasks, evaluation methodology, and empirical performance

ELM evaluation is unusually heterogeneous. ECG-LLM explicitly uses a four-layer evaluation stack: multiple-choice accuracy on Full ($d_{QRS}=24$5 MCQs), Special ($d_{QRS}=24$6), and Checked ($d_{QRS}=24$7) sets; automatic text similarity with BLEU-4, ROUGE-1/2/L F1, and BERTScore F1; LLM-as-judge using Deepseek R1 on question–candidate–reference triples; and human expert review on $d_{QRS}=24$8 factual and $d_{QRS}=24$9 complex questions with graded labels from $k_P=13$0 to $k_P=13$1, with $k_P=13$2-replicate bootstrapping and $k_P=13$3 confidence intervals (Ahrens et al., 21 Oct 2025). Under this protocol, Llama 3.1 $k_P=13$4B + FT scored $k_P=13$5, $k_P=13$6, and $k_P=13$7 on Special, Full, and Checked multiple-choice sets, respectively; its text-similarity scores were ROUGE-1 $k_P=13$8, ROUGE-2 $k_P=13$9, ROUGE-L $k_{QRS}=11$0, BLEU-4 $k_{QRS}=11$1, and BERTScore $k_{QRS}=11$2. Yet in LLM-as-judge Claude 3.7 answered $k_{QRS}=11$3 of $k_{QRS}=11$4 questions correctly versus $k_{QRS}=11$5 for $k_{QRS}=11$6B+FT, and human expert review on complex questions favored RAG $k_{QRS}=11$7B, base $k_{QRS}=11$8B, and Claude 3.7, while $k_{QRS}=11$9B+FT incurred a slight drop with $l=256$00 errors (Ahrens et al., 21 Oct 2025).

Few-shot ECG QA uses task-centric meta-test protocols rather than broad generative evaluation. The multimodal meta-learning ELM reports, in the $l=256$01-way $l=256$02-shot setting with LLaMA-3.1-8B, accuracies of $l=256$03, $l=256$04, and $l=256$05 on Single-Verify, Single-Choose, and Single-Query question types, respectively. Cross-domain adaptation from PTB-XL to MIMIC-IV-ECG improves from $l=256$06 without meta-adapt to $l=256$07 with meta-adapt in the $l=256$08-way $l=256$09-shot setting (Tang et al., 2024).

Report-generation benchmarks stress lexical and semantic overlap metrics. MEIT, trained on PTB-XL and MIMIC-IV-ECG, evaluated nine open-source decoder-only LLMs with nine metrics. On MIMIC-IV-ECG, LLaMA-2 and Mistral-Instruct led most metrics, with BLEU-4 approximately $l=256$10, METEOR approximately $l=256$11, ROUGE-L approximately $l=256$12, and BERTScore-F1 approximately $l=256$13. Zero-shot instruction-tuned models retained approximately $l=256$14 of their PTB-XL fine-tuned performance and outperformed “no-IT” by approximately $l=256$15–$l=256$16 averaged over BLEU-3, BLEU-4, METEOR, and ROUGE-L (Wan et al., 2024).

Open-source RAG studies focus on grounded natural language generation. The RAG pipeline for ELMs reports, on ECG-Chat Instruct, an ECG-Byte baseline BLEU-4 of $l=256$17 and Accuracy $l=256$18, improving to BLEU-4 $l=256$19 and Accuracy $l=256$20 with RAG. On ECG-QA MIMIC-IV, BLEU-4 rises from $l=256$21 to $l=256$22 and Accuracy from $l=256$23 to $l=256$24. The ablations show that RAG only at inference yields BLEU-4 $l=256$25 and Accuracy $l=256$26, RAG only at training degrades below baseline to $l=256$27 and $l=256$28, and using RAG at both training and inference gives the best result (Song et al., 30 Sep 2025).

Forecasting widens the task definition of ELMs beyond interpretation. CAMEL reports zero-shot evaluation over $l=256$29 tasks and $l=256$30 datasets, with an absolute $l=256$31 average gain on ECGBench over prior SOTA ELMs and $l=256$32 over fully supervised models on ECGForecastBench, including $l=256$33 macro-F1 at $l=256$34 s and $l=256$35 s, compared with $l=256$36 for XGB, $l=256$37 for CNN, and $l=256$38 for GPT-5.2 (Velingker et al., 17 Feb 2026). This broadening of scope implies that “ELM performance” cannot be reduced to a single benchmark family.

5. Retrieval, grounding, explainability, and deployment

Grounding mechanisms are central because free-form clinical language generation is vulnerable to hallucination. ECG-LLM implements a RAG pipeline over the same ECG literature used for fine-tuning, using recursive $l=256$39-token chunks with $l=256$40-token overlap, PubMedBERT embeddings of dimension $l=256$41, a Chroma vector database with approximate HNSW indexing, cosine similarity, retrieval of the top-$l=256$42 chunks, and reranking to top-$l=256$43 via dual-encoder rescoring (Ahrens et al., 21 Oct 2025). The open-source RAG study generalizes this design space by indexing ECG features and reports in FAISS IndexIVFFlat, retrieving top-$l=256$44 nearest neighbors under $l=256$45 or cosine similarity, and showing that $l=256$46 to $l=256$47 is sufficient, while larger $l=256$48 adds noise (Song et al., 30 Sep 2025).

ALFRED adds a more explicit diagnostic scaffold. It defines an ELM as the set $l=256$49Feature-Extraction Module + Rule Module + Retrieval Module + LLM Head$l=256$50, uses an expert-curated knowledge base split into $l=256$51-character chunks, indexes $l=256$52-dimensional embeddings in Milvus HNSW with $l=256$53 and $l=256$54, retrieves $l=256$55 passages for features and $l=256$56 for disease terms with a similarity threshold $l=256$57, and prompts GPT-4o-Mini to emit a JSON diagnosis over the five PTB-XL superclasses with explanations grounded in features and retrieved definitions (Yu et al., 30 Apr 2025). On PTB-XL fold $l=256$58, the proposed ELM attained PPV $l=256$59, NPV $l=256$60, sensitivity $l=256$61, and specificity $l=256$62, versus $l=256$63, $l=256$64, $l=256$65, and $l=256$66 for the base framework without rules (Yu et al., 30 Apr 2025). This architecture makes interpretability procedural rather than purely post hoc.

Textual context itself can function as a grounding modality. CLIC converts acquisition-time metadata into language either through a deterministic template or through an LLM-guided paragraph, then fuses the resulting text embedding with ECG features. On PTB-XL, ECG-Only achieved Accuracy $l=256$67 and Macro-F1 $l=256$68, ECG+Attr reached $l=256$69 and $l=256$70, CLIC-DET reached $l=256$71 and $l=256$72, and CLIC-LLM reached $l=256$73 and $l=256$74; for the minority CD class, F$l=256$75 improved from $l=256$76 in ECG-Only to $l=256$77 in CLIC-DET (Lucafo et al., 18 May 2026). The result indicates that controlled clinical text can be a stronger anchor than linguistically richer but more variable LLM-generated context under frozen text-encoder settings.

Explainability has also been explored through signal-to-token formulations. For intracardiac EGMs, tokenization into $l=256$78 discrete bins, vocabulary extension with signal and AFib label tokens, and joint MLM plus classification training enabled attention visualization, integrated gradients, and counterfactual analyses. Clinical Longformer achieved sensitivity $l=256$79, specificity $l=256$80, PPV $l=256$81, NPV $l=256$82, and accuracy $l=256$83 on AFib classification in the reported internal dataset (Han et al., 2024). Although EGMs are not standard surface ECGs, this line of work suggests that language-style tokenization can expose token-level interpretability not available in dense latent connectors.

Deployment concerns are unusually salient in ECG settings because local inference is often feasible. ECG-LLM reports that all fine-tuned and RAG models can be containerized on premise on NVIDIA A100/H100 hardware, with no PHI stored, retrieval over an encrypted vector database, optional ephemeral RAG contexts, static $l=256$84B+FT inference at approximately $l=256$85 ms/token on A100, an additional approximately $l=256$86 ms/query for vector search and embedding in the RAG pipeline, resource requirements of $l=256$87A100 $l=256$88 GB or $l=256$89H100 for $l=256$90B+FT, and a single $l=256$91 GB GPU for $l=256$92B+RAG (Ahrens et al., 21 Oct 2025). This makes privacy-preserving, locally deployable ELMs a distinct practical objective rather than a secondary implementation detail.

6. Limitations, misconceptions, and open directions

A common misconception is that high ELM benchmark scores necessarily imply strong ECG grounding. ELF directly challenges this assumption. On PTB-XL, training and inference with real ECG, zero tensors, or text only changed accuracy only modestly, from $l=256$93 to $l=256$94 to $l=256$95, and training on text alone while inferring on real ECG increased accuracy to $l=256$96 (Han et al., 5 Jan 2026). Similar perturbation behavior was observed for other models. This suggests that current benchmarks often permit substantial reliance on language priors, benchmark artifacts, or answer-format regularities rather than waveform-derived understanding.

A second misconception is that automatic metrics provide a stable ordering of systems. ECG-LLM shows strong disagreement across evaluation layers: fine-tuning dominates multiple-choice and text-overlap metrics, whereas human expert evaluation on complex questions prefers Claude 3.7 and RAG variants (Ahrens et al., 21 Oct 2025). The few-shot meta-learning paper likewise notes that generative answers evaluated by overlap metrics may miss clinical correctness nuances (Tang et al., 2024). These results suggest that ELM assessment requires task-specific triangulation rather than a single proxy score.

A third misconception is that more elaborate natural-language context is always better. CLIC demonstrates the opposite under its frozen-ClinicalBERT setup: deterministic template text outperforms LLM-generated contextual descriptions on both Accuracy and Macro-F1 (Lucafo et al., 18 May 2026). A plausible implication is that lexical consistency and controlled semantics may matter more than narrative richness when the downstream fusion mechanism is simple concatenation.

Open directions in the literature are correspondingly diverse. RhythmBERT identifies multi-lead extension, dynamic rhythm generation, fine-grained interpretability aligning learned tokens with expert-defined morphologies, and cross-modal integration with metadata as key next steps (Wang et al., 26 Feb 2026). MELP points toward external medical knowledge bases and more explicit clinically interpretable token-level objectives (Wang et al., 11 Jun 2025). The few-shot multimodal ELM proposes multi-ECG comparison, additional modalities such as echo and X-ray, lightweight adapters or prompt tuning, and uncertainty quantification with human-in-the-loop verification (Tang et al., 2024). CAMEL emphasizes longer context windows, alternative tokenization, and multimodal prognostication with other continuous vitals or structured records (Velingker et al., 17 Feb 2026).

Taken together, these papers indicate that the field is moving in two simultaneous directions. One direction seeks stronger physiological inductive bias through rhythm-aware tokenization, multi-scale supervision, and longer temporal context. The other seeks stronger clinical grounding through retrieval, rules, metadata text, and privacy-preserving local deployment. Whether these strands ultimately converge into a single dominant ELM paradigm remains unresolved, but the literature already establishes that electrocardiography can be modeled as language at the levels of signal structure, clinical knowledge, and interactive reasoning.