HEALNet: Self-Supervised Wound & Multimodal Fusion

• HEALNet employs a self-supervised framework using temporal coherence to classify wound healing stages, achieving 97.7% pretext accuracy and 90.6% test accuracy.
• It introduces a hybrid early-fusion model that integrates histopathology, omics, and clinical data, delivering up to a +10% uplift in survival prediction performance.
• Both approaches overcome challenges of limited annotations and missing modalities by utilizing biologically informed clustering and cross-attention mechanisms for improved interpretability.

HEALNet is a designation for two distinct methodologies in biomedical machine learning: a self-supervised framework for wound heal-stage classification without human labeling (Carrión et al., 2022), and a flexible deep learning architecture for multimodal biomedical data fusion (Hybrid Early-Fusion Attention Learning Network) with robust handling of missing modalities and interpretability for survival prediction tasks (Hemker et al., 2023). Both frameworks address key challenges in biomedical data analysis—scarcity of labeled data and integration of heterogeneous sources—using state-of-the-art neural modeling approaches.

1. Self-Supervised Temporal Embedding for Wound Healing Progression

HEALNet (Carrión et al., 2022) introduces a fully self-supervised approach to acute wound heal-stage classification. The framework is biologically inspired by the classical four-stage wound-healing trajectory (hemostasis, inflammation, proliferation, maturation). Its key innovation is in leveraging the inherent temporal order of image acquisitions from a mouse model (8 C57BL/6 mice, wounds imaged daily for 16 days, 256 total images) to learn meaningful representations of wound progression without expert annotations.

• Pretext (Temporal Coherency) Task: Ordered image pairs are generated and labeled as positive ($y_{ij}=1$) if the second image is temporally ahead, or negative ($y_{ij}=0$) otherwise. A DenseNet-121 backbone (ImageNet initialized) with a 16-dimensional projection head forms a Siamese architecture, producing pairwise embeddings $[z_i;z_j]$ used to predict temporal direction via a binary cross-entropy loss:

$$\mathcal L_{\text{pretext}} = -\frac{1}{N}\sum_{(i,j)} \bigl[y_{ij}\log\hat p_{ij} + (1-y_{ij})\log(1-\hat p_{ij})\bigr].$$

The model achieves 97.7% test-set pretext accuracy.

• Clustering for Label Discovery: The trained encoder produces 16-dimensional embeddings for each wound image. $k$-means ($k=4$) clustering in latent space establishes pseudo-labels corresponding to the canonical wound-healing stages. PCA visualization confirms cluster structure, and per-cluster day distributions align with biological expectations. Human annotators match pseudo-labels with 80.5% top-1 agreement.
• Fine-Tuned Classification: The encoder, now supervised using cluster-derived labels, is repurposed with a single 16 → 4 FC+softmax layer and trained with cross-entropy loss. Data augmentation combats imaging artifacts. Test accuracy reaches 90.6% across all four stages, compared to 78.1% for direct supervised training with limited human-labeled data.
• Strengths and Limitations: The method requires no human expert input, is effective on small datasets, and uncovers biologically meaningful structure. Its limitations include applicability only to mouse tissue (not yet validated for human wounds), reliance on RGB images, and an assumed fixed number of stages.

2. Hybrid Early-Fusion Attention for Multimodal Biomedical Data

In a separate line of work, "HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data" (Hemker et al., 2023) proposes a unified model for fusing structurally diverse biomedical modalities: whole-slide histopathology (WSI), tabular omics/clinical data, and optional graph-structured molecular profiles.

• Modality-Specific Encoders:
  • Image patches (e.g., 256×256 WSI tiles) are mapped via a CNN (e.g., ResNet50) to vector-valued representations.
  • Tabular omics or clinical variables are encoded using a fully connected or self-normalizing network (SNN).
  • Graphs, where applicable, are encoded via graph attention/convolutional layers.
• Shared Latent Array and Fusion:

An array $S_0\in\mathbb{R}^{d_s\times N_s}$ is iteratively updated using successive modality-specific cross-attention modules:

$S_{t+m} = \psi(S_t,\,C_m;\,\rho)$

where $C_m$ is the attended context from modality $m$. Weight sharing across layers reduces parameter count and ensures robustness.

• Loss and Survival Analysis:

A pooled summary of $S_T$ feeds a linear head yielding log-hazard predictions for discrete-time Cox models. The survival loss is the negative log-partial likelihood:

$$\mathcal{L} = -\sum_{i:\,\delta_i=1} \left[h(x_i) - \log\sum_{j\in R_i}e^{h(x_j)}\right]$$

enabling end-to-end hazard modeling.

3. Handling Scarce and Missing Data

Both HEALNet frameworks resolve challenges of limited or incomplete annotations.

• No-Label Wound Progression: The wound-healing HEALNet avoids all expert labeling, relying entirely on temporal coherence and emergent clusters as stage pseudo-labels (Carrión et al., 2022).
• Multimodal Imputation-Free Fusion: In the multimodal HEALNet, missing modalities are natively handled: steps for absent modality encoders and attention updates are skipped during training and inference. The design does not require data imputation or synthetic augmentation and exhibits graceful degradation under missingness, e.g., C-Index on BLCA test data dropping only from 0.668→0.612, compared to late fusion baselines degrading to 0.547 (Hemker et al., 2023).

4. Quantitative Performance and Comparative Results

Wound Heal-Stage Classification (Carrión et al., 2022)

• Pretext Pairwise Ordering: Test accuracy 97.7%
• Downstream Stage Classification: Test accuracy 90.6%
• Direct Supervised CNN Baseline: Test accuracy 78.1%
• Human–Pseudo-Label Agreement: 80.5%
• Impact of Fine-Tuning: Freezing backbone reduces accuracy by 5–7%, highlighting the necessity of end-to-end optimization.

Multimodal Survival Prediction (Hemker et al., 2023)

Below, results (mean ± std C-Index, 5-fold CV) compare HEALNet’s “Hybrid Early Fusion” to competing fusion approaches across four TCGA cohorts:

Dataset	Uni-modal Best	Intermediate/Late Fusion	HEALNet
BLCA	0.606 ± 0.019	0.620 ± 0.040/0.048	0.668 ± 0.036
BRCA	0.580 ± 0.027	0.589 ± 0.073/0.040	0.638 ± 0.073
KIRP	0.780 ± 0.035	0.789 ± 0.087/0.041	0.812 ± 0.055
UCEC	0.630 ± 0.028	0.589 ± 0.062/0.034	0.626 ± 0.037

HEALNet outperforms all late and intermediate fusion baselines, with multi-modal uplift of up to +10% over the best single-modality model.

5. Interpretability and Explainability

• Raw Input Attention (Multimodal HEALNet):

The cross-attention mechanism yields by-design attentional scores linking each output slot to raw input patches (for WSI) or individual features (omics/clinical). Averaged attention weights construct spatial heatmaps or feature importances, providing instance-level model interpretability. No separate post hoc explainer (e.g., LIME/SHAP) is required. In empirical evaluations, attention maps reliably highlight pathologically relevant regions or biomarkers, serving both as validation and as insights into learned cross-modal dependencies (Hemker et al., 2023).

6. Biological and Algorithmic Motivations; Future Directions

The wound-specific HEALNet is fundamentally motivated by the absence of large, fully annotated wound-healing datasets and the biological assumption of monotonic temporal progression across four canonical stages (Carrión et al., 2022). The multimodal HEALNet is driven by the need for flexible and robust integration of heterogeneous biomedical data, with architectures capable of exploiting cross-modal information without discarding modality-specific structure (Hemker et al., 2023).

Both frameworks exhibit limitations—mouse model specificity and fixed stage assumptions for the former; the latter’s deployment on human clinical datasets, integration with more modalities (e.g., thermal, multispectral, or graph-based inputs), and the need for uncertainty estimation or fuzzy clustering are cited as future research directions.

A plausible implication is that the HEALNet paradigms—self-supervised learning from temporal structures and raw-input cross-attention fusion—offer a generalizable template for data-constrained domains in biomedical imaging and multi-omics, where annotated data is scarce and structural heterogeneity is the norm.