Predicting when semi-supervised fine-tuning adds value after pretraining

Determine in advance whether a given self-supervised pretraining objective leaves sufficient room for semi-supervised fine-tuning to improve ECG delineation performance.

Background

The paper evaluates supervised and semi-supervised fine-tuning following several self-supervised pretraining objectives. It finds that semi-supervised learning substantially improves models trained from scratch and benefits MoCo-pretrained models, but adds little to MAE-pretrained models because the supervised MAE model is already near saturation.

These results suggest that the usefulness of semi-supervised fine-tuning depends on how much performance improvement remains available after pretraining. The paper does not provide a method for predicting this relationship before training, leaving unresolved how practitioners should determine whether unlabeled data will provide additional benefits for a particular pretraining objective. The problem is relevant for designing efficient ECG delineation training pipelines and avoiding unnecessary semi-supervised training when pretraining has already captured most of the available information.

References

Because it is difficult to know in advance whether a given objective leaves such room, semi-supervised fine-tuning is a safe default across the objectives we examine: it helps when room remains and at least does not degrade performance.

— Label-Efficient Deep Learning for ECG Delineation: A Multi-Dataset Benchmark against Widely Used Delineation Tools  (2610.07885 - Lim et al., 6 Oct 2026) in Section 5.1, Discussion, subsection “Effectiveness of Label-Efficient Deep Learning”