TAPE: A two-stage parameter-efficient adaptation framework for foundation models in OCT-OCTA analysis

Abstract: Automated analysis of optical coherence tomography (OCT) and OCT angiography (OCTA) images is critical for robust ophthalmic diagnosis. Existing mainstream methods trained from scratch rely heavily on massive data and model scale, thereby hindering their practical deployment in resource-constrained clinical settings. Although transfer learning based on foundation models (FMs) is promising, it still faces significant challenges: domain shift and task misalignment. To address these, we propose TAPE: A Two-stage Adaptation Framework via Parameter-Efficient Fine-tuning, which strategically decouples adaptation into domain alignment and task fitting for downstream segmentation. The domain adaptation stage notably applies parameter-efficient fine-tuning (PEFT) in the context of masked image modeling for medical image domain adaptation, a novel approach to the best of our knowledge. Applying TAPE to retinal layer segmentation on both universal (masked auto-encoder, MAE) and specialized (RETFound) FMs, it demonstrates superior parameter efficiency and achieves state-of-the-art generalization performance across diverse pathologies.

• The paper introduces TAPE, a novel two-stage adaptation framework that decouples domain alignment from task reconfiguration for OCT-OCTA segmentation.
• It employs self-supervised MIM-based domain adaptation using LoRA and a task-specific adapter for efficient, robust retinal layer segmentation.
• Experimental results on the OCTA-500 dataset demonstrate state-of-the-art performance with improved test MSE, Dice, and mIoU across diverse pathologies.

TAPE: A Two-Stage Parameter-Efficient Adaptation Framework for Foundation Models in OCT-OCTA Analysis

Introduction

Automated segmentation of retinal layers in OCT and OCTA images is fundamental for diagnosing retinal pathologies such as DR, RVO, and AMD. Current deep learning approaches—largely based on CNNs, ViTs, or tailored SSMs—suffer from excessive data and compute requirements, impeding routine clinical deployment. Transfer learning with large-scale visual and specialized ophthalmic FMs (e.g., MAE, RETFound) promises improved sample efficiency, but is stymied by critical domain shift (natural images or single-modality data versus clinical OCT/OCTA) and task misalignment (pre-training with MIM-SSL vs. downstream pixel-wise segmentation). The paper introduces TAPE, a two-stage, parameter-efficient adaptation architecture, which decouples the FM adaptation process into domain alignment and task reconfiguration, leveraging PEFT techniques for computational efficiency and improved generalization across varied disease classes.

Methodology

TAPE employs a sequential two-stage adaptation strategy as delineated in (Figure 1):

Figure 1: Stage I performs MIM-based self-supervised PEFT domain adaptation, while Stage II conducts task-specific adapter-driven segmentation fine-tuning.

Stage I: Domain Adaptation via MIM with PEFT

To address domain shift, TAPE adapts the MAE (trained on ImageNet-1K) and RETFound (pre-trained on 0.74M OCTs) to OCT-OCTA data using MIM-SSL. Three PEFT mechanisms are systematically evaluated: LoRA (applied to all linear layers in ViT blocks), ViT-Adapter modules (after MSA, FFN), and VPT (prompt tokens at input). During this generative self-supervised adaptation, only a tiny subset of new parameters is learned while the FM is largely frozen. MAE reconstructs both modalities, whereas RETFound focuses adaptation solely on OCTA channels. LoRA establishes clear gains in test MSE, outperforming both FFT of the full model and other PEFT methods in both general visual and ophthalmic FMs—highlighting its efficacy for medical MIM-based SSL and making it the default choice for TAPE.

Stage II: Task Adaptation for Retinal Layer Segmentation

From the domain-adapted FM, the original decoder is replaced with a segmentation-specific head. A task adapter (also realized via LoRA) is injected into the encoder to enhance task-specialized feature modulation; sequence-to-spatial adapters route multi-modal embeddings to 2D feature maps, which are then concatenated and fed to the segmentation head, comprised of residual blocks and convolutional upsampling. This modular, parameter-light adaptation scheme is motivated by the need to maintain the integrity of learned domain representations (freezing the domain adapter) while introducing minimal, segmentation-specific flexibility.

Experimental Results

PEFT for Domain Adaptation in MIM

Empirical analysis on the OCTA-500 dataset confirms the superiority of LoRA for domain adaptation at less than 1% of the parameter-count of FFT. Notably, LoRA-adapted models consistently achieved lower test MSEs versus full fine-tuning and alternatives (e.g., ViT-Adapter, VPT), confirming that PEFT can offer both improved transfer and compute/memory efficiency in generative SSL for medical images.

Segmentation Performance Across Pathologies

Rigorous comparisons with single-stage transfer learning, FFT, and recent fine-tuning baselines, as well as the standard scratch-trained methods ReLayNet, MGUNet, and EMVNet, demonstrate robust, state-of-the-art layer segmentation by TAPE (Figure 2).

Figure 2: Qualitative outputs for NORMAL and three disease types; TAPE offers substantially improved anatomical accuracy over all baselines especially in pathologies.

Across all disease categories and foundation architectures, TAPE achieves peak mean Dice and mIoU (cf. 93.86/89.22 on RETFound), outperforming direct transfer or task-only adaptation (e.g., STL/FFT/Task-LoRA variants), and surpassing even specialized models trained from scratch. Notably, MAE in the STL regime lags behind EMVNet/MGUNet, highlighting the necessity of dedicated domain and task adaptation for robust clinical deployment. The improvements are largest in challenging pathological presentations, validating the architectural hypothesis of distinct, efficient domain and task modules.

Theoretical and Practical Implications

A key contribution is the empirical demonstration that PEFT (especially LoRA) is applicable—and indeed advantageous—not just in downstream discriminative adaptation, but also in upstream generative domain realignment via MIM, a regime largely unexplored for SSL on biomedical imaging FMs. The two-stage decoupling, with frozen domain adapters during task adaptation, offers an actionable protocol for high-performing, compute-bound clinical deployment of FMs. TAPE generalizes across FM architectures (image-based, domain-specialized), reinforcing the modularity of adaptation and the possibility for future vertical extensibility (e.g., different imaging modalities or clinical endpoints).

Practically, TAPE enables state-of-the-art segmentation on routine hardware (single H100), opening the door for scalable translation into resource-limited clinical workflows while maintaining strict performance generalization over diverse and rare disease classes.

Future Directions

The authors propose to:

• Extend TAPE to multi-task frameworks (e.g., OCT fluid segmentation, multi-disease stratification).
• Assess and refine TAPE under few-shot learning constraints, directly addressing the scarcity of detailed annotations in clinical imaging.
• Leverage TAPE across further modalities and mixed-data settings, enabling streamlined adaptation for novel tasks or domains with existing FMs and limited bespoke tuning.

Conclusion

TAPE presents a formally-motivated and empirically-validated architecture for PEFT-driven adaptation of general and specialized visual FMs in OCT-OCTA analysis. By separating domain and task adaptation, and systematically evaluating PEFT variants, the framework achieves substantial gains in segmentation metrics, compute/memory efficiency, and robustness across pathologies. These results invite its adoption as a default protocol for the deployment of FMs under domain and task shift in medical imaging, and suggest broad applicability to future adaptation scenarios in resource-constrained AI for healthcare.

(2604.04571)