PECFT: Efficient Continual Fine-Tuning

• PECFT is a continual learning approach that adapts large pre-trained models by updating a small set of task-specific parameters while preserving previously learned knowledge.
• It employs techniques such as adapters, low-rank updates, and prompt tuning to minimize computational overhead and mitigate catastrophic forgetting.
• Empirical studies reveal PECFT methods achieve near full fine-tuning performance with less than 5% additional parameters, enhancing stability across multiple tasks.

Parameter-Efficient Continual Fine-Tuning (PECFT) refers to a family of continual learning techniques designed to adapt large pre-trained models to new tasks over time while only introducing a small set of task-specific trainable parameters. By limiting the number of parameters that are updated per task—using modules such as adapters, low-rank updates (LoRA), or prompt tokens—PECFT methods aim to preserve previous knowledge (preventing catastrophic forgetting) and enable scalable, efficient adaptation across many tasks and evolving domains (Coleman et al., 18 Apr 2025, Chen et al., 2023, Liu et al., 2024, Gu et al., 7 Jun 2025). This paradigm is motivated by the high computational and storage costs associated with repeated full-model fine-tuning, as well as the brittle generalization and privacy constraints of deploying large language or vision models in dynamic, real-world environments.

1. Formal Problem Setting and Methodological Foundations

In continual learning, a model $f_{\Theta}$ (with parameters $\Theta$) encounters a sequence of $T$ tasks, each with its own distribution $p^t(x, y)$ and dataset $D^t$ (Coleman et al., 18 Apr 2025). The goal is to incrementally update $f_\Theta$ for each task $t$ to achieve high accuracy on all learned tasks—without revisiting earlier datasets. In PECFT, rather than updating all of $\Theta$, a small set of per-task parameters $Z^t$ is introduced, keeping the backbone frozen.

Typical PEFT parameterizations include:

• Adapters: Bottleneck networks inserted in transformer blocks: $$\text{Adapter}(h) = W_\text{up} \sigma(W_\text{down} h)$$.
• LoRA: Low-rank decomposition for trainable weight updates: $W = W_0 + BA$ with $$B \in \mathbb{R}^{d \times r}, A \in \mathbb{R}^{r \times k}, r \ll \min(d, k)$$.
• Prompt/Prefix-tuning: Prepending learnable virtual tokens $P \in \mathbb{R}^{\ell \times d}$.

The adaptation optimization is:

$$Z^t = \arg\min_{Z} \mathcal{L}(D^t ; \Theta_0, Z) \quad \text{subject to} \quad |Z| \ll |\Theta_0|$$

(Coleman et al., 18 Apr 2025, Chen et al., 2023).

The primary challenge in this regime is catastrophic forgetting. As only the PEFT module is updated per task, without proper regularization or architectural isolation, knowledge acquired on earlier tasks can be inadvertently erased.

2. Core Taxonomies and Algorithmic Strategies

PECFT methods are categorized by their approach to stability–plasticity, parameter isolation, and handling of representation overlap.

Regularization-based Methods

• Penalize deviation from past PEFT parameters (e.g., EWC on LoRA; prompt-based distillation).
• Example: Orthogonal Gradient Projection (PEGP) forces all PEFT parameter updates $\Delta E$ to be orthogonal to the old feature subspace, thus bounding forgetting by ensuring $x_t \Delta E = 0$ for old features $x_t$ (Qiao et al., 2024).

Replay-based Methods

• Maintain a small memory of exemplars for regularization or pseudo-replay.
• Example: Episodic Replay with LoRA (ERI-LoRA), task arithmetic with stored adapters.

Parameter Isolation/Dynamic Architecture

• Allocate distinct adapters/prompts for each task (e.g., DualPrompt, Continuous Adapter, SEMA), with task-specific routing.

Optimization-based Approaches

• Enforce orthogonality or subspace constraints in PEFT gradients (e.g., InfLoRA, Interpolation-LoRA).

Strategy	Key Mechanism	Application in PECFT
Regularization	Penalty on parameter drift	Orthogonal projection, EWC
Replay	Limited memory buffer	Episodic replay w/ LoRA
Param. Isolation	Per-task isolation (adapters)	DualPrompt, CAL, SEMA
Optimization-based	Subspace/gradient orthogonality	InfLoRA, PEGP

All approaches target the competing demands of rapid new task adaptation (plasticity) and retention of prior task performance (stability) with stringent parameter budgets.

3. Theoretical Analysis and Model Dynamics

Recent work applies the Neural Tangent Kernel (NTK) framework to analyze PECFT (Liu et al., 2024). For a frozen base model $f_0^*$ and small per-task subnetwork $p_\tau$, the NTK

$$K(x, x') = \langle \nabla_\theta f(x; \theta), \nabla_\theta f(x'; \theta) \rangle$$

describes how parameter changes induce function-level adaptation. Three central levers emerge:

• Sample size expansion: Increasing task data reduces generalization gap.
• Feature orthogonality: Enforcing cross-task orthogonality in PEFT representations ($\Phi_k(X_\tau, X_k) \approx 0$) minimizes interference and forgetting.
• Regularization: Ridge penalties on PEFT updates control the effect of adaptation.

Orthogonal gradient projection methods (Qiao et al., 2024) formalize this at the update level:

$g_\text{proj} = (I - P_U) g = g - P_U g$

ensuring outputs on old data remain unchanged up to a forgetting bound $O(\epsilon \|\Sigma_t\|)$ where $\epsilon$ is projection error and $\Sigma_t$ is feature variance.

4. Representative Algorithms and Empirical Advances

Trans-PEFT (Gu et al., 7 Jun 2025): Focuses PEFT adaptation on components robust to base model updates. Empirically, attention sublayer activations in transformer LLMs are stable under continual pre-training, while FFN activations drift substantially. Thus, Trans-PEFT regularizes fine-tuning to minimize reliance on FFN components via intra-layer masking and cross-layer dropping, preserving transferability of PEFT modules across evolving model versions with negligible accuracy loss.

Teacher–Student Prompt Distillation (Chen et al., 2023): Applies task-adaptive prompt tuning with a frozen backbone ($>$99.95% parameters unchanged), eliminating forgetting. Demonstration-based in-context tuning (ICT) with a separate “teacher” model enhances few-shot generalization, with knowledge compressed into the prompt via KL/Cross-Entropy distillation. No data or exemplars are stored beyond the compact prompt, achieving strict privacy and state-of-the-art performance in table semantic parsing.

NTK-CL (Liu et al., 2024): Triples feature representation per sample using multiple PEFT modules to improve intra-task generalization. An adaptive EMA maintains cumulative historical parameters, and prototype-based losses as well as orthogonality and dissimilarity terms enforce feature separation across tasks. Achieves superior accuracy and minimal forgetting across vision classification benchmarks.

Adaptive Entropy-Annealed Policy Gradient (aEPG) (Zhang et al., 15 Feb 2026): By recasting classification as a one-step MDP, proposes a policy gradient loss that interpolates between cross-entropy (promoting exploration) and expected policy gradient (promoting exploitation), with a decaying entropy regularization schedule. Lower-entropy prediction distributions are shown to enhance stability and accuracy in PEFT continual adaptation.

Feature Transformation Tuning (FeTT) (Qiang et al., 2024): Applies a parameter-free, non-parametric channel-wise transform to concatenated frozen and PEFT backbone features. This addresses channel suppression and distribution shift in frozen representations, especially when training on the first task is limited, yielding consistent $+1$–$2$ point gains in accuracy without any new parameters or exemplars.

5. Evaluation Methodologies and Comparative Results

PECFT research employs several principal metrics:

• Average Accuracy (AA): Mean across all tasks after the final task.
• Average Forgetting (AF): Difference between best historical accuracy per task and final accuracy.
• Model Size Efficiency (MS): Ratio of base to total model memory.
• Forward/Backward Transfer: Gains or losses from order of training.

Empirical results consistently show PECFT (LoRA, Adapter, Prompt) with regularization, replay, or architectural innovations delivers AA within $1$–$2$ points of full fine-tuning, while using $<5\%$ of additional parameters per task and halving average forgetting versus vanilla replay (Coleman et al., 18 Apr 2025, Qiao et al., 2024, Chen et al., 2023, Liu et al., 2024).

Trans-PEFT achieves direct transfer accuracy on updated LLMs within $<1$ point of ideal re-tuning, slashing maintenance compute by nearly $50\%$ (Gu et al., 7 Jun 2025). PEGP reduces forgetting by $1$–$4$ points in incremental and domain transfer settings for ViT and CLIP (Qiao et al., 2024). NTK-CL achieves $\bar{A}=93.76\%$ on CIFAR-100 with ViT, outperforming previous methods (Liu et al., 2024). FeTT realizes $+1$–$2$ points average accuracy improvement with zero new parameters across 14 class-incremental benchmarks (Qiang et al., 2024).

6. Practical Implications, Limitations, and Future Directions

Practical Scenarios and Guidelines

• Model/Task Scalability: PECFT is well-suited for serving massive numbers of users or clients, each with a small PEFT module, on frequently updated foundation models (Gu et al., 7 Jun 2025, Coleman et al., 18 Apr 2025).
• Data Privacy: Many approaches require no data storage; only minuscule prompt/module state is retained, ensuring privacy (Chen et al., 2023).
• Transfer Across Model Updates: Methods such as Trans-PEFT provide PEFT module transferability without re-tuning after base model upgrades, provided the architecture remains unchanged (Gu et al., 7 Jun 2025).

Limitations and Open Questions

• Architectural Shifts: Most PECFT frameworks cannot handle architecture-level changes (e.g., LLaMA2→LLaMA3) due to parameter incompatibility (Gu et al., 7 Jun 2025).
• Task-Agnostic Inference: Reliance on task IDs remains common; input-driven or meta-learned routing is an ongoing research challenge (Coleman et al., 18 Apr 2025).
• Balance of Stability–Plasticity: Orthogonality and regularization can marginally reduce plasticity on new tasks (Qiao et al., 2024).
• Replay and Memory Constraints: Effective replay under PEFT is still memory-intensive; synthetic latent replay and adaptive module merging are promising directions (Coleman et al., 18 Apr 2025).
• Modalities and Reasoning: Extensions to multi-modal, cross-modal, or symbolic reasoning domains require further architectural developments (Coleman et al., 18 Apr 2025).

Prospective Research Avenues

• Sublinear expansion of PEFT modules as $T$ grows.
• Module merging/composition techniques for on-the-fly reduction of parameter overhead.
• Improved task-agnostic routing to remove explicit task identification at inference.
• Expanding to complex generative or structured tasks (e.g., VQA, program synthesis).

7. Summary Table: Comparison of Recent PECFT Approaches

Method	Regularization/Strategy	Parameter Sharing	Notable Empirical Result(s)	Reference
Trans-PEFT	FFN masking/dropping	LoRA/Adapter	Δacc < 1pt to ideal, −50% retrain compute	(Gu et al., 7 Jun 2025)
PEGP	Orthogonal projection in PET space	Adapter/LoRA/VPT	−1~−4pt forgetting, better zero-shot/few-shot	(Qiao et al., 2024)
NTK-CL	Multiple subnets + NTK loss	Prompts, LoRA	SOTA on 10-task CIFAR-100 & IN-R benchmarks	(Liu et al., 2024)
Teacher-Student Prompt Distill.	Prompt-only, frozen backbone	Prompt tokens	MD≈0, no data storage, SOTA on parsing	(Chen et al., 2023)
FeTT	Param-free feature transformation	Any PEFT	+1–2pt accuracy, 0 param. overhead	(Qiang et al., 2024)
aEPG	Adaptive policy gradient w/ entropy	LoRA/Adapters	+2–3pt accuracy, stable learning curve	(Zhang et al., 15 Feb 2026)

• Parameter-Efficient Continual Fine-Tuning: A Survey (Coleman et al., 18 Apr 2025)
• Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models (Gu et al., 7 Jun 2025)
• Parameterizing Context: Unleashing the Power of Parameter-Efficient Fine-Tuning and In-Context Tuning for Continual Table Semantic Parsing (Chen et al., 2023)
• Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective (Liu et al., 2024)
• Gradient Projection For Continual Parameter-Efficient Tuning (Qiao et al., 2024)
• FeTT: Continual Class Incremental Learning via Feature Transformation Tuning (Qiang et al., 2024)
• Policy Gradient with Adaptive Entropy Annealing for Continual Fine-Tuning (Zhang et al., 15 Feb 2026)