Plasticity and Stability Profiles

• Plasticity and Stability Profiles are analytical characterizations that define a system's ability to adapt to new data while retaining prior knowledge.
• Various measurement frameworks, such as accuracy trajectories, representational similarity, and weight geometry, quantify stability and plasticity in continual learning.
• Engineered profiles use explicit mechanisms like role assignment, regularization, and adapter modules to optimize the balance between plasticity and stability.

Plasticity and stability profiles are empirical or analytical characterizations of how a learning system distributes adaptation and retention over time, layers, parameters, or tasks. In continual learning, such profiles are used to distinguish systems that merely preserve a strong initial representation from systems whose representations actually improve as new data arrive; in recommendation and parameter-efficient finetuning, the same idea appears as a two-dimensional target-versus-retention characterization under retraining or adaptation (Kim et al., 2023, Lavoura et al., 5 Aug 2025, Huang et al., 27 May 2026). Taken together, these works suggest that no single universal formalization has emerged: some define explicit profile metrics, whereas others infer profiles from accuracy trajectories, representational similarity, or architectural role separation.

1. Conceptual scope of the stability–plasticity profile

The underlying dilemma is classical: a system should remain stable enough to preserve previously acquired knowledge, yet plastic enough to absorb new structure. In class-incremental learning, this appears at stage $i$ as a model $M_i = G_i \circ F_i$, where excessive plasticity causes catastrophic forgetting in the feature extractor and excessive stability prevents meaningful representation growth (Kim et al., 2023). In rehearsal-free lifelong image classification, the same tension is framed between a shared backbone that should accumulate task-invariant structure and task-specific adapters that should absorb task-local variation (Wang et al., 8 Mar 2025). In continual reinforcement learning under gradual environmental drift, the balance shifts again: the dominant issue may be insufficient stability rather than insufficient plasticity, because the environment changes smoothly and often revisits related regimes (Chua et al., 25 May 2026). In parameter-efficient finetuning of LLMs, plasticity is operationalized as target-domain adaptation and stability as retention of pretrained general capabilities, so the profile becomes a target-versus-retention frontier rather than a single downstream score (Huang et al., 27 May 2026).

This breadth matters because the term “profile” does not denote a single object. In some settings it is an explicit pair of scalars; in others it is a representation-centered diagnostic, a layerwise similarity map, an architectural specialization, or a trajectory through time. A stability–plasticity profile is therefore best understood as a structured description of where, how, and at what timescale a system remembers and changes.

2. Measurement frameworks

A prominent representation-centered formulation evaluates how much the feature extractor actually improves across incremental stages. In class-incremental learning, one freezes $F_j$, retrains a classifier $G'$ on the full dataset, and evaluates $M'_j \equiv G' \circ F_j$. The quantity

$$\Delta M'_i := \mathrm{Acc}(M'_i, D) - \mathrm{Acc}(M'_0, D)$$

directly measures whether the stage-$i$ representation is better than the base representation: large positive values indicate successful incremental representation learning, values near zero indicate strong stability but little plasticity, and large negative values indicate forgetting (Kim et al., 2023). The same work complements this with same-layer CKA between early and late models, treating high CKA as evidence of feature stability and low CKA as representational drift.

Other domains use more explicitly two-dimensional profile scores. In recommender systems, two models are trained—$M_1$ on earlier data and $M_2$ on earlier plus later data—and both are evaluated on old and new holdouts, yielding $S_{1,1}, S_{1,2}, S_{2,1}, S_{2,2}$. Stability and plasticity are then defined as

$M_i = G_i \circ F_i$0

This makes the profile a literal pair of scores describing old-regime retention and new-regime adaptation under retraining (Lavoura et al., 5 Aug 2025).

Minimal continual-learning agents admit even simpler geometric profiles. One such agent maintains a persistent state vector and measures representational change through cosine similarity,

$M_i = G_i \circ F_i$1

with interval stability

$M_i = G_i \circ F_i$2

Here, high similarity corresponds to stability, while temporary drops indicate adaptation or plasticity (Subramanian, 23 Feb 2026).

A more intervention-oriented definition appears in reinitialization-based continual learning. FIRE defines stability by Squared Frobenius Error,

$M_i = G_i \circ F_i$3

and plasticity by Deviation from Isometry,

$M_i = G_i \circ F_i$4

The profile is then the joint location of a model in the low-SFE/low-DfI plane, with the proposed reset computed as the closest isometric point to the current weights (Han et al., 8 Feb 2026).

These measurement schemes are not interchangeable. Some isolate representation quality, some compare pre- and post-retraining behavior, and some evaluate geometric properties of weights or states. This suggests that profile measurement is tightly coupled to the intervention space in which stability and plasticity are being managed.

3. Internal structure: layers, neurons, architectures, and temporal depth

Several studies show that stability and plasticity are not uniformly distributed within a model. In continual self-supervised learning, layerwise CKA analysis on ResNet-18 indicates that batch normalization layers play a critical role for stability while convolutional layers are the main source of plasticity. This observation motivates freezing BN layers and preserving old convolutional paths while adding temporary trainable branches for adaptation (Liu et al., 2024).

A finer-grained version appears in deep reinforcement learning. NBSP identifies task-relevant “RL skill neurons” using a goal-oriented score based on the consistency between neuron activation states and a Goal Proximity Metric. These neurons are then given attenuated gradients through

$M_i = G_i \circ F_i$5

so that high-score neurons become stability-critical while the remainder of the network stays fully plastic (Lan et al., 9 Apr 2025).

At a coarser scale, architecture itself can induce a profile. Under approximately equal parameter budgets, deeper and narrower models show better plasticity, whereas wider and shallower models show better stability. This pattern is reported for ResNets on ImageNet100, MLPs on Split MNIST, and ViT-style models, and leads to a dual-architecture view in which one network is assigned the plastic role and another the stable role (Lu et al., 4 Jun 2025).

Temporal structure can also be hard-coded. In exemplar-free class-incremental learning, PlaStIL freezes the initial base and initial top for old classes while using partially fine-tuned recent tops for recent classes. The resulting hybrid classifier

$M_i = G_i \circ F_i$6

creates an age-structured profile: oldest classes are predicted through the most stable path, while newer classes receive more adaptive representation support (Petit et al., 2022).

These results collectively imply that a profile may be layerwise, neuronwise, architectural, or age-structured. Stability and plasticity are often best understood as distributed roles rather than global properties.

4. Mechanisms for engineering profiles

A broad design pattern is to construct explicit stable and plastic anchors, then optimize between them. ANCL embodies this directly. Given an old model $M_i = G_i \circ F_i$7 and an auxiliary network $M_i = G_i \circ F_i$8 trained only on the current task, the continual objective becomes

$M_i = G_i \circ F_i$9

For quadratic regularization methods this creates a parameter-space interpolation between the old and auxiliary models; for distillation-based methods it creates an interpolation in representation or logit space (Kim et al., 2023).

A related decomposition appears in AdaLL. The backbone $F_j$0 is updated under regularization to retain and accumulate task-invariant structure, while a task-specific adapter

$F_j$1

absorbs task-specific variation. The paper’s strongest mechanistic evidence comes from the comparison among frozen-backbone adaptation, co-trained backbone without regularization, and co-trained backbone with backbone regularization, where the last option yields the best balance (Wang et al., 8 Mar 2025).

Other methods treat the profile as a tunable continuum. The Simple Linear Connector trains a stability-oriented endpoint $F_j$2 and a plasticity-oriented endpoint $F_j$3, then interpolates them: $F_j$4 With $F_j$5 the model is maximally stable; with $F_j$6 it is maximally plastic; and the default choice $F_j$7 is analytically motivated as a compromise (Lin et al., 2021).

Flashback Learning turns this into a two-phase bidirectional regularization scheme. After a brief first phase produces a primary model $F_j$8, the trainable model is reset to the stable checkpoint $F_j$9 and optimized with

$G'$0

where $G'$1 is derived from a Stable Knowledge Base and $G'$2 from a Plastic Knowledge Base. The resulting gradient is no longer pulled only toward old-task targets; it is pulled toward an interpolation of old and newly adapted targets (Mahmoodi et al., 31 May 2025).

FIRE addresses the same problem from weight geometry. It defines the desired balance as the projection

$G'$3

with solution

$G'$4

This turns profile control into a constrained reinitialization problem: remain as close as possible to the current solution while restoring isometry for further adaptation (Han et al., 8 Feb 2026).

Across these methods, the common pattern is explicit role assignment. Stability is placed in old models, frozen paths, regularized backbones, or weight proximity; plasticity is placed in auxiliary networks, adapters, new branches, or isometry-restoring resets.

5. Cross-domain manifestations and theoretical formalisms

The profile perspective is not limited to supervised continual learning. In continual named entity recognition, SPT argues that standard knowledge distillation is overly stable because it enforces strict representation consistency. It relaxes this with pooled distillation,

$G'$5

and complements it with selective weight fusion and confidence-based pseudo-labeling for the shifting non-entity class. The reported pattern is balanced improvement on both old-type and new-type F1 across ten continual NER settings (Zhang et al., 5 Aug 2025).

In gradual-drift reinforcement learning, the profile leans more strongly toward stability. The environment is modeled as

$G'$6

with transition dynamics changing through a latent continuous process $G'$7. The main empirical result is that stability-favoring methods outperform plasticity-injection methods, and that synaptic consolidation applied to successor features outperforms consolidation of Q-values in the main continual-drift settings. Consolidation is most effective when successor features are stabilized across multiple timescales, with fast components supporting immediate adaptation and slower ones preserving long-term predictive structure (Chua et al., 25 May 2026).

In large-language-model adaptation, PEFT-Arena recasts PEFT as a target-versus-retention problem. The profile is evaluated on a two-axis plane: target-domain accuracy as plasticity and general-capability retention as stability. Under matched parameter budgets, orthogonal finetuning is reported to occupy the most favorable Pareto frontier, while final SFT checkpoints often overshoot a better target-retention operating point. This makes profile control a post-hoc path-selection problem as well as a training-time design problem (Huang et al., 27 May 2026).

Theoretical formalisms also exist outside benchmark-driven continual learning. In stochastic neural fields with dynamic connectivity, localized activity bumps carry stability profiles through presynaptic short-term plasticity variables $G'$8 and $G'$9, with effective synaptic efficacy $M'_j \equiv G' \circ F_j$0. Facilitation on excitatory synapses increases bump stability, depression on excitatory synapses decreases it, and the sign inverts for inhibitory synapses (Cihak et al., 2023). In continuous-time recurrent neural networks with plastic weights, matrix measure flows track the scalar quantity $M'_j \equiv G' \circ F_j$1 induced by a matrix flow $M'_j \equiv G' \circ F_j$2, yielding

$M'_j \equiv G' \circ F_j$3

This makes stability under plasticity analyzable as a scalar comparison process rather than only as a frozen-weight property (Kozachkov et al., 2022).

Taken together, these studies suggest that profiles can be behavioral, representational, geometric, or dynamical, depending on whether the dominant concern is accuracy, interference, retention of pretrained capabilities, or stability of continuous-time trajectories.

6. Limitations, ambiguities, and open questions

The literature is methodologically rich but terminologically uneven. Some works define explicit profile metrics, yet others infer stability and plasticity only indirectly from average accuracy, final accuracy, or task trajectories. For example, AdaLL does not define separate formal scalar metrics named stability and plasticity, nor does it report standard continual-learning quantities such as backward transfer, forward transfer, forgetting score, or intransigence numerically; the balance is inferred from slower accuracy decline and stronger current-task performance (Wang et al., 8 Mar 2025). PEFT-Arena likewise uses a two-dimensional frontier rather than a single official composite score (Huang et al., 27 May 2026). The recommender-systems framework provides explicit formulas, but only for a single old/new retraining transition and without multi-step trajectories, confidence intervals, or significance tests (Lavoura et al., 5 Aug 2025).

These differences suggest that profile quality is still partly benchmark-defined. In class-incremental learning with strong pretraining, a flat profile may indicate representation stagnation rather than successful retention (Kim et al., 2023). In gradual-drift RL, the same stability-heavy profile can be advantageous because the environment changes smoothly rather than abruptly (Chua et al., 25 May 2026). In Task-IL, task identity reduces interference at inference and can make profile improvements easier to obtain than in Class-IL (Wang et al., 8 Mar 2025). Claims that a method “eliminates” the dilemma are therefore stronger than the evidence usually establishes; the empirical record more clearly supports improved operating points than literal elimination.

A further ambiguity is documentary rather than conceptual. In one case, a document labeled as “Entropy-based Stability-Plasticity” does not actually contain a recoverable method description, equations, experiments, or profiles, and only its bibliography reveals topical proximity to continual learning, catastrophic forgetting, layer freezing, and branching (Araujo et al., 2022). This underscores a broader point: the term “plasticity and stability profile” can denote anything from a rigorously specified metric to a qualitative architectural intuition.

The open questions are correspondingly structural. A unified theory would need to relate representation growth, forgetting, parameter geometry, and architectural specialization under a common formalism. Existing work suggests several promising directions: measuring profiles beyond endpoint accuracy, separating profile components across timescales or modules, and designing interventions that preserve predictive or invariant structure while allowing targeted adaptation. The field has established that profiles are measurable and engineerable; it has not yet standardized what the profile should be.