---
title: Continual Alignment in Sequential Models
url: https://www.emergentmind.com/topics/continual-alignment
type: topic
---

# Continual Alignment in Sequential Models

Continual alignment denotes a family of sequential-learning formulations in which the principal object to be preserved is an alignment relation that would otherwise drift under continued adaptation. Across the recent literature, that relation may be crosslingual instruction-following behavior, latent representations in online self-supervised learning, historical and current features in medical segmentation, entity correspondence in growing knowledge graphs, classifier–backbone compatibility, or multimodal embedding structure linking text, video, audio, and images [2305.13627] [2507.10434] [2507.03638] [2207.11436] [2603.09888] [2601.20597]. The term therefore does not identify a single algorithmic template. It names a recurring principle: when models are updated sequentially, some structured dependency between old and new data, tasks, modalities, or behaviors must be explicitly maintained, re-imposed, or routed to mitigate catastrophic forgetting, feature drift, or alignment tax.

## 1. Conceptual scope and research usages

In the current literature, “alignment” is used in several technically distinct senses. In multilingual post-training, it can mean aligning newly introduced low-resource languages to an instruction-tuned competence space rather than performing preference alignment [2305.13627]. In self-supervised continual learning, it can mean aligning current latent representations to past latents through an EMA teacher or stored replay embeddings [2507.10434]. In medical segmentation, it can mean jointly aligning current and previous networks, and aligning historical and current representations inside the current network [2507.03638]. In knowledge graphs, it refers to the discovery and revision of entity correspondences as graphs grow over time [2207.11436]. In multimodal retrieval and generation, it refers to preserving a shared cross-modal embedding geometry or an alignment module that connects frozen visual and language backbones [2601.20597] [2606.12633]. In sequential safety post-training, it refers to preserving safe behavior and general capability under heterogeneous fine-tuning stages [2602.07892].

| Setting | What is aligned | Representative papers |
|---|---|---|
| Multilingual instruction tuning | New-language inputs with prior instruction-following behavior | InstructAlign [2305.13627] |
| Online SSL and rehearsal CL | Current latents with past latents or replay features | CLA [2507.10434], DualHSIC [2305.00380] |
| Medical and dense prediction | Cross-network and cross-representation dependencies | DAKR-HSIC [2507.03638], CA-SAM [2511.17201] |
| Knowledge graphs | Entity correspondences in growing KGs | ContEA [2207.11436] |
| Continual discovery and retrieval | Features with fixed geometric or cross-modal targets | GOAL [2602.19872], StructAlign [2601.20597] |
| LLM safety and post-training | Safety constraints with retained general capability | OGPSA [2602.07892], Alignment Dynamics [2605.18309] |

This multiplicity is not merely terminological. It reflects a substantive shift from viewing continual learning as parameter retention alone to viewing it as preservation of a relation: language-to-language, past-to-present latent, teacher-to-student feature, class-to-geometry, or prompt-to-safe behavior.

## 2. Recurrent failure modes that motivate continual alignment

The most common motivating failure mode is catastrophic forgetting, but the cited works sharpen that diagnosis in domain-specific ways. In continual text-to-video retrieval, forgetting is decomposed into intra-modal feature drift and non-cooperative feature drift across modalities, the latter causing text and video encoders to become misaligned even when each remains internally organized [2601.20597]. In pre-trained class-incremental learning, the central problem is that task-specific classifiers can become incompatible with the new shared feature space after the backbone is adapted or merged, creating classifier–backbone mismatch rather than simple feature loss [2603.09888]. In online continual self-supervised learning, the instability arises because data arrive in small minibatches, task boundaries are absent, and feature drift accumulates too quickly for ordinary replay alone [2507.10434].

In sequential post-training of language models, the literature increasingly casts the problem as heterogeneous continual learning. OGPSA explicitly describes safety alignment as objective-heterogeneous sequential optimization in which safety updates can overwrite reasoning, truthfulness, or instruction-following behaviors, producing alignment tax [2602.07892]. A data-centric variant reaches a related conclusion from the opposite direction: high-gradient samples cause greater safety degradation and drive models toward pretrained distributions, while moderate-gradient samples enable task learning with minimal alignment loss [2604.17215]. “Alignment Dynamics in LLM Fine-Tuning” further argues that fragility cannot be understood solely as either gradient geometry or output-distribution shift, because both interact through alignment dynamics [2605.18309].

Other domains expose additional pathologies. In growing knowledge graphs, continual entity alignment must cope with new entities and triples, revision of old predictions, and the presence of non-matchable entities for which ordinary one-way nearest-neighbor matching is unreliable [2207.11436]. In multilingual adaptation, InstructAlign argues that directly adapting new languages to instruction-tuned LLMs can cause catastrophic forgetting, and that modular adapter approaches such as MAD-X can harm multilingual inference because language competence is isolated into modules instead of being integrated into a single prompting-capable model [2305.13627]. In continual audio-video pre-training, STELLA identifies sparse spatio-temporal correlation between audio-video pairs and multimodal correlation overwriting that forgets audio-video relations [2310.08204].

## 3. Main methodological families

One large family treats continual alignment as explicit replay-conditioned representation regularization. DualHSIC adds HSIC-Bottleneck for Rehearsal and HSIC Alignment to rehearsal-based class-incremental learning: the first lessens inter-task interference on replayed data, and the second maximizes statistical dependence between current-task and buffered final-layer representations to promote task-invariant knowledge sharing [2305.00380]. CLA makes a closely related move in online continual self-supervised learning, but replaces task-boundary-dependent teachers with online targets from an EMA network or stored replay embeddings; its variants CLA-b, CLA-E, and CLA-R align present latents to temporally earlier ones while keeping the method compatible with a fixed computational budget [2507.10434]. In continual UDA, “Multi-scale Feature Alignment for Continual Learning of Unlabeled Domains” combines generative feature-driven image replay with a dual-purpose discriminator and multi-scale feature aggregation, so that the same mechanism supports both replay realism and domain alignment across multiple feature depths [2302.01287].

A second family aligns historical and current structure inside the model. DAKR-HSIC for domain-continual medical segmentation couples Cross-Network Alignment, which aligns bottleneck features from current and previous networks on buffered data, with Cross-Representation Alignment, which maximizes nonlinear HSIC between bottleneck features extracted from buffered historical data and current-domain data after feature mapping and permutation-based pairing [2507.03638]. Adapt & Align adopts a similar consolidation perspective for generative models: a local model is first trained on the new task, and then a translator aligns the local latent space with a shared global latent space so that a persistent decoder or generator can consolidate old and new knowledge without old real data [2312.13699].

A third family imposes a fixed geometric target. GOAL replaces dynamic classifier updates in continual generalized category discovery with a fixed Equiangular Tight Frame classifier; supervised alignment anchors labeled base-session features to assigned ETF directions, and confidence-guided alignment maps confidently clustered novel samples to unused ETF prototypes, thereby integrating new classes without changing old class anchors [2602.19872]. StructAlign extends the same geometric intuition to multimodal retrieval: a cross-modal ETF alignment loss pulls both text and video features toward category-level ETF prototypes, while Cross-modal Relation Preserving loss distills similarity relations from the previous model to suppress intra-modal drift [2601.20597]. Local Classifier Alignment, by contrast, is local rather than global: it samples Gaussian class features under the current backbone and trains classifiers with a class-conditioned local consistency term so that loss varies little between nearby same-class samples [2603.09888].

A fourth family isolates or routes alignment modules rather than full backbones. CA-SAM inserts a lightweight Alignment Layer between frozen SAM encoder and decoder, learns one such layer per task, and uses a VAE-based task router with OOD fallback to the identity alignment, thereby preserving both prior task adapters and SAM’s zero-shot priors on unseen domains [2511.17201]. ECA makes the alignment module itself the object of continual adaptation in BLIP-2-style image-to-text generation: Mixture of Query adapts task-specific query tokens, Fisher Dynamic Expansion adds new Parallel Adapters only when a Fisher-based conflict score exceeds \(0.5\), and Dictionary Replay preserves the alignment function in embedding space without raw exemplars [2606.12633]. STELLA likewise acts at the level of patch selection rather than full-model replay, using Localized Patch Importance Scoring and Replay-guided Correlation Assessment to perform probabilistic patch selection in continual audio-video pre-training [2310.08204].

A fifth family works in gradient space or data space. OGPSA estimates a low-rank capability subspace from gradients on a small reference set and projects each safety gradient onto its orthogonal complement before the update, so that safety-directed changes minimally perturb general capabilities [2602.07892]. “Continual Safety Alignment via Gradient-Based Sample Selection” instead keeps the model and objective fixed and filters the data: it computes per-sample gradient norms, keeps samples near the median gradient norm after loss pre-filtering, and trains only on those selected samples [2604.17215].

## 4. Evaluation regimes and empirical regularities

Continual alignment is evaluated with strongly domain-specific metrics rather than a single standardized protocol. Multilingual instruction tuning uses weighted F1 on \(L1\), \(L2\), and unseen related \(L3\) languages, with replay-size ablations over \(r \in \{0,1000,10000,100000\}\) [2305.13627]. Online SSL uses Final Accuracy and Average Accuracy under matched Cumulative Backward Passes [2507.10434]. Medical segmentation reports Dice, IoU, HD95, AVG, and BWT [2507.03638]. Continual discovery evaluates All, Old, New, forgetting rate \(\mathcal{M}_f\), and discovery rate \(\mathcal{M}_d\) [2602.19872]. Continual retrieval uses Recall@1/5/10 and BWF [2601.20597]. Safety-preservation work uses VISAGE, ASR, TruthfulQA, BWT, FM, and Max Drop [2604.17215]. This diversity is itself informative: the field measures retention of an aligned relation, but the relation is task-specific.

Despite that heterogeneity, several empirical regularities recur. InstructAlign shows that replay is not merely preservative but enabling: when \(r=0\), performance on \(L1\) drops significantly and newly added \(L2\) languages also often drop, whereas increasing replay improves both retention and adaptation; improvement on \(L2\) strongly correlates with improvement on unseen related \(L3\), with Pearson correlation \(0.96\) [2305.13627]. CLA reports that explicit latent alignment can improve not only end-of-stream representation quality but also early-stage convergence, and even that continuing i.i.d. pretraining from a CLA-based initialization can outperform full i.i.d. training from scratch under the same total budget [2507.10434]. DAKR-HSIC shows the same complementarity pattern in dense prediction: on optic cup segmentation, REKD alone yields Dice AVG/BWT \(75.99/-8.85\), REKD+CNA yields \(77.36/-6.54\), and REKD+CRA+CNA yields \(77.91/-5.73\), indicating that cross-network and cross-representation alignment contribute additively [2507.03638].

Geometric continual alignment also produces consistent retention gains. GOAL lowers average forgetting rate from \(26.37\) to \(10.27\) and raises average discovery rate from \(55.19\) to \(58.38\), with especially large gains in the 10-stage setting where a fixed ETF scaffold accumulates less drift than dynamic classifier schemes [2602.19872]. StructAlign reports the lowest Backward Forgetting across all evaluated settings while improving or matching state-of-the-art retrieval quality [2601.20597]. In continual image captioning, CLICITA’s strongest gains appear on semantic metrics: on the ContCap benchmark, METEOR rises to \(27.90\) from \(11.2\) and \(14.5\) for the best ContCap variants, while forgetting on the RATT split improves from \(-11.22\) for the pretrained base model to \(-7.06\) [2510.06009]. STELLA reports a \(3.69\%\)p relative performance gain in zero-shot retrieval tasks compared to strong continual learning baselines while reducing memory consumption by about \(45\%\) [2310.08204].

## 5. Post-training, safety, and foundation-model interfaces

In language-model post-training, continual alignment has become a way of describing sequential adaptation after instruction tuning or safety tuning. InstructAlign is explicit that, in its setting, alignment does not mean preference alignment or RLHF-style human-value alignment; it means crosslingual alignment of linguistic representations and instruction-following behavior, instantiated through bilingual denoising, machine translation, and crosslingual semantic similarity objectives interleaved with replay from old instruction data [2305.13627]. ECA makes a related architectural claim for pre-trained VLMs: in BLIP-2, the most important object to preserve under sequential adaptation is the cross-modal alignment mechanism, operationalized as the Q-Former, while the visual encoder and LLM remain frozen [2606.12633].

Safety alignment as continual learning is made fully explicit by OGPSA. There, the alignment tax is modeled as catastrophic forgetting under sequential SFT, DPO, or SFT\(\rightarrow\)DPO, and the proposed solution is to estimate a capability subspace from reference gradients and project safety gradients orthogonally to it before updating [2602.07892]. On Qwen2.5-7B-Instruct under SFT\(\rightarrow\)DPO, this recovers SimpleQA from \(0.53\%\) to \(3.03\%\) and IFEval from \(51.94\%\) to \(63.96\%\) while preserving strong safety [2602.07892]. The sample-selection line reaches a similar goal without architectural changes: by filtering high-gradient samples and keeping moderate-gradient ones, it reduces checkpoint-averaged ASR from \(36.7\) to \(10.2\) on Qwen2.5 and from \(44.2\) to \(18.3\) on LLaMA-3.1 while retaining competitive task performance [2604.17215].

“Alignment Dynamics in LLM Fine-Tuning” provides the most explicit dynamical theory in this group. It defines a sequence-level alignment score \(S(\theta)\), derives its first-order update under SFT, and decomposes the change into a Rebound Force, governed by current alignment state and posterior narrowness, and a Driving Force, governed by how the training distribution aligns with outcome-conditioned posteriors over aligned and non-aligned completions [2605.18309]. The same framework predicts a Rehearsal Priming Effect: prior alignment leaves a latent posterior imprint that makes re-exposure unusually effective, a result validated in safety, emergent misalignment, and sentiment settings [2605.18309].

Foundation-model adaptation beyond language exhibits closely related structures. CA-SAM treats continual alignment as alignment of the latent interface between a frozen SAM encoder and frozen decoder; task-specific Alignment Layers are selected by a VAE router, and truly OOD samples are sent through the identity alignment so that frozen SAM handles them directly [2511.17201]. This suggests that continual alignment in foundation models often migrates from full-parameter retention to interface preservation: the bridge, adapter, or router becomes the continual object.

## 6. Limitations, controversies, and open directions

Several limitations recur across the literature. Many methods depend on some form of replay, generated replay, or stored auxiliary structure. InstructAlign assumes access to small amounts of parallel data and to old supervised instruction-tuning data for replay [2305.13627]. ContEA relies on previously predicted trustworthy alignments and on seen neighbors for inductive reconstruction of new entities [2207.11436]. ECA is exemplar-free in raw data but still stores an embedding dictionary, and its authors note that a fixed-size dictionary may become insufficient on very long task sequences [2606.12633]. CA-SAM avoids exemplars but grows linearly with the number of tasks because each task adds one Alignment Layer and one VAE router [2511.17201].

Scalability and task structure remain unresolved. DAKR-HSIC’s Feature Pairing block is practical because the batch size is \(N=4\), so exhaustive search over \(4!=24\) permutations is feasible; the same design is not obviously scalable to larger batches or 3D medical imaging [2507.03638]. GOAL assumes the number of novel classes per stage is given or estimated, and its ETF capacity is fixed in advance, which leaves adaptive ETF expansion as future work [2602.19872]. Several settings are only weakly sequential in the long-horizon sense: InstructAlign introduces seven \(L2\) languages together in one adaptation phase rather than one-by-one, and many benchmarks still rely on known stage orderings or bounded numbers of tasks [2305.13627].

Theory is also uneven. DualHSIC states explicitly that a stronger theoretical link between HSIC optimization and catastrophic forgetting is still missing [2305.00380]. Alignment Dynamics provides a much sharper account, but only under the Relatively Stable Kernel assumption and mainly for SFT rather than DPO or RLHF [2605.18309]. The sample-selection method currently computes exact per-sample gradients, which the paper reports as about \(1.5\times\) baseline cost and \(51\%\) additional training overhead, leaving cheaper approximations as future work [2604.17215].

A broader interpretive issue is that “continual alignment” is not yet a unified subfield with a single benchmark, metric, or ontology. Some papers use alignment to denote task-invariant representation sharing, some use it for multimodal geometry, some for crosslingual instruction-following, and some for safety preservation. This suggests a productive but still unsettled research area. A plausible implication is that future work will increasingly focus on interfaces between these usages: whether safety alignment can profit from geometric or replay-conditioned latent alignment, whether multimodal continual alignment should be treated as posterior-structure preservation rather than only embedding matching, and whether foundation-model adaptation should preserve not merely old accuracy but the specific relational structure that makes new knowledge attachable to existing behavior.

Source: https://www.emergentmind.com/topics/continual-alignment