DSCA: Dynamic Subspace Concept Alignment for Lifelong VLM Editing

Abstract: Model editing aims to update knowledge to add new concepts and change relevant information without retraining. Lifelong editing is a challenging task, prone to disrupting previously learned concepts, especially for Vision LLMs (VLMs), because sequential edits can lead to degraded reasoning and cross modal misalignment. Existing VLM knowledge editing methods based on gated adapters, activation edits, and parameter merging techniques address catastrophic forgetting seen in full fine tuning; however, they still operate in the shared representation space of the VLM, where concepts are entangled, so edits interfere with other non relevant concepts. We hypothesize that this instability persists because current methods algorithmically control edits via optimization rather than structurally separating knowledge. We introduce Dynamic Subspace Concept Alignment (DSCA) which by design mitigates this limitation by decomposing the representation space into a set of orthogonal semantic subspaces and proposing edits only in those transformed spaces. These subspaces are obtained through incremental clustering and PCA on joint vision language representations. This process structurally isolates concepts, enabling precise, non interfering edits by turning isolation from a soft training objective into an architectural property. The surgical edits are guided by a multi term loss function for maintaining task fidelity, edit locality, and cross modal alignment. With the base model frozen, our method achieves 98 percent single edit success, remains over 95 percent after 1000 sequential edits, lowers hallucination by 3 to 5 percent, and achieves the best backward transfer (BWT) scores on continual instruction tuning benchmarks. Extensive experiments demonstrate DSCA state of the art stability and knowledge retention capability in continual lifelong editing across various datasets and benchmarks.

• The paper introduces DSCA, a novel framework that isolates concept-specific edits using orthogonal subspaces to prevent catastrophic forgetting in VLMs.
• DSCA leverages Dynamic Structured Alignment Modules and residual PCA to achieve >98% edit success and dramatically reduce vision hallucination rates.
• Empirical results show DSCA maintains >95% reliability over 1,000 edits, highlighting its robust lifelong learning performance and minimal interference.

Dynamic Subspace Concept Alignment for Lifelong Vision-LLM Editing

Motivation and Problem Setting

Contemporary Vision-LLMs (VLMs) deployed in long-lived, real-world applications must undergo continual knowledge evolution—incorporating new facts, correcting errors, or adapting to user-specific domains—without global retraining. A central challenge in VLM editing is avoiding catastrophic forgetting and unintentional interference: when one concept is altered, unrelated concepts are often distorted due to coupled interactions within the shared high-dimensional latent manifold. Existing solutions (e.g., parameter-efficient adapters, activation edits, and parameter merging) restrict updates but still operate within entangled representations, and thus do not guarantee locality.

A driving hypothesis of this work is that this problem arises not merely from algorithmic limitations, but from a deeper architectural shortcoming: effective lifelong editing requires structural disentanglement of concepts at the representation level. Model modifications should occur within concept-specific, isolated subspaces instead of the monolithic space underpinning all concepts.

DSCA: Subspace-Based Knowledge Editing Architecture

Dynamic Subspace Concept Alignment (DSCA) is introduced to resolve these issues by architecting explicit orthogonality and dynamic routing at the representation level for VLM knowledge editing.

Figure 1: (a) Concepts initially are isolated; (b) global fine-tuning perturbs all, causing interference; (c) LoRA/adapters reduce but do not eliminate coupling; (d) DSCA structurally isolates edits to orthogonal subspaces, maintaining the integrity of unrelated concepts.

Concept Discovery and Subspace Decomposition

DSCA maintains an evolving set of concept clusters using online assignment and prototype adaptation. For each concept, an orthogonal low-rank semantic subspace is identified and regularly refined via residual Incremental PCA over representations belonging to that concept. These subspaces serve as the actionable support for edits, turning the separation of concepts from an optimization problem into an architectural invariant.

Dynamic Structured Alignment Modules and Residual Gated Intervention

Each subspace is paired with a Dynamic Structured Alignment Module (DSAM), which learns an affine residual within the semantic basis, further modulated via input-dependent sparse gating. This design ensures that interventions are:

• Sparse (active only for relevant inputs)
• Localized (confined to the concept subspace)
• Composable (multiple modules can be blended for ambiguous cases)

Routing is achieved in two hierarchical stages (coarse visual, fine fused), drastically improving computational efficiency and minimizing interference.

Multi-Objective Optimization for Editing

The loss guiding DSCA is a combination of:

• Task Fidelity: Causal language modeling loss for edited examples
• Locality: InfoNCE-based contrastive distillation to preserve unedited knowledge
• Cross-Modal Alignment: Regularization to preserve vision-text consistency
• Gate Sparsity: L1 regularization on gating weights to ensure minimal activation outside targeted concepts

Prototypes and subspace bases are updated with data-driven schedules, while the DSAM parameters are trained by gradient descent.

Empirical Results

Single-Edit and Sequential Editing

DSCA consistently yields state-of-the-art scores (>98% average) on benchmarked VLM editing datasets (E-VQA/E-IC). For sequential lifelong editing, DSCA substantially reduces degradation after 1,000 edits—maintaining >95% reliability and near-perfect locality, outperforming adapter-based and direct fine-tuning baselines by non-trivial margins.

Continual Learning Performance

On CoIN, DSCA achieves an average Backward Transfer (BWT) of -9.37, a marked reduction in forgetting compared to other parameter-isolating methods (e.g., -19.45 for PAM), without sacrificing plasticity.

Retention and Hallucination

DSCA matches or exceeds base model performance on core VLM evaluation metrics (MME, MM-Vet, VQA-v2, TextVQA, COCO-CIDEr) and further reduces vision hallucination rates (CHAIR-H 15.9 vs 21.1 for LiveEdit), demonstrating its robustness and safety.

Analysis: Non-Interference and Routing Sparsity

Ablation studies and diagnostic analyses confirm that edit locality is directly connected to the enforced orthogonality of subspaces. As mean pairwise subspace overlap increases, forgetting grows almost linearly (Pearson $r \approx 0.94$).

Figure 2: (a) DSCA maintains mean subspace overlap at $3 \times 10^{-3}$ over 1,000 edits—essential for non-interference; (b) Forgetting increases with subspace overlap; (c) Residualized PCA matches global orthonormalization in retention.

Interventions remain highly sparse: >95% of routing weights are near zero, with only ~3 DSAMs firing per input.

Figure 3: (a) Histogram of routing weights, over 95% $<$0.05; (b) Tuning the sparsity coefficient, the operational point achieves high selectivity without accuracy loss.

Empirical projections of concept representations further illustrate the compact, well-separated nature of DSCA's concept subspaces.

Figure 4: t-SNE visualization; projected representations form compact, isolated clusters per concept, showing structural locality.

Theoretical Characterization

A formal analysis underpins DSCA's effectiveness: with strictly orthogonal subspaces, cross-concept edits provably produce zero interference, and the forgetting bound grows only as $\sqrt{\varepsilon}$ for mean overlap $\varepsilon$. This yields a deep architectural guarantee rather than relying on soft training objectives.

Implications and Future Directions

The adoption of activation-space subspace decomposition transitions the prevention of forgetting from an optimization-theoretic property to an architectural constraint, opening the door to truly modular, extensible VLMs. DSCA's approach of hybrid continuous (gradient) and discrete (data-driven basis) updates situates it as a plausible foundation for further compositional, interpretable editing in future multimodal architectures.

However, scaling to a very large number of concepts will demand improvements in subspace management (e.g., non-linear subspaces, subspace sharing, or advanced routing for ambiguous concepts). Future work should explore richer geometric structures (e.g., manifold subspaces, hypernet-based edits), application to multi-modal and embodied agent domains, and tighter backbone integration for maximal stability and capacity.

Conclusion

DSCA establishes a rigorous framework for intervention in VLMs through semantic subspace decomposition and sparse, gated residual modules. The method achieves high knowledge retention, edit success, locality, and safety—a confluence unattained by prior approaches. DSCA's theoretical and empirical results demonstrate that structural modularity at the activation level is a key paradigm for robust lifelong knowledge editing in large-scale multimodal systems.

---

Cite as: "DSCA: Dynamic Subspace Concept Alignment for Lifelong VLM Editing" (2604.07965).