Cross-Subspace Knowledge Alignment & Aggregation

• CKAA is a framework that aggregates transferable features from diverse subspaces and aligns them with target representations to enhance continual learning.
• It employs dual-level knowledge alignment and task-confidence-guided mixture strategies to mitigate feature subspace misalignment in PEFT-based systems.
• CKAA’s principles have been applied in EMG, federated learning, multimodal editing, and medical LLM fine-tuning to improve robustness under heterogeneous conditions.

Searching arXiv for the cited papers and closely related work to ground the article. Cross-subspace Knowledge Alignment and Aggregation (CKAA) denotes a design principle for learning under heterogeneity, and, in the narrow sense, the name of a continual-learning framework proposed for parameter-efficient fine-tuning (PEFT)-based continual learning. In the continual-learning formulation, CKAA addresses “feature subspace misalignment from independently trained sub-modules,” which can produce ambiguous decisions under misleading task-ids, by combining “Dual-level Knowledge Alignment (DKA)” with “Task-Confidence-guided Mixture of Adapters (TC-MoA)” (He et al., 13 Jul 2025). In a broader technical usage reflected across recent work, the same phrase describes systems that first aggregate information across subjects, clients, variants, or tasks into a shared representation and then align that representation to a target subspace, decision space, or orthogonal parameter-update space. This broader pattern appears in EMG cross-subject adaptation (Colot et al., 2023), vertical federated learning (Yao et al., 7 Aug 2025), robust multimodal knowledge editing (Wang et al., 22 May 2026), and medical LLM fine-tuning (Liao et al., 2024).

1. Definition and conceptual scope

At its most general, CKAA couples two operations. “Aggregation” pools or compacts transferable structure across heterogeneous sources into a shared representation. “Alignment” then enforces compatibility between that shared representation and the subspace relevant to inference, transfer, or editing. The object being aggregated varies by domain: EMG windows across subjects, embeddings across VFL clients, adversarial latent variants within a knowledge unit, or low-rank LoRA updates across medical tasks. The object being aligned also varies: a target subject’s PCA subspace, a decision subspace, an edit-layer latent subspace, or an approximately orthogonal alignment subspace (Colot et al., 2023).

The literature therefore uses “subspace” in several mathematically distinct senses. In cross-subject EMG, it refers to low-dimensional linear subspaces estimated by PCA and compared by principal angles on the Grassmann manifold. In X-VFL, the “shared decision subspace is the output space of the top model $h$ fed with a common-dimensional embedding,” and alignment is imposed directly on decision outputs rather than raw features. In ASAM for multimodal knowledge editing, the relevant subspace is the low-rank structure of edit-layer hidden states induced by adversarially generated latent variants. In MedCare, the two subspaces are “the model’s parameter update space,” split into a knowledge subspace and an alignment subspace that are constrained to be nearly orthogonal (Yao et al., 7 Aug 2025).

The following summary organizes the principal CKAA-style instantiations described in the cited works.

Setting	Aggregation	Alignment
Continual learning	task-specific knowledge from relevant sub-modules	DKA across different subspaces; robust global classifier through feature simulation
EMG cross-subject classification	shared low-dimensional subspace across source subjects	alignment from shared source subspace to target subject subspace
Vertical federated learning	XCom with average aggregation	DS-Align in the decision subspace
Multimodal knowledge editing	multi-variant spectral aggregation over adversarial variants	low-rank alignment at the edit layer
Medical LLM fine-tuning	Knowledge Aggregator in FFNs	orthogonality-constrained alignment module

This suggests that CKAA is not a single algorithmic template with fixed operators. Rather, it is a recurring architecture-level pattern in which transfer robustness depends on separating a shared, task-relevant component from nuisance variation and then reconciling source and target representations in a space chosen for stability or controllability (Wang et al., 22 May 2026).

2. Canonical aggregation–alignment pattern

A canonical CKAA pipeline is explicit in the EMG formulation. Let subject $i$ provide labeled EMG windows $X_i \in \mathbb{R}^{n_i \times d}$ with labels $y_i \in \{1,\dots,C\}$. Aggregation builds a source-side shared basis either by pooled PCA or by per-subject PCA followed by Grassmannian averaging. In the pooled PCA form, source data are stacked as

$$X_s = [X_1; X_2; \dots; X_{S_s}] \in \mathbb{R}^{n_s \times d},$$

an SVD is computed, and the shared basis is $U_{\text{shared}} := V_{s,k}$. In the Grassmannian form, each subject contributes an orthonormal basis $V_{i,k}$, and a Karcher mean or an extrinsic average followed by re-orthonormalization yields the shared subspace. The target subject contributes $V_{t,k}$, and alignment is performed either by orthogonal Procrustes,

$$R^* = \arg\min_{R \in \mathbb{R}^{k \times k},\, R^\top R = I} \|U_{\text{shared}}R - V_{t,k}\|_F^2,$$

or by the standard subspace-alignment transform

$$M := U_s^\top U_t,\qquad Z_s = X_s U_s M,\qquad Z_t = X_t U_t.$$

Classification is then learned on $i$0 and applied to $i$1 (Colot et al., 2023).

The EMG case makes the operational logic explicit. The paper states that “an accurate generalization based on pooling multiple subjects is hardly achievable,” whereas it is possible “to improve the cross-subject estimation by identifying a robust low-dimensional subspace for multiple subjects and aligning it to a target subject.” The methodological explanation ties this to inter-subject variability, anatomy, physiology, electrode placement shifts, and varying skin-electrode impedance, all of which induce domain shift. The data regime is also concrete: 14 participants, 4 signs, 8 Cometa Pico electrodes on the right forearm, a sampling rate of 2000 Hz, non-overlapping 400 ms sliding windows during held postures, and approximately 8400 windows total (Colot et al., 2023).

The same two-stage structure reappears in other domains, but with different mathematical operators. In X-VFL, aggregation is average embedding plus feature completion; in ASAM, aggregation is the singular value structure of stacked hidden states; in MedCare, aggregation is a dedicated LoRA-based Knowledge Aggregator. Alignment is correspondingly implemented as decision-consistency losses, low-rank edit-layer collapse, or orthogonality penalties (Yao et al., 7 Aug 2025).

3. Continual-learning formulation of CKAA

In the strict terminological sense, CKAA is introduced as “Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning” (He et al., 13 Jul 2025). The abstract situates the method in PEFT-based continual learning, where a unique sub-module is typically allocated for each task and a task recognizer selects the appropriate sub-modules for testing images. The reported failure mode is that independently trained sub-modules induce feature subspace misalignment, so misleading task-ids can produce ambiguous decisions. CKAA addresses this through two named components.

The first component, Dual-level Knowledge Alignment, “align[s] intra-class feature distributions across different subspaces and learn[s] a robust global classifier through a feature simulation process.” The second, Task-Confidence-guided Mixture of Adapters, is “a robust inference scheme that adaptively aggregates task-specific knowledge from relevant sub-modules based on task-confidence scores, avoiding overconfidence in misleading task-id predictions.” The abstract further states that “Extensive experiments demonstrate that CKAA outperforms existing PEFT-based CL methods” (He et al., 13 Jul 2025).

Because the supplied material for this paper contains the abstract but not the full method section, the public summary establishes the framework’s conceptual center more clearly than its exact algebra. Even so, the naming of DKA and TC-MoA is informative. DKA indicates that alignment occurs at more than one representational level, while TC-MoA indicates that aggregation at inference is not a hard routing decision but a confidence-weighted mixture over task-specific sub-modules. A plausible implication is that CKAA’s distinctive contribution within continual learning is not merely adapter averaging, but robustness to subspace mismatch under task-recognition error.

4. Decision-space, latent-space, and orthogonal-subspace variants

X-VFL provides an explicit decision-space instantiation. The framework addresses two stated VFL problems: “the requirement for perfectly aligned data samples across all clients” and “the requirement for joint collaborative inference/prediction involving all clients.” Its aggregation module, Cross Completion (XCom), reconstructs missing local features from other clients’ embeddings, while average aggregation ensures that “the decision subspace becomes identical in dimensionality for each client, enabling independent inference.” Alignment is realized by Decision Subspace Alignment (DS-Align), with two losses:

$i$2

and

$i$3

with overall objective

$i$4

The paper reports convergence theorems with an $i$5 convergence rate for SGD-type algorithms and an $i$6 rate for PAGE-type algorithms, and states “achieving a 15% improvement in accuracy on the image CIFAR-10 dataset and a 43% improvement on the medical MIMIC-III dataset” (Yao et al., 7 Aug 2025).

ASAM, from robust multimodal knowledge editing, provides a latent-space and edit-layer variant. Knowledge is formalized as “knowledge units” $i$7 with a shared target $i$8. Latent Adversarial Robustification generates adversarial yet semantically coherent joint latent variants by solving

$i$9

At the edit layer, normalized hidden states are stacked as $X_i \in \mathbb{R}^{n_i \times d}$0, and Rank-Constrained Subspace Learning imposes near-rank-1 alignment through

$X_i \in \mathbb{R}^{n_i \times d}$1

where $X_i \in \mathbb{R}^{n_i \times d}$2 are singular values of $X_i \in \mathbb{R}^{n_i \times d}$3. The paper states that robust editing corresponds to collapsing all adversarial variants onto a shared principal semantic direction at the edit layer, and reports consistent gains in generality while preserving reliability and locality; for example, on BLIP2, WISE to ASAMWISE raises E-VQA generality from 84.80 to 93.97 with reliability unchanged at 99.00 (Wang et al., 22 May 2026).

MedCare offers an orthogonal-subspace instantiation in medical LLM fine-tuning. It decouples a “Knowledge Aggregator” (KA) and an alignment module “Align” across FFN LoRA updates, defining a knowledge subspace $X_i \in \mathbb{R}^{n_i \times d}$4 from $X_i \in \mathbb{R}^{n_i \times d}$5 and an alignment subspace $X_i \in \mathbb{R}^{n_i \times d}$6 from $X_i \in \mathbb{R}^{n_i \times d}$7. Orthogonality is enforced by

$X_i \in \mathbb{R}^{n_i \times d}$8

and the downstream alignment objective is

$X_i \in \mathbb{R}^{n_i \times d}$9

The implementation uses a two-stage pipeline: Miscellaneous Knowledge Aggregation followed by Downstream Alignment, with the Noise Aggregator dropped after stage 1. The paper states that MedCare is designed to achieve state-of-the-art performance on over 20 medical tasks and reports, among other results, a 69.69 average for MedCare-14B on medical knowledge exams and CCTE averages of 4.38 for MedCare-7B and 4.39 for MedCare-14B (Liao et al., 2024).

5. Representative application domains

The application space of CKAA-style methods is unusually broad because the central problem—heterogeneous sources that share only a partial task-relevant structure—recurs across sensing, distributed learning, editing, and instruction tuning.

In biosignal processing, the EMG formulation frames cross-subject generalization as the need to transfer from multiple source subjects to a held-out target subject without subject-specific calibration or with minimal calibration. The main limitation of EMG gesture recognition is the long calibration time needed for new users, and the paper’s central claim is that naïve pooling is ineffective whereas a robust low-dimensional source subspace, aligned to the target, improves estimation (Colot et al., 2023).

In vertical federated learning, CKAA is operationalized for non-aligned samples with partially missing features and for “locally independent inference on a single client.” The empirical profile emphasizes robustness under missingness: under $y_i \in \{1,\dots,C\}$0 on UTKFace in independent inference, X-VFL’s accuracy “decreases by less than 0.5% from its $y_i \in \{1,\dots,C\}$1 performance, whereas baselines drop by ~30%,” and on the same dataset the independent–collaborative gap is “<0.2% gap (independent 88.15% vs collaborative 88.32%), while baselines exhibit >20% drops” (Yao et al., 7 Aug 2025).

In multimodal knowledge editing, CKAA-style alignment is used to propagate an edit beyond the original prompt pair to “semantically equivalent multimodal inputs.” The core issue is limited generality caused by “biased anchoring to individual samples in high-dimensional multimodal spaces.” ASAM therefore constructs a local semantic neighborhood with adversarial variants and aligns that neighborhood to a principal direction at the edit layer. Sequential editing results underscore the point: on E-VQA at $y_i \in \{1,\dots,C\}$2, WISE generality 34.90 becomes ASAM 50.90, while T-Loc. rises from 88.76 to 99.23 (Wang et al., 22 May 2026).

In medical LLMs, CKAA appears as an explicit decoupling of “Clinical Alignment and Knowledge Aggregation.” The first stage encodes broad medical knowledge with KA while letting a Noise Aggregator absorb task-specific and potentially harmful alignment signals; the second stage introduces Align, optimized toward “an orthogonal direction to the knowledge space to mitigate knowledge forgetting.” An especially concrete diagnostic appears in the MoLoRA ablation: “On CEval, vanilla MoLoRA yields 45.00 accuracy, while dropping NA after MKA boosts CEval to 60.98” (Liao et al., 2024).

6. Limitations, misconceptions, and research directions

A common misconception is that CKAA names a single, standardized algorithm. The record is narrower and more heterogeneous. The continual-learning paper introduces CKAA explicitly as a named framework (He et al., 13 Jul 2025), but other works either instantiate the same principle under different names or are described as doing so retrospectively. X-VFL states that “The paper does not reference CKAA explicitly as prior work; theoretically, CKAA’s core ideas map directly onto X-VFL’s modules.” ASAM likewise “does not use the term ‘CKAA,’ but its framework (ASAM) is conceptually equivalent” (Yao et al., 7 Aug 2025).

Another misconception is that “alignment” always means linear feature matching. The surveyed instantiations contradict that view. EMG uses PCA subspaces, principal angles, and Procrustes or subspace-alignment transforms. X-VFL aligns decision outputs through similarity losses in the top-model output space. ASAM enforces low-rank agreement of edit-layer hidden states through a singular value–based objective. MedCare aligns low-rank update directions by pushing one adapter family toward the orthogonal complement of another (Colot et al., 2023).

The limitations are correspondingly domain specific. EMG alignment is linear and “severe nonlinear shifts (e.g., large placement changes) may require nonlinear or deep alignment.” X-VFL provides no explicit secure aggregation, encryption, or differential privacy, so “formal privacy guarantees are not provided.” ASAM depends on the adversarial budget $y_i \in \{1,\dots,C\}$3, temperature $y_i \in \{1,\dots,C\}$4, weight $y_i \in \{1,\dots,C\}$5, and number of variants $y_i \in \{1,\dots,C\}$6; too large an $y_i \in \{1,\dots,C\}$7 may break semantic coherence, and rank-1 pressure may be insufficient under heavy distribution shift. MedCare still incurs a non-zero “alignment tax,” and its router instability motivates dropping the Noise Aggregator after stage 1 rather than retaining a more adaptive expert-selection mechanism (Yao et al., 7 Aug 2025).

Across these limitations, a unifying research direction is visible. This suggests that CKAA is most promising when the shared structure is real but only partially observable, and when the alignment space is chosen to suppress nuisance variation without erasing task signal. Current extensions already point in that direction: learned weighted averaging beyond simple averages in VFL, stronger privacy mechanisms for embedding sharing, richer completer architectures, additional decision-level metrics, adaptive or projection-based orthogonality in medical LLMs, and broader theoretical treatment of generalization under missingness or subspace shift (Liao et al., 2024).