Decision Subspace Alignment in X-VFL

• Decision Subspace Alignment (DS-Align) is a mechanism that aligns decision outputs from local, reconstructed, and collaborative embeddings in vertical federated learning.
• It ensures that locally computed predictions approximate collaboratively trained outputs, enabling accurate inference even with missing or non-aligned features.
• Empirical results indicate significant improvements, with up to 15-43% accuracy gains and minimal performance gaps between independent and collaborative inference.

Decision Subspace Alignment (DS-Align) is a decision-space consistency mechanism introduced as a core module of the X-VFL framework for vertical federated learning with non-aligned samples, partially missing features, and locally independent inference (Yao et al., 7 Aug 2025). In that formulation, multiple clients hold disjoint feature subsets for the same prediction task, and DS-Align is used after local embeddings are formed, not at the raw-feature level. Its central function is to align decision outputs induced by local embeddings, reconstructed embeddings, and collaborative averaged embeddings so that a client can later perform inference with only its own information while remaining close to the behavior of the jointly trained system. The term “decision subspace” is operational rather than geometric in the strict linear-algebraic sense: it denotes the output space induced by the shared top model $h$, and the method does not introduce a separate projector, orthogonality constraint, or explicit basis for that space (Yao et al., 7 Aug 2025).

1. Problem setting and conceptual scope

In X-VFL, client $i$ holds local data

$\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$

where $\bm{X}_i^{al}$ are aligned samples and $\bm{X}_i^{nl}$ are non-aligned samples with at least one client missing features (Yao et al., 7 Aug 2025). The framework is designed to address two limitations of standard vertical federated learning: the reliance on perfectly aligned samples during training, and the requirement that all clients participate during inference.

X-VFL changes the aggregation architecture by replacing concatenation with averaging,

$$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$

so that each local embedding $\bm{E}_i$ has the same dimensionality as the collaborative representation (Yao et al., 7 Aug 2025). Dimensional compatibility alone, however, does not ensure behavioral compatibility: a shared top model $h$ can accept $\bm{E}_i$ as input, but $h(\bm{E}_i)$ may still differ substantially from $i$0. DS-Align is introduced precisely to reduce that gap.

Within X-VFL, Cross Completion (XCom) reconstructs missing features for non-aligned samples, whereas DS-Align aligns the resulting local, reconstructed, and global signals in decision space (Yao et al., 7 Aug 2025). The framework therefore separates two problems: recovering missing information and making local-only decisions behave like collaborative decisions. This division is important because DS-Align is not a feature imputation method; it is a consistency regularizer applied after representation formation and after the shared predictor $i$1.

A recurrent misconception is to treat DS-Align as a classical subspace-alignment method over PCA bases. In X-VFL, it is not. The aligned objects are outputs of the decision head $i$2, such as $i$3, $i$4, and $i$5, rather than raw features or explicit linear subspace bases (Yao et al., 7 Aug 2025).

2. Exact formulation in X-VFL

For two clients $i$6, local embeddings are defined by bottom models

$i$7

XCom reconstructs missing features from the other client’s embedding,

$i$8

and re-encodes them as

$i$9

The shared top model $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$0 maps local, reconstructed, or averaged embeddings to decision outputs (Yao et al., 7 Aug 2025).

The supervised decision loss used by X-VFL is

$\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$1

where $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$2 is the classification loss, such as cross-entropy (Yao et al., 7 Aug 2025).

DS-Align contributes two additional terms. The first matches reconstructed and original decisions: $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$3 where $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$4 is now a similarity loss, such as mean square error (Yao et al., 7 Aug 2025). This term enforces decision consistency between completed information and the corresponding true local information.

The second matches local and collaborative decisions: $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$5 This term is the direct mechanism by which locally independent inference is enabled, because it explicitly pushes each client’s local prediction toward the collaboratively trained prediction (Yao et al., 7 Aug 2025).

The full objective is

$\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$6

In this construction, DS-Align is label-informed but not directly label-supervised: Eqs. for $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$7 and $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$8 compare outputs to outputs rather than outputs to $\bm{X}_i = (\bm{X}_i^{al}, \bm{X}_i^{nl}),$9 (Yao et al., 7 Aug 2025).

For $\bm{X}_i^{al}$0 clients, the generalization is

$\bm{X}_i^{al}$1

$\bm{X}_i^{al}$2

This preserves the same two principles: reconstructed-vs-original alignment and local-vs-global decision alignment (Yao et al., 7 Aug 2025).

3. Architectural role and inference mechanism

DS-Align is integrated into training after embeddings have been produced and after the top model $\bm{X}_i^{al}$3 has mapped them into decision outputs (Yao et al., 7 Aug 2025). The shared objects in X-VFL are embeddings, not raw features. These embeddings are consumed both by other clients’ completion modules and by the server-side predictor.

The training flow is structurally simple. Each client computes a local embedding $\bm{X}_i^{al}$4. If features are missing, XCom reconstructs them and re-encodes the result as $\bm{X}_i^{al}$5. The top model receives local embeddings, collaborative averages such as $\bm{X}_i^{al}$6, and partially completed averages such as $\bm{X}_i^{al}$7. Backpropagation then jointly updates the bottom models $\bm{X}_i^{al}$8, the completion modules $\bm{X}_i^{al}$9, and the top model $\bm{X}_i^{nl}$0 under the combined task and DS-Align losses (Yao et al., 7 Aug 2025).

The conceptual importance of DS-Align lies in the fact that averaging by itself guarantees only shape compatibility. A client can technically apply $\bm{X}_i^{nl}$1 to $\bm{X}_i^{nl}$2, because $\bm{X}_i^{nl}$3 and $\bm{X}_i^{nl}$4 have the same dimensionality, but without additional regularization there is no reason for the predictions to agree. DS-Align introduces that missing compatibility condition at the level that actually matters for deployment: the decision output of $\bm{X}_i^{nl}$5.

At inference time, a client can retain its local bottom model $\bm{X}_i^{nl}$6, optionally its XCom module $\bm{X}_i^{nl}$7 when missing-feature reconstruction is needed, and the shared top model $\bm{X}_i^{nl}$8. Prediction then uses either

$\bm{X}_i^{nl}$9

or

$$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$0

This makes DS-Align the bridge from collaborative training to local-only prediction (Yao et al., 7 Aug 2025). The paper does not provide a theorem proving correctness of local inference from DS-Align alone, but it states this as the intended mechanism of the framework.

4. Optimization properties, empirical support, and limitations

The theoretical analysis in X-VFL studies optimization of the full nonconvex objective containing DS-Align,

$$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$1

under average smoothness and bounded-variance assumptions (Yao et al., 7 Aug 2025). For SGD-type updates, the paper proves an $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$2 convergence rate. For PAGE-type variance-reduced updates, it proves an $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$3 rate. Because each update step requires one communication round, the corresponding communication complexities are $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$4 for SGD-type methods and $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$5 for PAGE-type methods (Yao et al., 7 Aug 2025).

The empirical case for DS-Align is indirect but substantial. X-VFL reports that it “significantly outperforms existing methods,” including a 15% improvement in accuracy on CIFAR-10 and a 43% improvement on MIMIC-III in settings with partially missing features and locally independent inference (Yao et al., 7 Aug 2025). The most diagnostic evidence for DS-Align itself comes from independent-inference behavior. On UTKFace at missing rate $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$6, X-VFL’s independent inference is 88.15% and collaborative inference is 88.32%, a gap of less than 0.2%, whereas baselines show drops of over 20% when moving from collaborative to independent inference (Yao et al., 7 Aug 2025). The paper also reports that at the same missing rate X-VFL decreases by less than 0.5% relative to its own $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$7 performance, which is consistent with the intended effect of $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$8.

Additional evidence is qualitative. Figure 1, presented as the “Effect of DS-Align,” shows that vanilla training yields a poorly defined decision boundary with overlap and misclassification, while X-VFL yields clearer class separation (Yao et al., 7 Aug 2025). This is not an isolated ablation, but it is the paper’s most direct visualization of DS-Align in action.

The limitations are equally important. X-VFL does not provide a clean DS-Align-only ablation, so its isolated numerical contribution is not separately quantified (Yao et al., 7 Aug 2025). The “decision subspace” is not given a deeper geometric formalization beyond the output space of $$\bm{E}_{\text{avg}} = \mathrm{Avg}(\bm{E}_1,\bm{E}_2,\dots,\bm{E}_k),$$9. Success also depends on the quality of local embeddings $\bm{E}_i$0, feature completion $\bm{E}_i$1, and the shared head $\bm{E}_i$2; poor reconstruction can make $\bm{E}_i$3 align low-quality reconstructed decisions to true ones. These limitations distinguish DS-Align in X-VFL from methods that define explicit orthogonal or low-rank subspaces with stronger structural constraints.

5. Relation to earlier feature-space subspace alignment

DS-Align emerged against a substantial background of subspace alignment methods that operate on feature geometry rather than on decision outputs. The classical reference is “Subspace Alignment for Domain Adaptation,” which represents source and target domains by PCA bases $\bm{E}_i$4 and $\bm{E}_i$5, then solves

$\bm{E}_i$6

with the aligned source basis $\bm{E}_i$7 (Fernando et al., 2014). That formulation is explicitly feature-subspace alignment: domains are aligned in covariance-derived coordinate systems, and classification follows afterward.

Subsequent work made feature-subspace alignment increasingly task-aware without moving fully into decision space. “Subspace Alignment Based Domain Adaptation for RCNN Detector” constructs class-specific localized proposal subspaces for object detection and aligns them across source and target domains; because target proposals are selected by detector scores, the method is detector-conditioned, but the aligned objects are still proposal CNN features rather than detector outputs or boundary geometry (Raj et al., 2015). “SALT: Subspace Alignment as an Auxiliary Learning Task for Domain Adaptation” preserves the classical linear subspace objective but optimizes the alignment map $\bm{E}_i$8 with classifier-derived entropy and class-balance losses, making the alignment decision-informed while remaining a feature-space method (Thopalli et al., 2019). “Revisiting Deep Subspace Alignment for Unsupervised Domain Adaptation” pushes this further by computing global source and target subspaces from deep latent features and refining the alignment map through an alternating primary-auxiliary procedure driven by source classification, target entropy, and class-balance objectives, yet the aligned entities remain representation subspaces $\bm{E}_i$9 rather than decision outputs (Thopalli et al., 2022).

Against this lineage, DS-Align in X-VFL is distinctive because the alignment terms are defined directly on $h$0. The method does not estimate source and target subspace bases, does not solve a Frobenius alignment between basis matrices, and does not backpropagate through PCA or SVD (Yao et al., 7 Aug 2025). A cautious summary is that DS-Align can be viewed as a shift from task-conditioned feature-subspace alignment to explicit decision-output consistency.

6. Neighboring subspace paradigms and broader interpretation

Several later or adjacent lines of work clarify what DS-Align is and is not. In test-time adaptation for vision-LLMs, SubTTA aligns a visual principal subspace to a textual semantic anchor by minimizing chordal distance and then projects aligned visual features into the textual task subspace; the aligned object is a multimodal semantic span anchored by class text embeddings, which makes it closer to semantic or decision-relevant subspace purification than to classical PCA-only domain alignment (Zeng et al., 13 Jan 2026). In regression test-time adaptation, Significant-subspace Alignment (SSA) first detects a source feature subspace by PCA and then weights its dimensions according to readout relevance $h$1, explicitly arguing that only an output-significant subspace should be aligned strongly (Adachi et al., 2024). Both methods suggest that subspace alignment becomes more decision-like when the selected span is tied to task semantics or output sensitivity rather than global variance alone.

Other neighboring methods operate on different objects altogether. OrthAlign decomposes parameter-update space into orthogonal subspaces so that different alignment objectives optimize in non-interfering directions; its subspaces are LoRA-update or curvature subspaces, not output-space decision embeddings (Lin et al., 29 Sep 2025). DSCA decomposes fused vision-language representation space into approximately orthogonal concept-specific semantic subspaces for continual editing, enforcing non-interfering edits through routed basis-residual interventions (Das et al., 9 Apr 2026). “Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptation” identifies semantically grounded low-rank subspaces from sparse autoencoder features and uses them to initialize LoRA output subspaces for safety alignment, making the alignment subspace explicit and interpretable but still situated primarily in activation and parameter-update space rather than in the output space of a predictor $h$2 (Wang et al., 29 Dec 2025).

These comparisons delimit the term. Decision Subspace Alignment in the narrow sense refers most concretely to the X-VFL module that aligns decision outputs across local, reconstructed, and collaborative representations (Yao et al., 7 Aug 2025). More broadly, the surrounding literature suggests a family resemblance among methods that try to isolate the task-relevant or behavior-relevant span of variation. A plausible implication is that “decision subspace” has become a useful organizing metaphor across federated learning, domain adaptation, test-time adaptation, model editing, and safety alignment, even though the mathematical object being aligned may be an output space, a semantic span, a concept subspace, or a parameter-update subspace depending on the framework.