Vertical Federated Learning

• Vertical Learning is a collaborative machine learning paradigm where datasets are partitioned by features, enabling privacy-preserving multi-view and split learning.
• It employs architectures with local feature processing, secure embedding aggregation, and differential privacy to mitigate exposure risks.
• Vertical Learning facilitates cross-domain collaboration in finance, healthcare, and advertising by overcoming data silos through secure entity alignment and distributed optimization.

Vertical learning, in the sense used in contemporary federated-learning research, is essentially understood as Vertical Federated Learning (VFL): a collaborative machine-learning paradigm in which multiple parties jointly train a model on shared or aligned samples whose features are vertically partitioned across parties, without directly sharing raw data (Ye et al., 2024). In this setting, the same entities—such as users, patients, or customers—appear in multiple datasets, but each party holds a different subset of attributes, and typically one active client holds labels while other passive clients hold additional feature sets. As a result, vertical learning addresses cross-domain collaboration under privacy, regulatory, and business constraints, and can be viewed simultaneously as a privacy-preserving form of multi-view learning, an instantiation of split learning, and a problem in privacy-preserving distributed optimization (Ye et al., 2024).

1. Definition, partitioning regimes, and formal setting

Federated learning is commonly divided by how data is partitioned. In Horizontal Federated Learning (HFL), clients share the same feature space but have different samples; data is sample-partitioned. In Vertical Federated Learning (VFL), clients share some or all of the same samples but have different local features; data is feature-partitioned. In Federated Transfer Learning (FTL), both sample and feature overlap are partial and knowledge is transferred across partially overlapping spaces (Ye et al., 2024).

Setting	Samples across parties	Features across parties
HFL	Different	Same
VFL	Shared or overlapping	Different
FTL	Partially overlapping	Partially overlapping

A standard formalization assumes total samples $N$, number of clients $M$, and a global sample set

$$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$

Features are partitioned across clients, with client $k$ holding $d^k$ features such that

$d = \sum_{k=1}^{M-1} d^k.$

If $k$ is active, its local dataset is

$$D^{k} = \{(x_i^{k}, y_i^{k})\}_{i=1}^{N^k}, \quad x_i^{k} \in \mathbb{R}^{d^k},$$

whereas a passive client holds

$$D^{k} = \{x_i^{k}\}_{i=1}^{N^k}, \quad x_i^{k} \in \mathbb{R}^{d^k}.$$

The aligned sample set shared across parties is

$$D^S = \{(x_i^{S}, y_i^{S})\}_{i=1}^{N^S}, \quad x_i^{S} \in \mathbb{R}^d,$$

where labels $M$0 come from the active client (Ye et al., 2024).

The learning objective is usually expressed through local models $M$1 that map local features to embeddings,

$M$2

followed by aggregation

$M$3

and a global model $M$4 that produces predictions

$M$5

Collaborative training solves

$M$6

where $M$7 may be cross-entropy or another task-specific loss (Ye et al., 2024). Surveys and reviews consistently frame this as the canonical vertical-learning problem (Khan et al., 2022, Liu et al., 2022).

2. Canonical protocol and architectural patterns

A generic VFL workflow begins with sample alignment. Parties must discover shared users or samples without revealing raw identifiers. The survey literature lists several mechanisms: direct matching on common IDs when available, encoding identifiers into cryptographic long-term keys (CLK) and matching via a coordinator, and Private Set Intersection (PSI) or Private Set Union (PSU) protocols. In formal terms, alignment securely constructs $M$8 from local datasets $M$9 (Ye et al., 2024).

After alignment, each client performs local feature processing. The local model can be a tree model, a neural network, or a linear or logistic model. The corresponding local embedding

$$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$0

is then sent to an active client, trusted server, or coordinator. Because embeddings themselves can leak information, VFL systems often protect this exchange with Homomorphic Encryption (HE), Secret Sharing, or Differential Privacy (DP) (Ye et al., 2024).

The active side aggregates embeddings, computes predictions, evaluates the loss, and differentiates with respect to both global parameters and each party’s intermediate representation. If $$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$1 denotes the aggregated representation, the active side computes $$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$2 and $$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$3, separates the latter into per-client components $$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$4, and returns these to clients. Each client then backpropagates locally: $$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$5 Testing mirrors training but omits gradient exchange: each client computes embeddings for test samples and the global model outputs predictions (Ye et al., 2024).

Architecturally, VFL spans several patterns. The review literature distinguishes splitVFL, aggVFL, splitVFL$$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$6, and aggVFL$$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$7 according to whether the global module is trainable and whether the active party also holds features (Liu et al., 2022). Neural VFL is often implemented as split learning, with front layers at each client and back layers at the active side. Frameworks such as PyVertical adopt this design, while MMVFL extends VFL to multi-participant, multi-class settings with privacy-preserving label sharing through local pseudo-label matrices $$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$8 and a shared consensus matrix $$D = \{(x_i, y_i)\}_{i=1}^{N}, \quad x_i \in \mathbb{R}^d.$$9 (Ye et al., 2024, Feng et al., 2020).

A recurring misconception is that “no raw data sharing” is equivalent to strong privacy. The protocol literature shows otherwise: embeddings, gradients, and alignment metadata each create distinct attack surfaces, so privacy in vertical learning is fundamentally about protecting intermediate computation rather than merely keeping datasets local (Ye et al., 2024, Liu et al., 2022).

3. Model families, effectiveness, and optimization

The effectiveness literature is commonly organized along three subtopics: model design, feature and client selection, and optimization/communication efficiency (Ye et al., 2024).

For tabular data, tree-based VFL remains prominent. Representative systems include SecureBoost, SecureGBM, FederBoost, OpBoost, Secure XGB, and vertically trained random forests. Their usual pattern is to let each party compute local statistics such as gradient or Hessian sums for its features, after which a coordinator or active client securely aggregates these statistics to select splits and build the tree ensemble collaboratively (Ye et al., 2024).

For more complex modalities, VFL uses neural architectures. The canonical split-neural formulation is

$k$0

with backward propagation through the split interface. Examples include PyVertical, MMVFL, and FedSL for sequential data (Ye et al., 2024). The literature also connects vertical learning to multi-view representation learning, since local models act as view-specific encoders whose outputs are fused at a global head (Feng et al., 2020).

Feature and client selection methods seek to improve predictive performance while reducing noise and communication. In feature selection, FedSDG-FS uses dual-gate feature selection with Gini impurity and differential privacy; LESS-VFL formulates selection with group lasso regularization,

$k$1

and FEAST uses conditional mutual information to select informative, low-redundancy features. These methods effectively perform vertical dimensionality reduction under privacy constraints (Ye et al., 2024). For client selection and contribution evaluation, FedSV and VerFedSV use Shapley-value-based valuation, while VF-MINE maximizes mutual information between selected clients’ features and labels through secure estimation (Ye et al., 2024, Fan et al., 2022).

Optimization and systems efficiency are equally central. The survey highlights AsySQN, FedBCD, C-VFL, SparseVFL, and VFL-CZOFO as examples that reduce communication rounds or message sizes through approximate second-order information, block coordinate descent, local updates with compressed embeddings, sparsity exploitation, or hybrid zero-order/first-order gradients (Ye et al., 2024). FedOnce takes this logic further by requiring only one-shot communication among parties: guests first learn unsupervised representations locally, then send these representations once to the host, which trains the supervised aggregation model. Across 10 datasets, it achieves close performance to state-of-the-art VFL algorithms with much lower communication costs, and its privacy-preserving technique based on moments accountant significantly outperforms state-of-the-art approaches under the same privacy budget (Wu et al., 2022).

Resource-constrained vertical learning has prompted specialized designs. LVFL introduces two lightweighting strategies: structured pruning of local feature models with computation ratio $k$2, and unstructured pruning of feature embeddings with communication ratio $k$3. The resulting convergence bound explicitly contains both ratios, and the paper reports that LVFL effectively mitigates resource demands while maintaining competitive learning performance in smart-building IoT (Wang et al., 2024). VFLHLP addresses the few-overlap problem by pre-training local networks on all local data—supervised pre-training at the active party and self-supervised contrastive pre-training at passive parties—before downstream VFL on aligned data. On real-world advertising datasets, it achieves the best performance over baseline methods by large margins (Li et al., 2024).

The literature also relaxes the standard single-label-owner assumption. CVFL treats the case where features are vertically partitioned but labels are horizontally partitioned across multiple parties. It uses a cascade architecture with bottom subnetworks at all parties and a top subnetwork coordinated by a collaborator, and introduces an exponent-weighted objective to amplify the contribution of slow active parties. Experiments show comparable performance to centralized training and better mitigation of the straggler problem than asynchronous aggregation alone (Xia et al., 2021).

4. Privacy, security, and trust mechanisms

Security research in vertical learning is structured around privacy leakage, malicious attacks, and defenses (Ye et al., 2024).

Privacy leakage begins at the alignment stage. Secure entity alignment relies on mechanisms such as Privacy-Preserving Entity Resolution (PPER) with CLKs or multi-party PSI/PSU, where PSI is designed to reveal

$k$4

without exposing $k$5 or $k$6 (Ye et al., 2024). Leakage then continues through intermediate representations. Embeddings $k$7 may be inverted to recover local features, while gradients encode label and feature correlations. Defenses therefore include HE on embeddings, functional encryption for gradients, secret sharing, and DP at either the embedding or gradient level. A typical DP perturbation of an embedding is

$k$8

and a typical DP perturbation of a gradient is

$k$9

The defining condition for $d^k$0-DP is

$d^k$1

for neighboring datasets $d^k$2 and any event $d^k$3 (Ye et al., 2024).

The attack taxonomy includes feature inference attacks, label inference attacks, and destructive attacks such as backdoors and poisoning. Feature inference methods such as CAFE, Chebyshev-center attacks, and search-based attacks reconstruct unknown passive-party features by exploiting gradients, prediction outputs, and correlations between known and unknown features. Label inference attacks use gradient norms, gradient inversion, embedding-label correlations, or split-learning-specific leakage channels. Backdoor attacks manipulate gradients or latent representations so that

$d^k$4

for triggered inputs, while poisoning attacks tamper with raw data, IDs, or sampled clients (Ye et al., 2024).

Defenses are correspondingly diverse. For feature inference, the literature reports adversarial training, structured gradient obfuscation in VFLDefender, selective gradient transmission, privacy-budget allocation, and adaptive obfuscation passports in FedPass. For label inference, defenses include gradient noise, label disguise via autoencoders and entropy regularization (DCAE), shadow gradients (FLSG), projection-based perturbation (ProjPert), HashVFL, and potential energy loss (PELoss). For destructive attacks, anomaly detection on embeddings, contrastive-learning-based detection (VFedAD), normalization and dropout, and robust feature subspace recovery (RVFR) are all documented (Ye et al., 2024).

Differential privacy has become a dedicated research line. Differentially Private Vertical Federated Learning studies how to protect organization data in VFL by clipping local updates and adding Gaussian noise, emphasizing that a trade-off point must be found between VFL performance and privacy protection in terms of perturbation noise (Ranbaduge et al., 2022). DP-BBVFL combines local differential privacy with a blockchain-based smart contract that aggregates embeddings transparently; embeddings are privatized before being stored on-chain, and the system provides verifiability with a tradeoff in training time due to on-chain aggregation (Tran et al., 2024). This suggests that trust in VFL can be redistributed from a central server to auditable coordination infrastructure, although privacy-utility and latency trade-offs remain central.

5. Applicability, deployment constraints, and governance

Applicability research asks how to make VFL usable when aligned samples are scarce, communication is expensive, clients are asynchronous, or contribution valuation matters (Ye et al., 2024).

The few-overlap problem is a defining practical constraint. Conventional VFL trains only on the intersection of user IDs,

$d^k$5

discarding $d^k$6 and $d^k$7. As more parties join, the aligned set can shrink drastically. VFLHLP addresses this by pre-training locally on aligned and unaligned data, then transferring that knowledge into downstream federated training on aligned samples through initialization and a constraint loss

$d^k$8

combined with the downstream VFL objective as

$d^k$9

The reported gains are especially strong in low-alignment advertising settings (Li et al., 2024).

Communication-constrained settings have produced several specialized formulations. FedOnce reduces training-time coordination to one-shot communication through unsupervised representation learning (Wu et al., 2022). LVFL reduces both computation and communication through pruning and provides bounds involving $d = \sum_{k=1}^{M-1} d^k.$0 and $d = \sum_{k=1}^{M-1} d^k.$1 (Wang et al., 2024). In wireless settings, vertical distributed learning can be implemented through learned embeddings plus distributed max-pooling. The hierarchical framework in “Communication Efficient Distributed Learning over Wireless Channels” maps each local input to an embedding $d = \sum_{k=1}^{M-1} d^k.$2, pools dimensionwise via

$d = \sum_{k=1}^{M-1} d^k.$3

and uses opportunistic carrier sensing so that only the worker holding the maximum transmits for each dimension. The result is a communication load independent of the number of workers while achieving almost the same model accuracy as concatenating raw worker outputs (Achituve et al., 2022).

Client asynchrony and heterogeneous labels also reshape the protocol design. CVFL is explicitly motivated by vertically partitioned features with horizontally partitioned labels and by the straggler problem. Its exponent-weighted objective increases the influence of slow active parties, which is particularly important when label distributions are non-i.i.d. across parties (Xia et al., 2021).

Governance and incentive questions are likewise part of VFL applicability. VerFedSV proposes a contribution valuation metric based on Shapley value, computed from time-indexed utilities on embeddings rather than through exhaustive retraining. It satisfies symmetry, zero element, and periodic additivity, and is adapted to both synchronous and asynchronous VFL (Fan et al., 2022). This aligns with broader survey conclusions that long-term vertical collaboration requires not only privacy and accuracy but also fair contribution evaluation and participant selection (Khan et al., 2022).

Industrial deployment further depends on compliance and framework support. VFL is motivated by regulations such as GDPR, CCPA, and CDPA, and several industrial systems—FATE, FedLearner, FedML, Jupiter—support VFL protocols integrated with privacy and compliance rules (Ye et al., 2024). A plausible implication is that applicability in vertical learning is jointly constrained by protocol design, system heterogeneity, and institutional trust arrangements rather than by predictive modeling alone.

6. Applications, benchmarks, and open directions

The application profile of vertical learning is intrinsically cross-domain. The survey literature lists finance for credit scoring and fraud detection, healthcare for outcome prediction and cross-modal collaboration, and advertising/recommendation for joint modeling across platforms holding behavioral, financial, or transactional features (Ye et al., 2024). The motivating examples are structurally similar: different organizations or departments each hold a distinct feature view of the same entities, and no single party has a sufficiently rich feature space to train the desired model alone.

Benchmarking remains a bottleneck. Current evaluation often uses synthetic vertical splits of standard image or tabular datasets, and there is an explicit call for realistic cross-domain datasets where clients hold related but distinct feature spaces and modalities. Benchmarks such as VFLAIR and Vertibench are emerging, but the field still lacks standardized, large-scale vertical datasets for fair comparison (Ye et al., 2024). Reviews also note that existing frameworks only partially support VFL and often have limited model coverage, incomplete privacy mechanisms, or insufficient documentation (Khan et al., 2022).

Several research directions now recur across surveys. One is the triadic trade-off between effectiveness, security, and applicability: adding DP noise, HE, or obfuscation can reduce leakage but also hurt convergence and accuracy, while stronger model design or feature selection can simultaneously improve performance and reduce attack surface (Ye et al., 2024). Another is robustness and fairness: FairVFL and fairness-constrained optimization are described as early steps, and robustness to noisy features or imperfect alignment remains underexplored (Ye et al., 2024). A third is expansion beyond classical tabular settings toward multi-modal VFL, graph VFL, and VFL with foundation models, where communication and privacy budgets become more demanding (Ye et al., 2024).

Across surveys and method papers, the broad synthesis is stable. Vertical learning is the problem of minimizing a joint loss over feature-partitioned data while preserving data locality: $d = \sum_{k=1}^{M-1} d^k.$4 Its conceptual identity lies at the intersection of multi-view representation learning and privacy-preserving distributed optimization (Ye et al., 2024). Its technical trajectory has moved from two-party, binary, cryptography-heavy settings toward multi-party, multi-class, communication-aware, attack-aware, and deployment-oriented systems (Feng et al., 2020). Its unresolved problems—practical datasets, composable privacy guarantees, robust asynchronous protocols, fair contribution valuation, and scaling to more complex modalities—define the current frontier of the field (Ye et al., 2024, Liu et al., 2022).