Transfer learning under latent space model

Abstract: Latent space model plays a crucial role in network analysis, and accurate estimation of latent variables is essential for downstream tasks such as link prediction. However, the large number of parameters to be estimated presents a challenge, especially when the latent space dimension is not exceptionally small. In this paper, we propose a transfer learning method that leverages information from networks with latent variables similar to those in the target network, thereby improving the estimation accuracy for the target. Given transferable source networks, we introduce a two-stage transfer learning algorithm that accommodates differences in node numbers between source and target networks. In each stage, we derive sufficient identification conditions and design tailored projected gradient descent algorithms for estimation. Theoretical properties of the resulting estimators are established. When the transferable networks are unknown, a detection algorithm is introduced to identify suitable source networks. Simulation studies and analyses of two real datasets demonstrate the effectiveness of the proposed methods.

• The paper introduces a two-stage algorithm that aggregates latent information from multiple source networks and refines estimates through a debiasing process.
• It establishes theoretical guarantees with non-asymptotic error bounds, ensuring consistent and identifiable latent position estimation with improved accuracy.
• The methodology effectively mitigates negative transfer by detecting and utilizing only informative source networks, as demonstrated in trade and political event applications.

Transfer Learning under Latent Space Models: Methodology, Theory, and Empirical Evaluation

This paper introduces a principled transfer learning framework for latent space models (LSMs) in network analysis, addressing the challenge of accurate latent variable estimation when the target network is small or sparse. The proposed approach leverages multiple source networks with potentially different node sets, providing both algorithmic innovations and theoretical guarantees. The methodology is validated through extensive simulations and real-world applications to trade and political event networks.

Problem Formulation and Model Structure

The latent space model posits that each node in a network is associated with a low-dimensional latent vector, and the probability of an edge between two nodes is a function of their latent positions and node-specific degree heterogeneity parameters. For a target network $\mathcal{G}^t$ with $n$ nodes, the adjacency matrix $A^t$ is modeled as:

$$A_{ij}^t \sim \text{Bernoulli}(P_{ij}^t), \quad \text{logit}(P_{ij}^t) = \alpha_i^t + \alpha_j^t + Z_i^{t\top} Z_j^t$$

where $Z_i^t \in \mathbb{R}^k$ is the latent position and $\alpha_i^t$ is the degree parameter.

Multiple source networks $\mathcal{G}^{s_1}, \ldots, \mathcal{G}^{s_L}$ are available, each with its own node set (possibly larger than the target's). The key assumption is that the target's node set is a subset of each source's node set, enabling transfer of latent information.

Two-Stage Transfer Learning Algorithm

The core methodological contribution is a two-stage transfer learning algorithm (TLK) for LSMs:

1. Transferring Stage: Aggregate information from the latent variables of the transferable source networks to obtain an initial estimator $U_0$ for the target's latent positions. This is achieved by minimizing a joint negative log-likelihood over all source networks in the transferable set $\mathcal{A}$, accommodating differences in network sizes and node sets.
2. Debiasing Stage: Correct the bias in $U_0$ (arising from differences between source and target latent spaces) by solving a penalized likelihood problem on the target network, with a nuclear norm penalty on the correction term $\Delta$. The final estimator is $\hat{Z}^t = \hat{U}_0 + \hat{\Delta}$.

The algorithm employs projected and proximal gradient descent for efficient optimization, with identifiability ensured via centering constraints and orthogonal invariance.

Theoretical Guarantees

The paper establishes non-asymptotic error bounds for the latent variable estimators under mild conditions:

• Consistency: The Frobenius norm error between the estimated and true latent positions (up to orthogonal transformation) is $O_p(n^{-1/2})$.
• Identifiability: Sufficient conditions are provided to ensure that the latent positions and degree parameters are identifiable up to orthogonal transformation.
• Negative Transfer Mitigation: Theoretical analysis and empirical results demonstrate that including non-informative source networks can degrade performance, motivating the need for transferable set detection.

Transferable Set Detection

When the set of informative source networks is unknown, the paper proposes a data-driven detection algorithm (TLD) based on edge sampling and predictive loss evaluation. Each source network is evaluated for its contribution to predictive performance on held-out edges in the target network. Only those source networks that yield predictive loss not significantly worse than the baseline are included in the transfer process, effectively controlling for negative transfer.

Empirical Evaluation

Simulation Studies

Comprehensive simulations assess the performance of TLK, TLD, and several baselines (including naive pooling and single-network estimation) across varying network sizes, numbers of informative sources, and degrees of similarity between source and target latent spaces. Key findings include:

• Superior Estimation Accuracy: Both TLK and TLD achieve lower relative errors in latent position, degree parameter, and edge probability estimation compared to baselines, especially when the number of informative sources increases or their latent spaces are more similar to the target.
• Robustness to Network Size and Sparsity: Performance improves with larger target networks and more informative sources.
• Effective Negative Transfer Control: TLD maintains high true positive rates and low false positive rates in identifying informative sources, preventing performance degradation from non-informative networks.

Real-World Applications

FAO Trade Network

The method is applied to a global food trade network, with the beet pulp trade network as the target and six auxiliary product networks as sources. TLD outperforms all baselines in link prediction (Brier score), and the detected transferable networks (e.g., Beer of barley, Bread, Chicken meat) align with domain knowledge regarding shared supply patterns.

Figure 1: The prediction results of FAO trade network, showing improved Brier scores for TLD over baselines.

POLECAT Political Event Network

The framework is further validated on the POLECAT dataset, modeling geopolitical interactions across 16 event types. Using the "concede" network as the target, TLD again achieves the best predictive performance and identifies relevant event types (protests, assaults, threats) as informative sources, consistent with political science literature.

Figure 2: The prediction results of POLECAT network, demonstrating the benefit of transfer learning with detected transferable sources.

Implementation Considerations

• Computational Efficiency: The two-stage algorithm is scalable, with moderate computational requirements for networks of up to several hundred nodes and multiple sources.
• Hyperparameter Selection: The nuclear norm penalty parameter $\lambda$ is selected via cross-validation; the latent space dimension $k$ is typically fixed at a small value (e.g., 2) for interpretability.
• Extension to Heterogeneous Node Sets: The framework accommodates source networks with larger node sets than the target, provided the target's nodes are present in each source.
• Negative Transfer Avoidance: The detection algorithm is critical in practice, as naive pooling of all sources can lead to substantial performance loss.

Implications and Future Directions

The proposed transfer learning framework for LSMs provides a statistically principled and computationally tractable approach to leveraging auxiliary network data for improved latent variable estimation. Theoretical results guarantee consistency and identifiability, while empirical studies confirm practical utility in diverse domains.

Potential future research directions include:

• Quantifying the Reduction in Estimation Error: Formal analysis of how the number and quality of source networks affect the minimax error rates for latent variable estimation.
• Distributed and Privacy-Preserving Extensions: Adapting the framework to settings where source networks are distributed across multiple sites, possibly with privacy constraints.
• Generalization to Other Network Models: Extending the methodology to dynamic, directed, or attributed networks, and to other probabilistic network models beyond LSMs.

Conclusion

This work advances the state of transfer learning in network analysis by providing a rigorous, generalizable, and empirically validated framework for latent space models. The combination of multi-source aggregation, debiasing, and negative transfer control yields substantial improvements in latent variable estimation and downstream tasks, with broad applicability to real-world network data.