Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approach

Abstract: MultiModal Federated Graph Learning (MM-FGL) offers a natural collaborative training paradigm, but its practical deployment is challenged by two granularities of modality imbalance. Client-level imbalance occurs when certain clients lack entire modalities, while node-level imbalance occurs when individual nodes exhibit missing visual or textual attributes. While several relevant studies exist, our investigation reveals that they predominantly target graph-agnostic or centralized scenarios, rendering them difficult to adapt directly. To address these challenges, we formalize modality-imbalanced MM-FGL as an implicit graph-aware latent semantic representation synthesis problem. This paradigm recovers missing modal semantics directly within the representation space, thereby maximizing alignment with the original data's semantic distribution and mitigating the high variance induced by missing modalities. To this end, we propose FedMGS (Federated Modality-aware Graph Synthesis), which integrates three core components. The availability-aware graph encoder prevents missing modalities from contaminating local structural propagation. The prototype-guided latent semantic synthesizer establishes cross-client semantic anchors for unavailable modalities. The reliability-calibrated semantic fusion mechanism regulates the impact of recovered latent representations prior to predictive readout. Extensive experiments on four tasks show that FedMGS consistently outperforms competitive baselines with gains up to 17.41% with best efficiency-performance tradeoff.

• The paper presents FedMGS, a novel method that employs latent synthesis to recover missing modalities in federated graph learning, significantly enhancing node performance.
• It integrates availability-aware encoding, prototype-guided synthesis, and reliability-calibrated fusion to mitigate propagation contamination and semantic inaccessibility.
• Experimental evaluations demonstrate up to 17.41% accuracy gains in node classification and robust performance across varying missing data scenarios.

Modality-Imbalanced Federated Graph Learning via Latent Synthesis

Problem Formulation and Motivation

The paper "Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approach" (2606.20382) addresses a critical gap in distributed multimodal graph learning, focusing on scenarios where modality imbalance arises both at the client and node level. In the federated landscape, institutions typically possess only partial modalities due to privacy restrictions, disparate data pipelines, and heterogeneous operational constraints. This two-granularity imbalance, involving both fully absent modalities in certain clients and sporadically missing attributes at the node level, severely impairs downstream predictive quality by disrupting the graph propagation and semantic learning processes.

Conventional multimodal federated learning (MM-FL) approaches treat missing modalities by either placeholder completion or prototype regularization. Centralized multimodal graph learning exploits full access to topology and attributes, yet this is unattainable under privacy-preserving federated constraints. Existing methods in MM-FL largely ignore the topological propagation dynamics inherent in graphs, reducing modality recovery to independent entity completion that blends poorly with the underlying graph structure. This deficiency results in propagation contamination, cross-client semantic inaccessibility, and reliability uncertainty in synthesized latent representations.

FedMGS: Framework and Core Mechanisms

FedMGS (Federated Modality-aware Graph Synthesis) is introduced to resolve modality-imbalanced MM-FGL through implicit latent synthesis, avoiding explicit feature reconstruction or raw data transfer. The method formalizes missing modalities as a representation-space recovery problem, capitalizing on graph-contextual signals and federated class-modality prototypes. It comprises three core mechanisms:

1. Availability-aware graph encoding (AGE): Modality availability indicators gate input features, ensuring that absent modalities do not contaminate graph message passing. Initial node representations are constructed only from observed modalities, normalizing for their count to maintain semantic integrity.
2. Prototype-guided latent semantic synthesis (PLSS): Class-wise modality prototypes, aggregated across clients, serve as semantic anchors for latent synthesis. Clients lacking an entire modality utilize federated prototypes, giving access to modality-specific semantics without transmitting raw features or local embeddings. Synthesis combines local graph context, complementary observed modalities, and prototype queries.
3. Reliability-calibrated semantic fusion (RSF): Synthesized latents are treated as uncertain estimates. Node-level confidence scores and prototype stability statistics calibrate the final fusion, adaptively controlling the influence of synthesized versus observed representations. Reliability weighting ensures conservative usage of synthesized latents under semantic dispersion or ambiguity.

FedMGS strictly restricts communication to model parameters, class-modality prototypes, observation counts, and inter-class spread statistics—raw features, edges, and node-level embeddings are not exchanged.

Theoretical Guarantees

Rigorous analysis substantiates three core design principles:

• Conditioning latent synthesis on both graph context and federated prototypes provably lowers reconstruction risk over the complementary modality alone.
• Prototype aggregation exclusively from observed entries avoids systematic bias, especially under client-level missingness.
• Reliability weighting minimizes fusion error when synthesis uncertainty is non-negligible, corroborated by mean squared error analysis.

Experimental Evaluation

FedMGS was evaluated across four task families: node classification, link prediction, modality matching, and modality retrieval, using eight datasets spanning e-commerce and image/video domains. Modality imbalance was simulated at missing rates $p \in \{0.3, 0.5, 0.7\}$ at both client and node levels. Comprehensive comparisons with state-of-the-art FGL, MM-FL, and prototype-based baselines (e.g., FedAvg, FedMAC, FedMVP, FedProto) were performed under a unified protocol.

Numerical Results and Robustness

FedMGS demonstrated consistent superiority across all tasks:

• Up to 17.41% gain in node classification accuracy on Movies compared to the strongest baseline.
• Best R@5 scores on modality retrieval (e.g., 79.40 on Toys) and highest AUC in modality matching (85.92 on KU).
• Robustness confirmed under varying missingness: FedMGS sustained leading performance as $p$ increased, with only moderate absolute score decline.

Ablation confirmed complementary contributions from AGE, PLSS, and RSF—omission of any component led to significant performance drops, especially AGE (4.30% accuracy reduction).

Efficiency and Hyperparameter Sensitivity

FedMGS achieves favorable efficiency-performance tradeoff. Extra communication cost is limited to class-modality prototypes and counts, scaling only with the number of classes and prototype dimensions. Empirical running-time overhead is modest relative to accuracy gains.

Hyperparameter analysis revealed stable sensitivity, with performance favoring strong prototype alignment and moderate reconstruction regularization. The response surfaces were smooth, indicating robustness to hyperparameter tuning and negating the need for dataset-specific calibration.

Implications and Future Directions

FedMGS presents a practical solution for real-world MM-FGL deployments where modality imbalance is endemic. By shifting recovery from raw data synthesis to graph-aware latent synthesis and uncertainty-calibrated fusion, FedMGS achieves both semantic fidelity and privacy preservation with minimal communication overhead. The theoretical analysis generalizes across topological settings and federated compositions, enabling flexible adaptation.

Practically, FedMGS enables multimodal graph learning in distributed environments such as healthcare, financial security, and recommendation systems, where cross-institutional semantic anchors are crucial and privacy is non-negotiable. Its implicit latent synthesis paradigm offers a unified backbone for diverse downstream tasks, including node-centric and cross-modal retrieval.

Future research should investigate dynamic, evolving modality availability, policy-driven constraints, and integration with foundation models for richer semantic anchors. Adaptive synthesis strategies conditioned on usage context and label sparsity could further enhance cross-client generalization and robustness.

Conclusion

This paper establishes modality-imbalanced MM-FGL as a fundamentally latent synthesis problem and proposes a theoretically grounded, experimentally validated solution in FedMGS (2606.20382). Its triadic mechanisms collectively mitigate propagation contamination, semantic inaccessibility, and reliability uncertainty. FedMGS consistently outperforms baselines over node-level and client-level modality imbalance, achieving notable accuracy gains and efficiency advantages. The work lays the foundation for further advances in privacy-aware, modality-adaptive federated graph learning, with broad applicability across multimodal distributed domains.