Domain-Aware Aggregation Approach

• Domain-aware aggregation approaches are techniques that adaptively fuse multi-domain data by weighting contributions based on domain-specific characteristics to optimize generalization and personalization.
• They employ mechanisms such as fine-grained expert modules, hierarchical fusion, and discrepancy metrics to outperform traditional, uniform aggregation strategies.
• Empirical evidence demonstrates improvements in fairness, robustness, and communication efficiency, while ongoing challenges include scalability and privacy concerns.

A domain-aware aggregation approach is any technique that adaptively integrates information—parameters, representations, or outputs—from multiple data sources, clients, or models based on the domains from which these data originate, with the explicit goal of maximizing generalization and/or personalization in the presence of domain shift or heterogeneity. Such methods address the inadequacies of domain-agnostic fusion strategies (e.g., naive averaging) by weighting, pairing, or fusing contributions according to inter-domain similarities, statistical discrepancies, or specialized embeddings. Recent advances in federated learning, multi-source domain adaptation, graph domain adaptation, and cross-domain recommendation have demonstrated significant performance gains and efficiency advantages when incorporating domain-aware aggregation, compared to traditional aggregation protocols.

1. Motivation and Theoretical Principles

Classical model aggregation procedures such as FedAvg or uniform parameter mixing typically neglect the potentially large heterogeneity between domains, leading to poor robustness or user-level performance drop under domain shift. Main challenges include domain dominance (where high-sample domains bias the global model) (Nguyen et al., 30 Nov 2025), semantic/feature dislocation across domains (Yuan et al., 2021), and the need for meaningful adaptation to unseen or underrepresented domains (Zhang et al., 2023, Jeon et al., 24 Nov 2025).

Theoretical frameworks underpin these methods by providing generalization error bounds that explicitly depend on domain discrepancy terms—measured via distances such as the $d_1$ distance (Su et al., 2022), adversarial discrepancy (Wen et al., 2019), or geometric alignments (Nguyen et al., 7 Aug 2025). These analyses justify adaptive aggregation by minimizing bounds that combine within-domain performance and between-domain alignment metrics, often under explicit simplex or attention-based constraints to avoid overfitting to single domains.

Domains are typically assumed to be either observed (with domain labels), deducible (with proxies such as client or data-specific embeddings), or implicit (latent domains emerging from task or context structure).

2. Architectural and Algorithmic Paradigms

Domain-aware aggregation has been instantiated in a broad range of learning architectures, each tailored to the properties of the domain structure and the context of deployment. Prominent frameworks include:

• Fine-Grained Expert or Prototype Aggregation: Per-client Mixture-of-Experts (MoE) modules are selectively fused across clients based on cross-client similarity in gating or latent space, as in FedMoE-DA (Zhan et al., 4 Nov 2024). Similarly, in cross-client prototype aggregation, a transformer-based attention mechanism adaptively weights client class means, conditioned on disentangled content/style representations (Jeon et al., 24 Nov 2025).
• Two-Stage (Hierarchical) or Multi-Level Aggregation: Multi-level server-side aggregation mitigates domain dominance; e.g., local client updates are first combined within each domain, followed by uniform or attention-based averaging across domains (Nguyen et al., 30 Nov 2025). Hierarchical federated settings further introduce cross-station optimal transport alignment and regularized fusion for scaling to many clients/stations (Nguyen et al., 7 Aug 2025).
• Attention and Cross-Domain Weighting: Attention mechanisms (including cross-attention blocks in recommender systems (Luo et al., 22 Jan 2024), or prototype-based softmax weighting (Zhang et al., 2023)) adaptively blend signals from multiple domains depending on local semantic consistency or latent similarities.
• Domain Discrepancy Measurement: Explicit computation of sample- or domain-level distances, such as subspace projection similarities (Su et al., 2022), cosine similarity of proxy vectors (Zhan et al., 4 Nov 2024), or adversarial domain classifiers (Wen et al., 2019), underlies many domain-aware aggregation schemes. These metrics drive the assignment of aggregation neighborhoods, attention weights, or teacher ensemble coefficients.

3. Mathematical Formulations

Domain-aware aggregation is mathematically characterized by adaptive fusion operators. Key instances include:

Approach	Aggregation rule	Domain similarity metric
FedMoE-DA (Zhan et al., 4 Nov 2024)	$\bar\Phi_i = \sum_{j\in S_i} a_{ij}\Phi_j$	$r_{ij} = \cos(\pi_i, \pi_j)$ via gate proxies
FedDoM (Nguyen et al., 30 Nov 2025)	$$\theta_g^{t+1} = \sum_{u=1}^n \alpha_u \theta^{t+1}_{u,g},\ \alpha_u=1/n$$	Explicit domain partition, fixed $\alpha_u$
DDN (Zhang et al., 2023)	$R^T_m = \sum_{s=1}^S w_{Ts}R^s_m$, $w_{Ts} = \mathrm{softmax}(\cos(E(x), q^s) / \tau)$	Embedding-prototype cosine
GraphATA (Zhang et al., 5 Feb 2025)	$$W_v^l = \sum_{i=1}^m \alpha_{v,i}^l \Lambda(c_v^l) W_i^l + \lambda W_g^l$$, $\alpha_{v,i}^l = \mathrm{sparsemax}(e_{v,i}^l)$	Local context, attention over $W_i^l$
FedD3A (Su et al., 2022)	$\hat{\mathbf y} = \sum_{i=1}^m \alpha_i h_{M_i}(x)$, $\alpha_i = \mathrm{Softmax}(r_i)$	Feature subspace-projection cosine

These aggregation operators are always non-uniform and modulated either per-sample, per-class, or per-layer, according to similarity or discrepancy between domain-specific statistics.

4. Representative Applications

Domain-aware aggregation has been successfully deployed in:

• Federated Learning: Personalization mechanisms leveraging domain-aware MoE and P2P expert transfer (Zhan et al., 4 Nov 2024); hierarchical aggregation scalable to hundreds of clients with optimal transport constraints (Nguyen et al., 7 Aug 2025); communication-efficient, style disentangling federated protocols (Jeon et al., 24 Nov 2025).
• Domain Generalization/Adaptation: Prototype-level and expert-level strategies for unseen-domain robustness (Zhang et al., 2023, D'Innocente et al., 2018); multi-source adversarial adaptation for classification and segmentation (Zhao et al., 2020, Wen et al., 2019); graph domain adaptation with node-centric parameterization (Zhang et al., 5 Feb 2025).
• Cross-Domain Recommendation: Two-step cross-attention capturing domain and item-level transfer (Luo et al., 22 Jan 2024).
• Knowledge Distillation in Aggregation: Sample-level, subspace-based adaptive distillation in federated settings with heterogeneous domains and models (Su et al., 2022).

5. Empirical Evidence and Comparative Performance

Domain-aware aggregation approaches consistently demonstrate:

• Significant mitigation of domain dominance: Equalizing domain weights or adaptively upweighting under-represented domains (even under severe sample imbalance) increases fairness and performance across all domains (Nguyen et al., 30 Nov 2025).
• Superior out-of-domain generalization: Aggregation strategies leveraging prototype alignment, dynamic weighting, or per-domain attention achieve higher accuracy or lower reconstruction error in OOD tests (Zhang et al., 2023, Jeon et al., 24 Nov 2025, Zhan et al., 4 Nov 2024).
• Communication Efficiency: Selective synchronization schemes—transmitting only necessary expert parameters or per-class aggregates—reduce bandwidth while retaining or improving personalization (Zhan et al., 4 Nov 2024, Jeon et al., 24 Nov 2025).
• Convergence Speed and Stability: Adaptive weighting in gradient or parameter space (e.g., using loss-based or learned weights) yields faster convergence than uniform approaches and reduces gradient noise (Dimitriadis et al., 2021).

Representative results include: cutting server communication by >80% on CIFAR-10 while increasing accuracy under non-i.i.d. splits (Zhan et al., 4 Nov 2024); PSNR improvements up to +1.8 dB on minority domains in cross-domain federated semantic communication (Nguyen et al., 30 Nov 2025); test accuracy boosts of several percentage points in federated, graph, and recommender settings (Jeon et al., 24 Nov 2025, Zhang et al., 5 Feb 2025, Luo et al., 22 Jan 2024).

6. Limitations, Open Questions, and Future Directions

While domain-aware aggregation methods provide marked improvements, several challenges remain:

• Scalability: The cost of computing and communicating per-domain similarity matrices or projection operators increases with the number of domains and clients. Hierarchical and P2P strategies partially mitigate but do not eliminate this cost (Nguyen et al., 7 Aug 2025).
• Privacy Guarantees: Many methods exchange statistical representations (e.g., projection matrices, content prototypes), which require further analysis for privacy leakage, especially under sophisticated reconstruction attacks (Su et al., 2022).
• Automatic Domain Discovery and Matching: Implicit or latent domain identification and adaptive aggregation in the absence of explicit domain labels remain active areas, especially for settings with unstructured or dynamically evolving domain partitions.
• Combination with Data-Free or Robust Aggregation: Integration with data-free knowledge distillation, OOD detection, and adversarial robustness techniques could yield further improvements, but the impact of domain-aware weighting in such settings is an open research question.
• Theory-Practice Gap: While generalization bounds motivate many algorithms, real-world performance sometimes diverges from theoretical predictions, especially under extreme sample skew or adversarial clients.

Domain-aware aggregation remains a critical research area for enabling scalable, robust, and communication-efficient learning over heterogeneous data sources, with ongoing advances in federated learning, adaptation, and multi-domain modeling (Zhan et al., 4 Nov 2024, Nguyen et al., 30 Nov 2025, Zhang et al., 2023, Su et al., 2022, Nguyen et al., 7 Aug 2025, Jeon et al., 24 Nov 2025, Zhang et al., 5 Feb 2025).