Prototype-Regularized Federated Learning for Cross-Domain Aspect Sentiment Triplet Extraction

Abstract: Aspect Sentiment Triplet Extraction (ASTE) aims to extract all sentiment triplets of aspect terms, opinion terms, and sentiment polarities from a sentence. Existing methods are typically trained on individual datasets in isolation, failing to jointly capture the common feature representations shared across domains. Moreover, data privacy constraints prevent centralized data aggregation. To address these challenges, we propose Prototype-based Cross-Domain Span Prototype extraction (PCD-SpanProto), a prototype-regularized federated learning framework to enable distributed clients to exchange class-level prototypes instead of full model parameters. Specifically, we design a weighted performance-aware aggregation strategy and a contrastive regularization module to improve the global prototype under domain heterogeneity and the promotion between intra-class compactness and inter-class separability across clients. Extensive experiments on four ASTE datasets demonstrate that our method outperforms baselines and reduces communication costs, validating the effectiveness of prototype-based cross-domain knowledge transfer.

• The paper presents a novel federated learning framework (PCD-SpanProto) that exchanges class-level prototypes to enable privacy-preserving and efficient ASTE.
• It leverages performance-aware aggregation and contrastive regularization to boost F1 scores by up to 4.45 points across multiple datasets.
• The study demonstrates significant reductions in communication overhead, making cross-domain sentiment analysis scalable in heterogeneous and privacy-sensitive environments.

Prototype-Regularized Federated Learning for Cross-Domain ASTE: Technical Analysis

Introduction

Aspect Sentiment Triplet Extraction (ASTE) encapsulates the challenge of extracting tuples comprising aspect terms, opinion terms, and sentiment polarities from sentences, providing a fine-grained sentiment analysis framework that supersedes coarse-grained ABSA approaches. Existing solutions predominantly train models in isolation for each domain, which incurs suboptimal performance in cross-domain scenarios due to the failure to capture generalizable semantic structures. Furthermore, privacy constraints and organizational boundaries present significant obstacles to centralized data aggregation.

The referenced research introduces a federated learning paradigm—Prototype-based Cross-Domain Span Prototype extraction (PCD-SpanProto)—explicitly designed for ASTE by leveraging prototype-based inter-client knowledge transfer and contrastive regularization. This essay provides an expert-level review of the framework, its empirical results, and implications for the broader landscape of federated natural language understanding.

Prototype-Regularized Federated Learning Framework

PCD-SpanProto occupies a novel space in federated systems by engaging span-level semantic structures, as opposed to traditional prototype-based personalized federated learning (pFL) that targets only flat classification. The primary departure is the exchange of class-level prototypes rather than full parameter sets or gradients, mitigating communication overhead and addressing client drift due to non-IID data distributions.

The workflow involves:

• Client-Side: Each client builds local class-level span prototypes from the BERT-encoded representations of labeled spans, optimized via a joint loss integrating STAGE span-tagging objectives and prototype-based contrastive regularization.
• Server-Side: Aggregates local prototypes using performance-aware (F1 score-based) weighting, constructs global prototypes, and disseminates them for the next training cycle.
• Cross-Domain Regularization: The prototype-based contrastive module enforces intra-class embedding compactness and inter-class discrimination, leveraging global prototypes as semantic anchors.

  Figure 1: Schematic depiction of the prototype-regularized federated learning system for cross-domain ASTE.

Span Tagging and Prototype Construction

The model enumerates all candidate spans in input sentences, assigning a three-dimensional tag $(t_a, t_o, t_s)$ indicating aspect, opinion, and sentiment roles based on a composite labeling scheme. Span representations are derived from reconstructed word-level embeddings using attention-weighted BERT outputs. Prototypes per semantic class are computed as momentum-based averages over constituent span embeddings, resulting in robust representations for each composite tag.

Figure 2: The hierarchical span tagging process for aspect, opinion, and sentiment.

Performance-Aware Prototype Aggregation

Innovatively, the aggregation mechanism is adaptive: clients with superior validation F1 metrics exert proportional influence on the synthesized global prototypes, counterbalancing issues from noisy or less-representative local spaces. This resolves the traditional pitfall of uniform averaging that can dilute domain-specific or minority features in heterogeneous environments.

Empirical Evaluation: Numerical Outcomes & Analyses

PCD-SpanProto was validated on four ASTE datasets (14Lap, 14Res, 15Res, 16Res) structured as four federated clients. Main findings include:

• Performance Superiority: The method consistently surpasses strong span-based and GNN-based ASTE baselines across all datasets, with F1 improvements up to 4.45 points compared to the best non-federated STAGE baseline. Gains are consistently reflected across both precision and recall, indicating both better extraction and reduced false positives.
• Communication Efficiency: The system requires only 3,200 parameters per round per client to be transmitted, compared to over 110M in model parameter-sharing federated approaches.
• Cross-Domain Generalization: Federated training yields better in-domain and substantial cross-domain improvements; notably, single-domain models deteriorate steeply when transferred out-of-domain.
• Aggregation Strategy Impact: Performance-aware aggregation provides a measurable advantage over uniform aggregation, especially in domains with low inter-client prototype similarity.

  Figure 3: Matrix of pairwise cosine similarity among client prototypes across domains.

  

  Figure 4: Triplet-level F1 score progression over federated rounds for each client.

Prototype Space Dynamics and Visualization

Cross-client prototype analysis via t-SNE reveals emergent clustering, where prototypes of the same semantic class (across clients) tend to cluster, yet domain-specific variations are preserved, balancing generalization with local adaptation. Prototype similarity matrices and convergence graphs further substantiate the efficacy of the aggregation and contrastive objectives.

Figure 5: t-SNE projection of class-level prototypes, with visible class-wise clusters and domain-specific variance.

Ablation and Sensitivity Analyses

Experiments on alignment ($\alpha$) and separation ($\beta$) weights in the contrastive regularization loss function demonstrate stable sensitivity profiles, with optimal ranges consistent across multiple domains. This robustness facilitates practical deployment without laborious hyperparameter recalibration for new domains.

Figure 6: Sensitivity of client F1 scores to the alignment weight $\alpha$ and separation weight $\beta$ in the prototype contrastive loss.

Theoretical and Practical Implications

The approach validates several theoretical tenets:

• Prototype-level sharing suffices for structured tasks requiring span-level predictions, mitigating the communication and convergence issues of parameter-level FL.
• Performance-aware aggregation addresses domain heterogeneity more effectively than naive averaging.
• Momentum-based prototype updating and contrastive regularization stabilize federated convergence for complex structured prediction.

Practically, these features enable privacy-preserving, multi-institutional training of fine-grained sentiment analysis models, without exposing raw data or incurring prohibitive communication costs. The demonstrated gains in both efficiency and performance position prototype-based federated frameworks as a front-runner for other multi-domain structured NLP tasks—potentially extending into event extraction, named entity recognition, and beyond.

Future Directions

Open research directions include refined prototype design for more granular semantic categories, adaptive per-client regularization strategies, and extending the paradigm to multimodal or multilingual federated settings. Another direction is the investigation of prototype privacy leakage, requiring theoretical privacy analyses under adversarial server assumptions.

Conclusion

This work establishes a new framework for cross-domain ASTE using prototype-regularized federated learning, combining communication efficiency, privacy preservation, and robust performance improvements. The approach demonstrates that structured tasks like span-level sentiment triplet extraction considerably benefit from prototype-based federated strategies coupled with contrastive regularization and adaptive aggregation, setting a precedent for broader adoption in privacy-sensitive, heterogeneous environments.