Beyond Weights and Gradients: A Taxonomy of Federated Learning Messages

Abstract: Federated Learning is rapidly evolving beyond the exchange of traditional model weights and gradients, yet existing definitions fail to capture the full scope of modern payloads like synthetic data and federated analytics. This paper addresses the gap by proposing a formal mathematical definition of a federated message that accounts for both utility and privacy. We introduce a taxonomy that organizes these exchanges into three categories: model structures, statistical summaries, and data-conditioned representations. By evaluating these groups based on computational demands, communication costs, and privacy risks, we provide a clearer understanding of the trade-offs involved in decentralized training. Our review of 202 recent publications highlights a significant shift since 2021 toward diverse messaging paradigms, signaling a move away from standard deep learning updates toward more specialized information sharing. This framework provides a structured path for future research to optimize federated systems for varying hardware and security requirements.

• The paper introduces a formal framework and mathematical definition for federated messages by categorizing payloads into model parameters, statistical summaries, and data-conditioned representations.
• It reveals that federated learning now integrates not only weights and gradients but also analytics-driven summaries and synthetic data, accommodating diverse hardware and privacy constraints.
• The study provides a robust comparative analysis of computation, bandwidth, and privacy trade-offs, offering actionable insights for future FL system benchmarks and regulatory compliance.

Formal Taxonomy and Evolution of Federated Learning Messages

Motivation and Context

Federated Learning (FL) originated as a paradigm for collaborative machine learning under stringent data privacy constraints, relying on the decentralized exchange of model parameters such as weights or gradients. The foundational view, exemplified by FedAvg, assumed that payloads were strictly optimization artifacts of neural networks. However, the operational requirements of modern FL have evolved: privacy-preserving computation, heterogeneous hardware, vertical and horizontal partitioning, and emergent analytics workflows now demand a broader set of communication payloads. The paper "Beyond Weights and Gradients: A Taxonomy of Federated Learning Messages" (2606.16891) addresses the inadequacy of legacy definitions, proposing a formal mathematical definition and a heterogeneous taxonomy that encompasses both utility and privacy axes.

Formal Framework and Taxonomy

The authors introduce a functional definition of federated messages. For each party $k$ holding a local dataset $D_k$ and shared state $\theta$, a federated message is defined as $M_k = \mathcal{F}_k(D_k, \theta)$, where $\mathcal{F}_k$ is a mechanism embedding non-trivial dependence on $D_k$ without disclosing raw data. Utility and privacy are conceptualized along axes of task relevance and leakage boundaries, respectively. Crucially, the multi-round composition is analyzed: utility is emergent across iterative transcripts, but privacy strictly degrades with each round, necessitating compositional accounting.

To systematically categorize message forms, the paper subdivides them into three semantic groups:

• Model Structures and Parameters: Encompasses deep and non-deep models, from high-dimensional neural net weights and gradients to parametric coefficients, clustering prototypes, subspace factors (e.g., PCA), rule-based tree splits, graph structures for causal discovery, and reinforcement learning artifacts (Q-tables, policy nets).
• Statistical Summaries: Payloads targeting analytics (FA) and monitoring, such as basic moments, histograms, quantile sketches, and frequency approximations, often transmitted with lightweight privacy mechanisms.
• Data-Conditioned Representations: Embeddings, synthetic twin data, and distillation targets—objects derived directly or indirectly from local data, supporting tasks such as vertical FL, generative proxy modeling, and knowledge distillation.

  Figure 1: Hierarchical structure of model-based messages, illustrating the breadth from deep models to parametric, rule-based, clustering, and control artifacts.

Comparative Analysis of Message Groups

A rigorous trade-off analysis is presented for each message group:

• Local Computation: Deep model update generation is computationally intensive; non-deep parameters and summaries are lightweight; data-conditioned messages have varied profiles, with synthetic data generation often exceeding feasible client-side resources.
• Payload Size: Deep structures are bandwidth-heavy; summaries are minimal; embeddings and synthetic data show batch-dependent scaling.
• Aggregation Complexity: Traditional averaging for parameters versus instant aggregation for summaries and non-trivial optimization/training for synthetic/distillation-based payloads.
• Semantic Granularity: Weights and synthetic samples encode high-fidelity data abstractions, but may increase privacy risk; summaries offer transparency but limited predictive utility.
• Privacy and Defense: Parameter payloads and embeddings require cryptographic defenses (MPC, HE) and are empirically vulnerable to inversion; statistical summaries are naturally compatible with formal DP bounds; synthetic and distillation-based messages require regularization and DP at generation.

These dimensions delineate a Pareto frontier: there is no universally optimal payload, but system design can be matched to regulatory, hardware, and application constraints by appropriate message selection.

Literature Survey and Research Trends

A corpus of 202 publications was screened to empirically validate the taxonomy and track the evolution of federated communication payloads. The field has diversified substantially since 2021, with an observable shift from optimization-centric updates to analytics-driven summaries and data-conditioned representations.

Figure 2: Temporal evolution of FL research focus, showing diversification toward analytics and data-conditioned messaging post-2021.

The authors provide quantitative evidence of emergent frontiers:

• Vertical FL and Split Learning: rapid growth in embedding-based payloads.
• Federated Analytics: increased demand for lightweight, one-shot statistics in bandwidth-constrained and mobile deployments.
• Synthetic Data and Knowledge Distillation: new paradigms enabling centralized retraining or student-teacher transfer without direct parameter exchange.

  Figure 3: Breakdown of new messaging frontiers, highlighting the rise of vertical federation, synthetic proxy generation, and analytics.

A further breakdown of research contributions indicates methodological innovation as the driver, with novel algorithms comprising over 76% of the surveyed works.

Figure 4: Distribution of primary research contribution types, underscoring methodological advances as dominant.

Implications and Prospects

The formalization and categorization of federated messages offered by this work have substantial practical and theoretical ramifications:

• System Design: Enables tailor-fitting FL architectures to hardware or regulatory constraints via payload selection and defense pairing, rather than confining systems to parameter-centric protocols.
• Privacy Guarantees: Promotes the adoption of payloads amenable to formal guarantees (DP) and incentivizes research into empirically robust synthetic and embedding-based messaging.
• Task Specialization: Supports vertical FL, federated analytics, and cross-silo causal discovery beyond legacy deep learning objectives, broadening applicability.
• Benchmarking and Standardization: The emergence of the taxonomy will enable benchmarking suites that empirically quantify privacy/utility/cost trade-offs, fostering objective system design.

In terms of future work, additional literature curation and granular industry mapping, as well as standardized empirical evaluations, are proposed to further validate and operationalize the taxonomy.

Conclusion

The paper establishes a formal, inclusive framework for federated communication, advancing beyond the limited scope of weights and gradients. Through a functional definition, taxonomy, and literature-backed empirical analysis, it highlights the critical interplay between utility, privacy, computation, and communication in FL message design. The diversification of payload types and defensive mechanisms signals a shift toward strategic, context-aware architectures and the necessity for rigorous benchmarking. As federated learning matures, this taxonomy will guide the transition to holistic, robust, and efficient distributed systems that accommodate diverse industry-specific requirements.