Federated Daisy-Chaining (FedDC)

• Federated Daisy-Chaining is a federated learning approach that interleaves periodic random model shuffling (daisy-chaining) with standard aggregation to mix client influences.
• The algorithm offers rigorous theoretical guarantees, including optimization and PAC-style bounds, ensuring reliable performance on small, heterogeneous datasets.
• Empirical evaluations across synthetic, medical, and vision datasets demonstrate that FedDC outperforms traditional federated methods, achieving higher accuracy and robustness.

Federated Daisy-Chaining (FedDC) is a federated learning (FL) algorithm designed to enable robust, privacy-preserving model training across multiple clients whose local datasets are extremely small or highly heterogeneous. It introduces a daisy-chaining mechanism—periodic, random shuffling of client models between participants—interleaved with standard model aggregation, addressing failure modes of conventional federated methods in data-sparse regimes by mixing local data influences across sites while maintaining core privacy properties. The approach provides formal theoretical guarantees, practical guidance on hyperparameters, and demonstrates empirically superior performance to standard federated strategies, particularly when local data availability is scant or non-i.i.d. (Kamp et al., 2021).

1. Mathematical Framework

Federated Daisy-Chaining builds on the standard federated learning setup. Consider $K$ clients, with client $k$ holding private data $\mathcal{D}_k$ of size $n_k$, model parameters $w \in \mathbb{R}^d$, local objective $$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$, and a global objective $F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$, where $N=\sum_k n_k$.

In standard FedAvg, each client performs $T_{local}$ steps of local SGD:

$w_k^{t+1} = w_k^t - \eta \nabla \ell(w_k^t; S_k^t)$

with aggregation at the server via arithmetic mean. FedDC modifies this as follows:

• Daisy-Chain Step (period $k$0): For every $k$1 local-update steps, the server samples a uniform random permutation $k$2 of $k$3. Each model $k$4 is sent unchanged to client $k$5, so it traverses a "daisy-chain" of clients.
• Aggregation Step (period $k$6): Every $k$7 steps, server aggregates models using a combinator (arithmetic mean or Radon point), and broadcasts the aggregate to all clients.

The update process can be formalized as:

• Local Update: $k$8.
• Daisy-Chain Exchange (if $k$9d): $\mathcal{D}_k$0.
• Aggregation (if $\mathcal{D}_k$1b): $\mathcal{D}_k$2, $\mathcal{D}_k$3.

Parameters $\mathcal{D}_k$4 (daisy-chain period), $\mathcal{D}_k$5 (aggregation period), SGD rules, and permutation schedule govern the behavior.

2. Algorithmic Description

Federated Daisy-Chaining operates as follows:

$F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$7

Key hyperparameters:

• $\mathcal{D}_k$6: Daisy-chain period (frequency of model shuffling)
• $\mathcal{D}_k$7: Aggregation period (frequency of averaging)
• Permutation strategy: Uniform random recommended; stratified variants possible

3. Theoretical Guarantees

FedDC provides both optimization and learning-theoretic guarantees:

• Optimization Rate: Under $\mathcal{D}_k$8-smoothness and bounded variance/stochasticity, with arithmetic mean aggregation, the method achieves:

$\mathcal{D}_k$9

for $n_k$0 local-update steps per client (Corollary 1). This parallels optimal rates for parallel restarted SGD.

• PAC-Style Guarantees: When aggregation uses Radon point combinators, and each client empirical learner $n_k$1 achieves $n_k$2 on $n_k$3 local samples, FedDC ensures—with high probability—that each model traverses at least $n_k$4 distinct clients after $n_k$5 rounds, where $n_k$6 is the $n_k$7-th harmonic number. Thus, after sufficient mixing, the terminal aggregate $n_k$8 satisfies $n_k$9 for suitable $w \in \mathbb{R}^d$0, improving sample efficiency and reliability compared to baseline FL with very small per-client samples (Corollary 2, Proposition 5, Lemma A.1).

4. Empirical Evaluation

Evaluation covers synthetic, tabular, and real-world medical and vision datasets. Key benchmarks include:

• SUSY (K=441, n=2/client, linear, Radon aggregation):
  • Centralized: 77%
  • FedDC ($w \in \mathbb{R}^d$1=1, $w \in \mathbb{R}^d$2=50): 77%
  • FedAvg ($w \in \mathbb{R}^d$3=1): 68%
  • FedAvg ($w \in \mathbb{R}^d$4=50): 64%
• Synthetic data (K=50, n=10, MLP):
  • FedDC ($w \in \mathbb{R}^d$5=1, $w \in \mathbb{R}^d$6=200): 89%
  • FedAvg ($w \in \mathbb{R}^d$7=1): 80%, ($w \in \mathbb{R}^d$8=200): 76%
• CIFAR10 (ResNet18, K=150, n=64):
  • Centralized: 65.1%
  • FedDC ($w \in \mathbb{R}^d$9=1, $$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$0=10): 62.9%
  • FedAvg ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$1=1): 55.8%, ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$2=10): 48.7%
  • FedProx: 51.1%, FedAdam: 34.0%
• Non-IID CIFAR10 (2 classes/client):
  • FedDC: 34.2%
  • FedAvg ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$3=1): 30.2%, ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$4=10): 24.9%
  • FedProx: 32.8%
• Brain MRI (K=25, n=8):
  • Centralized: 82.1%
  • FedDC ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$5=1, $$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$6=10): 78.4%
  • FedAvg ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$7=1): 74.1%, ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$8=10): 75.6%
  • FedProx: 76.5%
• Pneumonia X-ray (K=150, n=8):
  • Centralized: 84.1%
  • FedDC: 83.2%
  • FedAvg ($$F_k(w) = \mathbb{E}_{(x,y)\sim \mathcal{D}_k}[\ell(w;x,y)]$$9=1): 80.1%, ($F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$0=10): 79.4%
  • FedProx: 80.0%

Combinations of FedDC+FedProx further improve IID CIFAR10 (to 63.2%). FedDC retains $F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$1 accuracy drop under Gaussian differential privacy noise (CIFAR10, K=250, small ResNet) (Kamp et al., 2021).

5. Practical Implementation and Hyperparameter Selection

• Privacy and Security: Daisy-chaining increases exposure of client models, raising potential membership-inference concerns. Mitigation involves applying differentially private noise (clipping plus Gaussian), secure encrypted communication (TLS, homomorphic aggregation), and robust aggregators (Radon point, coordinate-wise median) to defend against adversarial behaviors.
• Communication Overhead: FedDC incurs approximately $F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$2 communication rounds versus $F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$3 for FedAvg. For cross-silo FL (e.g., hospital consortia), this overhead is typically negligible; for cross-device deployments, larger $F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$4 or client subsampling can be used without loss of consistency or practical gains.
• Hyperparameter Guidance:
  • $F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$5 is recommended for maximal model mixing and dataset coverage.
  • Aggregation period $F(w) = \sum_{k=1}^K (n_k/N) F_k(w)$6 trades off bias and variance: selection can follow theoretical bounds or be optimized via grid search.
  • Uniform random permutation of the chain at each round is preferred, but stratified/weighted permutations are feasible for non-uniform client distributions.

6. Context and Impact in Federated Learning

Federated Daisy-Chaining addresses a fundamental limitation of standard FL—failure in small-sample, non-i.i.d. client regimes—by algorithmically interleaving model permutations with secure aggregations. Theoretical analysis confirms both convergent optimization and enhanced generalization in PAC terms with small or imbalanced local datasets. Benchmarks demonstrate substantive gains across diverse domains, typically preserving core privacy requirements and introducing modest, controllable communication cost. The approach is particularly suited for applications such as multi-institutional medical learning or decentralized vision analytics, where local data regimes are harsh and privacy/social constraints are paramount (Kamp et al., 2021).