SParSeFuL: Sparse Self-Federated Learning

• The paper introduces a novel framework that integrates peer-driven model comparison, decentralized federation, and neural network sparsification for resource-efficient learning.
• It employs a three-stage SCR protocol—sparse leader election, gradient diffusion, and converge-casting—to streamline model exchange and aggregation.
• Experimental results show minimal accuracy loss with significant energy and bandwidth savings, outperforming traditional FedAvg in non-IID, resource-constrained environments.

Sparse Proximity-based Self-Federated Learning (SParSeFuL) is a fully distributed federated learning framework that enables resource-constrained edge devices to self-organize into data-homogeneous federations based on pairwise model similarity, exchanging only sparsified neural representations to reduce energy and communication costs. SParSeFuL integrates three core principles: peer-driven model comparison (proximity), decentralized federation formation (self-federation), and explicit neural network sparsification (sparsity), to enable scalable, privacy-preserving collaborative intelligence in dynamic and heterogeneous environments (Domini et al., 2024, Domini et al., 10 Jul 2025).

1. Network Model, Objective, and Formalization

The SParSeFuL framework considers a set $S=\{s_1, \dots, s_n\}$ of devices deployed over a continuous domain $A$, partitioned into $k$ latent subregions $\{a_1,\dots,a_k\}$ with distinct data distributions $\Theta_j$. Each device $s_i$ possesses local data $\mathcal{D}_i$ sampled from an unknown $\Theta_j$, and can communicate with peers within a physical radius $r_c$ (one-hop neighborhood $\mathcal{N}_i$).

The optimization goal is to jointly:

• Self-organize $A$0 into federations $A$1 approximating the ground-truth partitions,
• Within each final federation $A$2 after $A$3 global rounds, learn a compressed model $A$4 minimizing both predictive loss and parameter count, i.e.,

$A$5

where $A$6 is the per-sample loss (e.g., cross-entropy) and $A$7 is the number of nonzero parameters, with $A$8 as a sparsity regularization parameter (Domini et al., 10 Jul 2025).

2. Proximity-based Self-Organization and Federation Dynamics

Federation formation in SParSeFuL uses a purely loss-driven dissimilarity metric without reference to geographic coordinates. Each node exchanges compressed models $A$9 with one-hop neighbors $k$0 and computes the pairwise cross-validation losses:

$k$1

The symmetric proximity score is:

$k$2

Devices cluster via the SCR (Self-Organizing Coordination Region) protocol, which proceeds in three stages:

• S-block (Sparse leader choice): Nodes randomly self-nominate as cluster seeds with probability $k$3.
• G-block (Gradient-cast): Dissimilarity values $k$4 are diffused to propagate cluster boundaries.
• C-block (Converge-cast): Models are aggregated at emergent leaders (Domini et al., 2024, Domini et al., 10 Jul 2025).

Each device joins the federation whose leader can be reached with cumulative dissimilarity $k$5, where $k$6 is a tunable path error.

3. Algorithmic Workflow: Sparse Model Exchange and Aggregation

At every round, SParSeFuL implements the following per-device steps:

1. Sparse Pruning: Each device computes a binary mask $k$7 keeping the largest $k$8 weights, achieving target sparsity ratio $k$9:

$\{a_1,\dots,a_k\}$0

with $\{a_1,\dots,a_k\}$1 set to the $\{a_1,\dots,a_k\}$2th percentile of $\{a_1,\dots,a_k\}$3.

1. Local Training: Devices perform $\{a_1,\dots,a_k\}$4 epochs of SGD on $\{a_1,\dots,a_k\}$5, updating only the nonzero weights.
2. Model Exchange: Compressed weights $\{a_1,\dots,a_k\}$6 are shared with $\{a_1,\dots,a_k\}$7; dissimilarity $\{a_1,\dots,a_k\}$8 is computed with each neighbor.
3. Federation Update via SCR: The three-block protocol elects leaders, diffuses cluster membership, and aggregates peers with $\{a_1,\dots,a_k\}$9.
4. Aggregation: Each cluster leader aggregates members' models using weighted FedAvg:

$\Theta_j$0

1. Distribution: Aggregated weights are distributed back, setting members’ models to $\Theta_j$1 for the next round.

This entire process exploits both explicit pruning and sparse leader election to minimize transmitted and updated parameters (Domini et al., 2024, Domini et al., 10 Jul 2025).

4. Communication, Energy, and Complexity Analysis

SParSeFuL analytically reduces both per-round communication and computation:

• Per-node computation (one epoch):

$\Theta_j$2

with $\Theta_j$3 model parameters, $\Theta_j$4 FLOPs per parameter, and $\Theta_j$5 energy/FLOP.

• Communication (compressed model):

$\Theta_j$6

where $\Theta_j$7 is bits per weight and $\Theta_j$8 is energy/bit transmitted.

• Bandwidth usage: decreases linearly with $\Theta_j$9.

For the leader, aggregation cost is $s_i$0 per federation per round. Sparse leader election and communication ensure the protocol scales with local density rather than total network size (Domini et al., 10 Jul 2025).

5. Theoretical Properties and Convergence

Analytical convergence of SParSeFuL is established under standard assumptions (smooth local loss, bounded gradient variance, fixed masks). The descent lemma and convergence theorem assert:

• With learning rate $s_i$1 and $s_i$2, the expected squared gradient norm approaches zero asymptotically:

$s_i$3

• The error term introduced by sparsification is explicitly controlled by the discrepancy in sparsity masks.

A plausible implication is that the protocol is robust to hyperparameter selection within broad ranges and the network exhibits strong self-stabilization properties after transient network churn (Domini et al., 10 Jul 2025).

6. Experimental Results and Comparative Performance

Extensive experiments were conducted on EMNIST and CIFAR-10, with $s_i$4 edge nodes and extreme non-IID splits. The main comparisons involve centralized FedAvg, dense PSFL (no sparsification), and SParSeFuL at various sparsity ratios.

Method	Accuracy	Rounds to 95%	Energy (%)	Bandwidth (%)
Dense PSFL ($s_i$5)	97.8	30	100	100
SParSeFuL ($s_i$6)	97.6	32	41	40
SParSeFuL ($s_i$7)	97.4	34	3	50
SParSeFuL ($s_i$8)	96.8	38	3	70

Key observations:

• $s_i$9 pruning yields only $\mathcal{D}_i$0 accuracy loss but $\mathcal{D}_i$1 energy and bandwidth reduction;
• At $\mathcal{D}_i$2, energy drops to under $\mathcal{D}_i$3 of baseline with $\mathcal{D}_i$4 accuracy loss;
• Convergence slows only modestly as sparsity increases (30 to 34 rounds);
• On strongly non-IID Extended-MNIST splits, SParSeFuL outperforms global FedAvg by $\mathcal{D}_i$5 percentage points in test accuracy when $\mathcal{D}_i$6 (Domini et al., 2024, Domini et al., 10 Jul 2025).

7. Applications and Broader Implications

SParSeFuL is positioned as a key solution for Society 5.0 scenarios, including large-scale IoT environments requiring privacy, sustainability, and resilience. Its combination of aggregate computing paradigms, resource-aware sparsification, and leaderless organization enables scalability and adaptability across highly dynamic or geographically skewed networks. The demonstrated reduction in energy and bandwidth cost aligns SParSeFuL with green AI initiatives for collaborative on-device intelligence (Domini et al., 10 Jul 2025).

This suggests SParSeFuL offers a practically viable and empirically validated direction towards sustainable federated learning at the network edge under resource and privacy constraints, surpassing classical methods on both accuracy and resource metrics.