Secure Aggregation with Top-K Sparsification in Decentralized Federated Learning

Abstract: Secure aggregation is a vital component for mitigating gradient leakage in federated learning, but its communication cost conventionally scales with the gradient dimension. This becomes prohibitive for large models and even more pronounced in decentralized federated learning with limited bandwidth and unreliable nodes. Top-K gradient sparsification is an effective approach to reduce communication by transmitting only a few entries of the full gradient, while maintaining competitive model accuracy. Nevertheless, the top-K entries selected by each user are unpredictable and vary across users, which poses a challenge for efficient sparse secure aggregation. This paper studies information-theoretic secure aggregation with top-K sparsification in decentralized federated learning under user dropouts and user collusion. We propose a communication-efficient sparse secure aggregation scheme that offloads dimension-dependent overhead to an offline phase and protects private gradients using random masks and permutations. Experimental results demonstrate that our scheme preserves accuracy comparable to full-gradient aggregation even with only 1% gradient sparsification, while substantially reducing the communication cost.

• The paper introduces an information-theoretic secure aggregation scheme that combines random permutations, one-time masks, and Lagrange-coded secret sharing to hide top-K supports while tolerating dropouts and colluding users.
• The construction achieves rates of approximately (K + log_q binom(L,K))/L in the masked-input phase and 1/(U−T) in mask elimination, making online communication nearly independent of model dimension for sparse updates.
• Experiments with CIFAR-10 and ResNet-20 show accuracy close to unsecured training at 1% sparsification and dropout rates up to 50%, although full-dimensional permutation sharing remains an offline scalability bottleneck.

This paper studies information-theoretic secure aggregation (SecAgg) with top-$K$ gradient sparsification in decentralized federated learning (DFL), addressing two practical constraints—user dropouts and user collusion—that prior information-theoretic DSA constructions did not handle jointly with sparsification (2606.10780).

Motivation and problem setting

Secure aggregation protects against gradient reconstruction attacks by transmitting only masked updates, but conventional protocols incur communication that scales with the full gradient dimension $L$, which is prohibitive for large models and especially acute in serverless DFL with bandwidth-limited, unreliable nodes. Gradient sparsification mitigates this: random-$K$ sampling has been treated information-theoretically by Sami and Güler, but top-$K$ selection—which retains the $K$ largest-magnitude entries and empirically yields better accuracy—had not been addressed. The core difficulty is that top-$K$ support sets $\mathcal{K}_n$ are functions of the gradients themselves, so hiding them requires additional machinery beyond masking values.

The system model comprises $N$ users over a fully connected peer-to-peer broadcast network, each holding a length-$L$ input $W_n$ over $L$0. A scheme operates in two phases—a masked-input phase broadcasting $L$1 and a mask-elimination phase broadcasting $L$2—and must satisfy correctness for all dropout patterns with at least $L$3 survivors across both phases, and $L$4-privacy in the strong sense that any coalition of up to $L$5 users learns nothing beyond the sparse aggregate and their own inputs. Communication rates are $L$6 and $L$7.

The proposed scheme

The construction combines three ingredients. First, each user samples an independent uniform permutation $L$8 of $L$9 (equivalently a permutation matrix $K$0) to conceal its top-$K$1 support set; since the permuted indices are uniform regardless of $K$2, index leakage is eliminated. Second, each user draws a random mask vector $K$3 to one-time-pad the selected gradient values. Third, the permutation matrix rows and the masked products $K$4 are secretly shared via secure Lagrange encoding polynomials of degree at most $K$5, evaluated at public points $K$6, with $K$7 noise coordinates providing collusion resilience.

In the masked-input phase, user $K$8 broadcasts only $K$9—that is, $K$0 masked symbols plus $K$1 obfuscated indices. In the mask-elimination phase, each survivor broadcasts

$K$2

which is an evaluation of a degree-$K$3 polynomial whose coefficients cancel the masks. With $K$4 and hence $K$5, any $K$6 evaluations suffice for Lagrange interpolation; evaluating the recovered polynomial at the points $K$7 yields the unmasked aggregate because $K$8.

The main result is stated as a theorem: for all parameters with $K$9 and $K$0, there exists a scheme achieving

$K$1

Two implications follow directly. The first-phase cost is now dimension-independent apart from the combinatorial term $K$2, which vanishes as $K$3 grows for fixed sparsity ratio—for $K$4 this overhead is negligible—so online communication decouples from model size, unlike prior DSA constructions whose costs scale with $K$5. The second-phase rate $K$6 matches existing two-phase SecAgg schemes, so the improvement comes entirely from the first phase. The paper also notes the scheme transfers verbatim to centralized FL, where the server's joint view with $K$7 colluding users coincides with the decentralized adversary's view.

Experiments

Experiments use CIFAR-10 with ResNet-20, $K$8 users under non-IID partitioning, $K$9, $K$0, and $K$1. Across dropout rates from 0% to 50%, the proposed scheme tracks the no-SecAgg baseline closely in test accuracy while substantially outperforming secure aggregation with random-$K$2 sparsification in both convergence speed and final accuracy. This supports the claim that top-$K$3 selection preserves accuracy even at 1% sparsification, and that the privacy mechanism does not degrade training despite dropout rates up to 50%.

Limitations and open questions

The paper is explicit about one significant cost: the offline phase requires secretly sharing full-dimensional permutation matrices, so offline communication and storage scale with $K$4. The authors state plainly that this limits scalability to large-scale models and identify relaxing information-theoretic security to computational security via shared short cryptographic seeds as the natural remedy—an open direction they do not pursue. Two further assumptions deserve note: the analysis presumes $K$5 (implicitly required by the block partition) and a finite field of size $K$6, while real-valued gradients must be quantized into $K$7, with top-$K$8 selection performed in the real domain before quantization. Finally, the scheme provides achievability only; no converse or optimality proof is given for the rates $K$9 and $\mathcal{K}_n$0, leaving open whether these rates are information-theoretically optimal for the top-$\mathcal{K}_n$1 DSA problem.

Conclusion

The paper formulates and solves information-theoretic secure aggregation with top-$\mathcal{K}_n$2 sparsification in DFL under simultaneous dropouts and collusion, using random permutations to hide gradient-dependent support sets and Lagrange-coded secret sharing to enable dropout-robust mask cancellation. It achieves a first-phase rate essentially independent of gradient dimension and matches the standard second-phase rate $\mathcal{K}_n$3, with experiments confirming baseline-comparable accuracy at 1% sparsification. The principal unresolved issues are the dimension-dependent offline overhead and the absence of matching lower bounds.