MUFFLe: Efficient Model Update Compression via Generalized Deduplication for Federated Learning

Abstract: Federated learning is well suited to edge environments but is often limited by the uplink cost of transmitting model updates. This Work-in-Progress paper presents MUFFLe, a communication-efficient update compression scheme that integrates generalized deduplication (GD) into the FedAvg pipeline. MUFFLe deduplicates repeated patterns across the update vector, yielding a fixed-rate, variable-count compression scheme. Preliminary experiments on IID MNIST with 20 clients show that MUFFLe reaches the target accuracy of $92.93\%$ with 38~MB cumulative uplink communication, compared with 75~MB for 8-bit quantization, 86~MB for Top-$k$ sparsification, and 310~MB for uncompressed FedAvg. These results demonstrate the feasibility of applying GD to communication-efficient federated learning.

• The paper introduces MUFFLe, a lossy generalized-deduplication method that compresses repeated bit patterns in full federated learning updates instead of uniformly quantizing or sparsifying entries.
• MUFFLe reaches 92.93% accuracy on IID MNIST in 195 rounds with 38 MB of cumulative uplink communication, cutting traffic by 8.3× versus uncompressed FedAvg and roughly half versus 8-bit quantization.
• The results show that update redundancy can reduce communication without error feedback, but validation on non-IID data, larger models, client computation costs, and combinations with other compression methods remains necessary.

MUFFLe is a work-in-progress compression scheme for federated learning (FL) that applies generalized deduplication (GD) to client-to-server model updates. Rather than uniformly reducing bit-width as in quantization, or transmitting a subset of entries as in Top-$k$ sparsification, MUFFLe exploits repeated bit patterns across the full update vector, producing a fixed-rate, variable-count encoding. On IID MNIST with 20 clients under FedAvg, the scheme reaches the target accuracy of 92.93% with 38 MB cumulative uplink communication, versus 75 MB for 8-bit quantization, 86 MB for Top-$k$, and 310 MB for uncompressed FedAvg — an 8.3× reduction over the baseline.

Motivation and problem formulation

The paper targets the uplink bottleneck in edge-deployed FL, where clients transmit local updates $\Delta w_t^u = w_{t,E}^u - w_t$ after $E$ epochs of local SGD, and the server aggregates them by sample-weighted averaging (2606.14354). The optimization objective is framed in terms of cumulative communication volume: minimize $$\mathcal{B} = \sum_{t=1}^{T^*}\sum_{u\in\mathcal{S}_t}\text{size}(\mathcal{C}(\Delta w_t^u))$$ required to reach a target accuracy $\mathcal{A}_{target}$, where $T^*$ is the first round at which the threshold is met. This formulation explicitly captures the trade-off that aggressive compression reduces per-round cost but may inflate $T^*$ through reconstruction error.

Two baselines are formalized: quantization, which maps each scalar to a $q$-bit signed integer scaled by $S = \|\Delta w_t^u\|_\infty/(2^{q-1}-1)$ at a cost of $k$0 bits; and Top-$k$1 sparsification, which transmits $k$2 (value, index) tuples at a cost of $k$3 bits.

Method

GD partitions data chunks into recurring high-order bit patterns ("bases") and residual "deviations," achieving lossless compression by storing each unique base once alongside base IDs and deviations. MUFFLe adapts this framework to lossy update compression:

• Updates are preprocessed into an unsigned integer binary representation following prior GD work on floating-point data.
• Constant bits shared across all $k$4 elements are identified, and only $k$5 non-constant most-significant bits (MSBs) are retained per element, forming the fixed-rate component.
• Elements sharing the same $k$6-bit MSB pattern are deduplicated into a single base entry referenced by ID.
• Crucially, deviations are discarded entirely, making the scheme lossy — unlike existing GD methods, which store deviations for exact reconstruction.

The transmitted payload consists of a dictionary of $k$7 unique bases plus one ID per element, giving a total cost of $k$8 bits. Because $k$9 depends on the redundancy structure of each round's update, the scheme is variable-count: redundancy across the update directly translates into savings. The authors note MUFFLe is orthogonal to quantization and sparsification and could be composed with them, though no such combination is evaluated here.

Experimental results

The evaluation uses IID MNIST partitioned across $\Delta w_t^u = w_{t,E}^u - w_t$0 clients, with $\Delta w_t^u = w_{t,E}^u - w_t$1 client sampling per round ($\Delta w_t^u = w_{t,E}^u - w_t$2 participants), an MLP with one 128-unit ReLU hidden layer, SGD at learning rate 0.01 with $\Delta w_t^u = w_{t,E}^u - w_t$3 local epoch, and fixed random seeds. Updates are rounded to four decimal places before preprocessing. Target accuracy is defined as the test accuracy of the uncompressed baseline at 200 rounds.

Method	Accuracy (%)	Round	Comm. cost (MB)	Gain
Uncompressed	92.93	200	310	1×
Quantization (8-bit)	92.93	193	75	4.1×
Top-$\Delta w_t^u = w_{t,E}^u - w_t$4	92.93	221	86	3.6×
MUFFLe	92.93	195	38	8.3×

Notably, MUFFLe reaches the target in 195 rounds — essentially matching the convergence speed of uncompressed FedAvg and faster than Top-$\Delta w_t^u = w_{t,E}^u - w_t$5 (221 rounds) — while using roughly half the communication of 8-bit quantization. The reported gain over baselines persists across accuracy levels and peaks near the target, exceeding 2× over both quantization and Top-$\Delta w_t^u = w_{t,E}^u - w_t$6. These numbers indicate that cross-element redundancy in FL updates is substantial enough to be exploited profitably without error feedback or compensation mechanisms, none of which are applied in this study.

Limitations and open questions

The paper is explicit about its preliminary scope. Evaluation is confined to IID MNIST with a lightweight MLP; non-IID data distributions and larger models — settings where update statistics, and hence base redundancy, may differ considerably — are deferred to future work. No error feedback or residual accumulation is used; the authors propose storing discarded deviation bits and carrying them into subsequent rounds as a possible extension that could also reduce device RAM. The choice of the retained bit-width $\Delta w_t^u = w_{t,E}^u - w_t$7 and its interaction with convergence is not analyzed, nor is the computational overhead of base extraction on resource-constrained clients measured. Whether the observed gains hold when MUFFLe is combined with quantization or sparsification remains untested.

Conclusion

MUFFLe demonstrates that generalized deduplication can serve as a viable mechanism for lossy federated update compression, delivering an 8.3× cumulative uplink reduction on IID MNIST while preserving convergence speed relative to uncompressed FedAvg. The result establishes feasibility rather than generality: extending the evidence to non-IID heterogeneity, deeper architectures, and error-feedback integration constitutes the immediate open agenda of this line of work.