Combining model circulation with alternative compression methods

Investigate the combination of model circulation with alternative momentum or model compression methods, including sparsification and knowledge-distillation-based compression.

Background

The paper uses dynamic affine quantization for momentum compression, while noting that sparsification could provide higher compression ratios and knowledge distillation could transfer knowledge rather than parameters. These alternatives have different state and data requirements that may interact with Tram-FL’s stateless model-circulation design. Their integration is explicitly deferred.

References

Combining such techniques with model circulation is left as future work, since they apply equally to all DFL baselines and do not affect the relative comparison in this paper.

— Tram-FL: Reducing Communication and Computation Costs through Sequential Model Circulation in Decentralized Federated Learning  (2610.07859 - Maejima et al., 6 Oct 2026) in Section 3.6, Momentum Compression