Federated Wasserstein Distance (2310.01973v1)
Abstract: We introduce a principled way of computing the Wasserstein distance between two distributions in a federated manner. Namely, we show how to estimate the Wasserstein distance between two samples stored and kept on different devices/clients whilst a central entity/server orchestrates the computations (again, without having access to the samples). To achieve this feat, we take advantage of the geometric properties of the Wasserstein distance -- in particular, the triangle inequality -- and that of the associated {\em geodesics}: our algorithm, FedWad (for Federated Wasserstein Distance), iteratively approximates the Wasserstein distance by manipulating and exchanging distributions from the space of geodesics in lieu of the input samples. In addition to establishing the convergence properties of FedWad, we provide empirical results on federated coresets and federate optimal transport dataset distance, that we respectively exploit for building a novel federated model and for boosting performance of popular federated learning algorithms.
- Geometric approximation via coresets. Combinatorial and computational geometry, 52(1):1–30, 2005.
- Barycenters in the wasserstein space. SIAM Journal on Mathematical Analysis, 43(2):904–924, 2011.
- Geometric dataset distances via optimal transport. Advances in Neural Information Processing Systems, 33:21428–21439, 2020.
- Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2005.
- Federated learning with personalization layers. arXiv preprint arXiv:1912.00818, 2019.
- Diffeomorphic density matching by optimal information transport. SIAM Journal on Imaging Sciences, 8(3):1718–1751, 2015.
- Wasserstein measure coresets. arXiv preprint arXiv:1805.07412, 2018.
- Exploiting Shared Representations for Personalized Federated Learning. In International Conference on Machine Learning, pp. 2089–2099, 2021.
- Learning wasserstein embeddings. In International Conference on Learning Representations (ICLR), 2018.
- On the rate of convergence in wasserstein distance of the empirical measure. Probability theory and related fields, 162(3-4):707–738, 2015.
- Sketching data sets for large-scale learning: Keeping only what you need. IEEE Signal Processing Magazine, 38(5):12–36, 2021.
- Manifold interpolating optimal-transport flows for trajectory inference. Advances in Neural Information Processing Systems, 35:29705–29718, 2022.
- Advances and Open Problems in Federated Learning. Foundations and Trends in Machine Learning, 14(1–2):1–210, 2021.
- Optimal mass transport: Signal processing and machine-learning applications. IEEE signal processing magazine, 34(4):43–59, 2017.
- Differentially private optimal transport: Application to domain adaptation. In IJCAI, pp. 2852–2858, 2019.
- Secure efficient federated knn for recommendation systems. In Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery, pp. 1808–1819. Springer, 2021.
- Transport based image morphing with intensity modulation. In Scale Space and Variational Methods in Computer Vision: 6th International Conference, SSVM 2017, Kolding, Denmark, June 4-8, 2017, Proceedings, pp. 563–577. Springer, 2017.
- Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp. 1273–1282. PMLR, 2017.
- On coresets for logistic regression. Advances in Neural Information Processing Systems, 31, 2018.
- Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
- Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019.
- Jeff M Phillips. Coresets and sketches. arXiv preprint arXiv:1601.00617, 2016.
- Private nearest neighbors classification in federated databases. IACR Cryptol. ePrint Arch., 2018:289, 2018.
- Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17:395–416, 2007.
- A Field Guide to Federated Optimization. arXiv preprint arXiv:2107.06917, 2021.
- Approximate k-nearest neighbor query over spatial data federation. In Database Systems for Advanced Applications: 28th International Conference, DASFAA 2023, Tianjin, China, April 17–20, 2023, Proceedings, Part I, pp. 351–368. Springer, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
