A Multi-Objective Optimization framework for Decentralized Learning with coordination constraints

Abstract: This article introduces a generalized framework for Decentralized Learning formulated as a Multi-Objective Optimization problem, in which both distributed agents and a central coordinator contribute independent, potentially conflicting objectives over a shared model parameter space. Unlike traditional approaches that merge local losses under a common goal, our formulation explicitly incorporates coordinator-side criteria, enabling more flexible and structured training dynamics. To navigate the resulting trade-offs, we explore scalarization strategies, particularly weighted sums, to construct tractable surrogate problems. These yield solutions that are provably Pareto optimal under standard convexity and smoothness assumptions, while embedding global preferences directly into local updates. We propose a decentralized optimization algorithm with convergence guarantees, and demonstrate its empirical performance through simulations, highlighting the impact of the coordinator's influence on local agent behavior. The proposed approach offers a principled and customizable strategy for balancing personalization, fairness, and coordination in decentralized learning systems.