Sharper Convergence Guarantees for Federated Learning with Partial Model Personalization

Abstract: Partial model personalization, which encompasses both shared and personal variables in its formulation, is a critical optimization problem in federated learning. It balances individual client needs with collective knowledge utilization, and serves as a general formulation covering various key scenarios, ranging from fully shared to fully personalized federated learning. This paper introduces two effective algorithms, FedAvg-P and Scaffold-P, to solve this problem and provides sharp convergence analyses, quantifying the influence of gradient variance, local steps, and partial client sampling on their performance. Our established rates surpass existing results and, meanwhile, are based on more relaxed assumptions. Additionally, our analyses are also applicable to fully shared or fully personalized federated learning, matching or even outperforming their best known convergence rates. Numerical experiments corroborate our theoretical findings.