Bilevel Distributed Optimization in Directed Networks

Abstract: Motivated by emerging applications in wireless sensor networks and large-scale data processing, we consider distributed optimization over directed networks where the agents communicate their information locally to their neighbors to cooperatively minimize a global cost function. We introduce a new unifying distributed constrained optimization model that is characterized as a bilevel optimization problem. This model captures a wide range of existing problems over directed networks including: (i) Distributed optimization with linear constraints; (ii) Distributed unconstrained nonstrongly convex optimization over directed networks. Employing a novel regularization-based relaxation approach and gradient-tracking schemes, we develop an iteratively regularized push-pull gradient algorithm. We establish the consensus and derive new convergence rate statements for suboptimality and infeasibility of the generated iterates for solving the bilevel model. The proposed algorithm and the complexity analysis obtained in this work appear to be new for addressing the bilevel model and also for the two sub-classes of problems. The numerical performance of the proposed algorithm is presented.