Distributed Online Optimization in Dynamic Environments Using Mirror Descent

Abstract: This work addresses decentralized online optimization in non-stationary environments. A network of agents aim to track the minimizer of a global time-varying convex function. The minimizer evolves according to a known dynamics corrupted by an unknown, unstructured noise. At each time, the global function can be cast as a sum of a finite number of local functions, each of which is assigned to one agent in the network. Moreover, the local functions become available to agents sequentially, and agents do not have a prior knowledge of the future cost functions. Therefore, agents must communicate with each other to build an online approximation of the global function. We propose a decentralized variation of the celebrated Mirror Descent, developed by Nemirovksi and Yudin. Using the notion of Bregman divergence in lieu of Euclidean distance for projection, Mirror Descent has been shown to be a powerful tool in large-scale optimization. Our algorithm builds on Mirror Descent, while ensuring that agents perform a consensus step to follow the global function and take into account the dynamics of the global minimizer. To measure the performance of the proposed online algorithm, we compare it to its offline counterpart, where the global functions are available a priori. The gap between the two is called dynamic regret. We establish a regret bound that scales inversely in the spectral gap of the network, and more notably it represents the deviation of minimizer sequence with respect to the given dynamics. We then show that our results subsume a number of results in distributed optimization. We demonstrate the application of our method to decentralized tracking of dynamic parameters and verify the results via numerical experiments.

• The paper introduces a decentralized mirror descent algorithm that tracks minimizers of dynamic convex functions despite adversarial noise.
• It establishes a dynamic regret bound that scales with network spectral gaps and the variation measure C_T, ensuring sub-linear regret when C_T is sub-linear.
• The study extends the analysis to stochastic gradients, demonstrating robust performance in noisy environments and promising adaptive online optimization.

Decentralized Online Optimization in Dynamic Environments Using Mirror Descent

The paper by Shahrampour and Jadbabaie presents a comprehensive study on decentralized online optimization under the constraints of non-stationary environments, leveraging the mirror descent algorithm. This research addresses the scenario where a network of agents aims to track the minimizer of a global time-varying convex function. The function's minimizer evolves following a known dynamic pattern, albeit corrupted by unknown and unstructured noise. The global function is represented as a sum of local functions distributed across agents, which require communication to collaboratively approximate and minimize it.

Key Contributions and Methodology

1. Problem Formulation: The paper considers a challenging class of problems—decentralized optimization in a dynamic setup. Each agent in the network only has partial and potentially outdated information about the global cost function, which evolves in time according to a specified dynamic model with an additive adversarial noise component.
2. Algorithmic Approach: The authors propose a decentralized variant of the mirror descent algorithm, well-known for its applicability in large-scale optimization. The implementation includes three major steps:
   • Local Update: Each agent adjusts its estimate by following the local gradient and minimizing its divergence from the local neighborhood average.
   • Consensus: Agents average their estimates with neighboring agents to achieve a consensus that aids in approximating the global minimizer.
   • Dynamic Adjustment: Incorporating known dynamics to accommodate time-varying function properties and tracking deviations caused by noise.
3. Performance Metric: The efficacy of the proposed algorithm is evaluated using dynamic regret, which measures the difference between the accumulated cost incurred by the algorithm and the ideal cost that would be achieved if future information were available. The authors derive a regret bound that highlights the algorithm's sensitivity to both network communication restrictions and inherent dynamic deviations in the parameter being tracked.
4. Regret Analysis: Shahrampour and Jadbabaie establish a regret bound that scales inversely with the spectral gap of the network, emphasizing the importance of inter-agent communication. They further express this bound in terms of a complexity measure related to the variation of the minimizer sequence, known as $C_T$, and argue that if $C_T$ remains sub-linear, so does the regret.
5. Stochastic Gradient Extension: A consideration of stochastic gradients revealed that the regret analysis extends to scenarios where agents have access to noisy gradient estimates, maintaining the original bound in expectation.

Implications and Future Directions

The research renders substantial implications for both theoretical and practical applications in decentralized optimization within AI and control systems frameworks. It extends the applicability of mirror descent methods to distributed systems subject to dynamic changes. The bounds developed serve as benchmarks for evaluating system resilience to adversarial conditions imposed by dynamic environments. Moreover, by providing a path to decentralized tracking and estimation through finite-horizon analysis, the paper introduces promising directions for adaptive algorithms in online learning.

Future developments could hone in on adaptive step-size selection and better handling of adversarial noise models that are not fully observable. Exploring the potential for leveraging multiple gradient queries per step might also reduce the tracking error in dynamic environments. These advancements could further streamline the integration of such algorithms in practical applications facing dynamic and uncertain settings.