Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications (1902.08294v2)

Abstract: The min-max problem, also known as the saddle point problem, is a class of optimization problems which minimizes and maximizes two subsets of variables simultaneously. This class of problems can be used to formulate a wide range of signal processing and communication (SPCOM) problems. Despite its popularity, most existing theory for this class has been mainly developed for problems with certain special convex-concave structure. Therefore, it cannot be used to guide the algorithm design for many interesting problems in SPCOM, where various kinds of non-convexity arise. In this work, we consider a block-wise one-sided non-convex min-max problem, in which the minimization problem consists of multiple blocks and is non-convex, while the maximization problem is (strongly) concave. We propose a class of simple algorithms named Hybrid Block Successive Approximation (HiBSA), which alternatingly perform gradient descent-type steps for the minimization blocks and gradient ascent-type steps for the maximization problem. A key element in the proposed algorithm is the use of certain regularization and penalty sequences, which stabilize the algorithm and ensure convergence. We show that HiBSA converges to some properly defined first-order stationary solutions with quantifiable global rates. To validate the efficiency of the proposed algorithms, we conduct numerical tests on a number of problems, including the robust learning problem, the non-convex min-utility maximization problems, and certain wireless jamming problem arising in interfering channels.

• The paper introduces the Hybrid Block Successive Approximation (HiBSA) algorithm, a novel iterative method combining gradient descent and ascent with regularization to solve one-sided non-convex min-max problems.
• The paper provides thorough convergence analysis for HiBSA under different conditions, establishing that the algorithm converges to first-order stationary points.
• Numerical evidence demonstrates HiBSA's effectiveness and robust convergence in various SPCOM applications like distributed optimization, robust learning, and wireless interference handling.

Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications

In the domain of signal processing and communication (SPCOM), min-max optimization problems frequently arise, where a function of multiple variables is minimized and maximized simultaneously. While there is considerable existing research on convex-concave min-max problems, the less structured non-convex nature of many practical SPCOM problems presents unique challenges, which were addressed in the paper "Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications".

This paper introduces a novel approach and theoretical framework for solving one-sided non-convex min-max problems where only part of the variable space is non-convex, while the other part is concave. This class specifically represents problems where concavity is assumed for the maximization component, such as power control scenarios involving jammers and robust learning frameworks across multiple domains.

Methodology

The authors propose the Hybrid Block Successive Approximation (HiBSA) algorithm, an iterative method combining techniques of gradient descent for the minimization component and gradient ascent for the maximization component. To stabilize the optimization process and ensure convergence, HiBSA introduces regularization and penalty sequences. The convergence of the proposed algorithm is thoroughly analyzed under three different conditions based on the structure of the non-convex and concave components and develops conditions under which the algorithm converges to first-order stationary points.

Results

The paper provides strong numerical evidence supporting the efficacy of HiBSA in various SPCOM applications, such as distributed optimization in network settings, robust learning models, and wireless channel interference handling. HiBSA achieves quantifiable global convergence rates, and the analysis reveals that its complexity is competitive relative to existing methods. Specifically, the paper demonstrates robust convergence for cases where the problem’s maximization component is only concave, under proper regulation and penalty conditions.

Implications and Future Directions

The theoretical guarantees and empirical success of HiBSA suggest it as a promising tool for scenarios where non-convex optimization is inevitable. The structured approach to handling non-convex elements block-by-block, paired with the stabilization achieved through carefully selected algorithm parameters, marks a significant advance in tackling these complex problems. Future developments could explore extending the framework to problems where non-convex structures are present in both minimization and maximization components or develop further integration with stochastic optimization techniques for scalability in large-scale systems and datasets. Such developments would broaden the applicability of these methods, offering deeper insights into dynamic SPCOM systems under real-world constraints.