A Faster Cutting Plane Method and its Implications for Combinatorial and Convex Optimization (1508.04874v2)

Abstract: We improve upon the running time for finding a point in a convex set given a separation oracle. In particular, given a separation oracle for a convex set $K\subset \mathbb{R}^n$ contained in a box of radius $R$, we show how to either find a point in $K$ or prove that $K$ does not contain a ball of radius $\epsilon$ using an expected $O(n\log(nR/\epsilon))$ oracle evaluations and additional time $O(n^3\log^{O(1)}(nR/\epsilon))$. This matches the oracle complexity and improves upon the $O(n^{\omega+1}\log(nR/\epsilon))$ additional time of the previous fastest algorithm achieved over 25 years ago by Vaidya for the current matrix multiplication constant $\omega<2.373$ when $R/\epsilon=n^{O(1)}$. Using a mix of standard reductions and new techniques, our algorithm yields improved runtimes for solving classic problems in continuous and combinatorial optimization: Submodular Minimization: Our weakly and strongly polynomial time algorithms have runtimes of $O(n^2\log nM\cdot\text{EO}+n^3\log^{O(1)}nM)$ and $O(n^3\log^2 n\cdot\text{EO}+n^4\log^{O(1)}n)$, improving upon the previous best of $O((n^4\text{EO}+n^5)\log M)$ and $O(n^5\text{EO}+n^6)$. Matroid Intersection: Our runtimes are $O(nrT_{\text{rank}}\log n\log (nM) +n^3\log^{O(1)}(nM))$ and $O(n^2\log (nM) T_{\text{ind}}+n^3 \log^{O(1)} (nM))$, achieving the first quadratic bound on the query complexity for the independence and rank oracles. In the unweighted case, this is the first improvement since 1986 for independence oracle. Submodular Flow: Our runtime is $O(n^2\log nCU\cdot\text{EO}+n^3\log^{O(1)}nCU)$, improving upon the previous bests from 15 years ago roughly by a factor of $O(n^4)$. Semidefinite Programming: Our runtime is $\tilde{O}(n(n^2+m^{\omega}+S))$, improving upon the previous best of $\tilde{O}(n(n^{\omega}+m^{\omega}+S))$ for the regime where the number of nonzeros $S$ is small.

• The paper presents a new cutting plane method that lowers oracle evaluation complexity to O(n log(nR/ε)), significantly improving runtime compared to previous methods.
• The algorithm enhances efficiency in various domains, notably submodular function minimization, matroid intersection, and semidefinite programming, by reducing high polynomial dependencies.
• The authors leverage improved, low variance unbiased leverage score estimations to achieve faster convergence and set a new theoretical benchmark in convex optimization.

An Analysis of an Advanced Cutting Plane Method for Convex Optimization Problems

This paper presents a significant advancement in the development of cutting plane methods aimed at solving convex optimization problems efficiently. The central contribution of the research is the introduction of a more rapid cutting plane algorithm that harmonizes with oracle complexity and surpasses previous computational timeframes. The paper's proposed algorithm achieves this by effectively calculating a point within a convex set or demonstrating the absence of such a point within an acceptable threshold.

Key Contributions

The authors propose an algorithm that, under a separation oracle for a convex set $K \subset \mathbb{R}^n$, achieves a complexity of $O(n \log(nR/\epsilon))$ evaluations of the oracle, accompanied by additional computational time of $O(n^3\log^{O(1)}(nR/\epsilon))$. This enhancement in computational efficacy surpasses the prior best available time of $O(n^{\omega+1}\log(nR/\epsilon))$ due to techniques described by Vaidya over two decades ago. As a point of reference, the parameter $\omega$, representing the matrix multiplication constant, currently stands at less than 2.373.

The breakthrough primarily leverages an insightful method of estimating low variance unbiased changes in leverage scores. The new mechanism provides substantial accuracy enhancements, critical for maintaining computational efficiency. These advances are built upon the foundation of existing methods and augmented with fresh analysis and novel reduction strategies.

Improved Application in Optimization Domains

The implications of this research extend into several realms of optimization. The authors articulate how their algorithm achieves superior running times specifically for combinatorial and continuous optimization problems, including:

1. Submodular Function Minimization: Enhanced algorithms present run times of $O(n^{2} \log nM\cdot\text{EO}+n^{3}\log^{O(1)}nM)$ and $O(n^{3}\log^{2}n\cdot\text{EO}+n^{4}\log^{O(1)}n)$. These metrics show a significant leap over previous methods, offering a substantial decrease in complexity by eliminating high polynomial dependencies on the problem size.
2. Matroid Intersection: The newly proposed solution reaches the unique milestone of a quadratic bound for query complexity concerning independence and rank oracles—establishing significant computational savings over the classical $O(n^{4})$ methodologies established in prior works.
3. Submodular Flow: This optimization problem sees similar improvements, with the proposed algorithm vastly reducing computational overhead, surpassing previous metrics by a factor of approximately $\tilde{O}(n^{4})$.
4. Semidefinite Programming: The paper reduces the dependency on zero-patterns in constraint matrices, reflected through a new bound of $\tilde{O}(n(n^{2}+m^{\omega}+S))$, which outperforms the precedent of $\tilde{O}(n(n^{\omega}+m^{\omega}+S))$.

Theoretical and Practical Implications

The contributions extend beyond achieving mere theoretical improvements. Cutting plane methods have been heavily utilized in practice, notably within semidefinite programming and other optimization tasks wherein the complexities of large-scale convex structures are frequent. Therefore, this method not only signifies an enhancement in theoretical understanding but also bears a potential to redefine execution frameworks in practical applications, including industrial processes and operational research initiatives.

Prospective Developments

The analytical strides made in this paper indicate potential avenues for future exploration, particularly in fine-tuning the strategies for leveraging score estimates and oracle communication. Enhancing the practicability of this method across even broader optimization contexts remains a viable and intriguing exploration avenue.

In conclusion, this paper epitomizes a blend of theoretical innovation with methodological evolution, substantiating the power and applicability of cutting plane methods in convex and combinatorial optimization domains. The results underline how methodological refinements can significantly reduce complexity and enhance computational speed, thereby setting a new benchmark for future research in the field.