Solving Linear Programs in the Current Matrix Multiplication Time (1810.07896v3)

Abstract: This paper shows how to solve linear programs of the form $\min_{Ax=b,x\geq0} c^\top x$ with $n$ variables in time $$O^*((n^{\omega}+n^{2.5-\alpha/2}+n^{2+1/6}) \log(n/\delta))$$ where $\omega$ is the exponent of matrix multiplication, $\alpha$ is the dual exponent of matrix multiplication, and $\delta$ is the relative accuracy. For the current value of $\omega\sim2.37$ and $\alpha\sim0.31$, our algorithm takes $O^*(n^{\omega} \log(n/\delta))$ time. When $\omega = 2$, our algorithm takes $O^*(n^{2+1/6} \log(n/\delta))$ time. Our algorithm utilizes several new concepts that we believe may be of independent interest: $\bullet$ We define a stochastic central path method. $\bullet$ We show how to maintain a projection matrix $\sqrt{W}A^{\top}(AWA^{\top})^{-1}A\sqrt{W}$ in sub-quadratic time under $\ell_{2}$ multiplicative changes in the diagonal matrix $W$.

• The paper introduces a stochastic central path method that leverages O(√n) coordinate updates and O(n) steps to improve linear program solving efficiency.
• It aligns the algorithm's runtime with the matrix multiplication exponent, translating advances in matrix multiplication directly into LP optimization gains.
• The approach incorporates a novel projection maintenance strategy and an adaptive potential function to ensure numerical stability and bound computational complexity.

Solving Linear Programs in the Current Matrix Multiplication Time: An Essay

The paper by Cohen, Lee, and Song endeavors to address a long-standing challenge in computational optimization: efficiently solving linear programs (LPs). The authors propose an innovative stochastic central path method designed to improve the efficiency of solving LPs, which are ubiquitous in both theoretical computer science and practical applications. Remarkably, the proposed method achieves a thorough integration with the current capabilities of matrix multiplication algorithms.

Key Contributions of the Paper

1. Algorithmic Framework: The authors present an algorithm with a runtime bounded by $$O(n^{2.5 - a/2 + o(1)} + n^{\omega + o(1)} + n^{1.5 + a + o(1)}) \cdot \log(1/\epsilon)$$, where $a$ is a parameter controlling the batch size. This runtime aligns with $\omega$, the exponent of matrix multiplication. This alignment is crucial because it translates matrix multiplication speed advancements directly into linear programming improvements.
2. Short Step Central Path Method: The proposed stochastic variant of the central path method takes $O(n)$ steps while only updating $O(\sqrt{n})$ coordinates per step with an expected total of $O(n)$ coordinate updates. The central path method maintains feasible solutions on the path toward optimality by utilizing a form of gradient descent, which has been modified for efficiency.
3. Projection Maintenance: A significant technical advancement in the paper is the introduction of a new projection maintenance approach, which aids the efficient updating of matrix inverses under multiplicative changes in the diagonal matrix $W$. This is crucial for maintaining computational efficiency in large-scale problems.
4. Adaptive Potential Function: The paper introduces a potential function that helps in bounding the complexity of the algorithm. This function decreases with each step, ensuring that the algorithm remains on target.
5. Numerical Considerations: Addressing the potential pitfalls of floating-point arithmetic, the authors ensure numerical stability by substantiating their claims with rigorous bounds on errors and numerical tolerances.

Theoretical and Practical Implications

The implications of this work are profound in both theoretical and practical domains. Theoretically, the algorithm's efficiency demonstrates the potential of tying LP solving to advancements in matrix multiplication. This integration offers a new dimension of problem-solving capability unmatched by previous efforts. Theoretically understanding the limit of the matrix multiplication exponent remains a topic of ongoing research, as underscored by the paper's exploration of improving the additive term.

Practically, the stochastic nature of the approach presents new opportunities for scaling linear programming to previously infeasible problem sizes—those with a sheer number of variables $n$ and constraints $d$. Given that many crucial problems, like maximum flow and minimum cost flow, map onto LP frameworks, the authors' advancements could catalyze broader breakthroughs across optimization-centric domains, including logistics, finance, and network design.

Future Directions

The paper opens numerous future research avenues, particularly the question of how further advancements in matrix multiplication algorithms—resulting in a reduced $\omega$—might continue to lower the complexity of solving LPs. Additionally, improvements in the additive term related to the exponent might yield faster algorithms. Scholars interested in matrix theory and computational linear algebra will find ample opportunity to substantiate or challenge the existing limits, potentially reformulating the fundamentals of these disciplines.

In conclusion, the work of Cohen, Lee, and Song is intellectually robust and technically intricate, offering a fresh perspective on solving linear programs by leveraging the ever-advancing state of matrix multiplication. While several open problems remain, this paper constitutes a significant step forward in linking two critical areas of algorithmic research.