Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow (1605.08285v5)

Abstract: This paper presents a new algorithm, termed \emph{truncated amplitude flow} (TAF), to recover an unknown vector $\bm{x}$ from a system of quadratic equations of the form $y_i=|\langle\bm{a}_i,\bm{x}\rangle|^2$, where $\bm{a}_i$'s are given random measurement vectors. This problem is known to be \emph{NP-hard} in general. We prove that as soon as the number of equations is on the order of the number of unknowns, TAF recovers the solution exactly (up to a global unimodular constant) with high probability and complexity growing linearly with both the number of unknowns and the number of equations. Our TAF approach adopts the \emph{amplitude-based} empirical loss function, and proceeds in two stages. In the first stage, we introduce an \emph{orthogonality-promoting} initialization that can be obtained with a few power iterations. Stage two refines the initial estimate by successive updates of scalable \emph{truncated generalized gradient iterations}, which are able to handle the rather challenging nonconvex and nonsmooth amplitude-based objective function. In particular, when vectors $\bm{x}$ and $\bm{a}_i$'s are real-valued, our gradient truncation rule provably eliminates erroneously estimated signs with high probability to markedly improve upon its untruncated version. Numerical tests using synthetic data and real images demonstrate that our initialization returns more accurate and robust estimates relative to spectral initializations. Furthermore, even under the same initialization, the proposed amplitude-based refinement outperforms existing Wirtinger flow variants, corroborating the superior performance of TAF over state-of-the-art algorithms.

• The paper introduces TAF, a two-stage algorithm that exactly recovers an unknown vector from quadratic equations with near-optimal sample complexity.
• It employs an innovative orthogonality-promoting initialization combined with truncated gradient refinement to achieve linear convergence and computational efficiency.
• Empirical results demonstrate that TAF outperforms existing methods, showing robust performance in noisy environments and practical relevance for phase retrieval applications.

Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow

The paper "Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow" introduces a computational algorithm named Truncated Amplitude Flow (TAF) to address the challenging problem of recovering an unknown vector $\bm{x}$ from a system of quadratic equations. This problem is formally represented as $y_i = |\langle \bm{a}_i, \bm{x} \rangle|^2$, with $\bm{a}_i$ being random measurement vectors, and it has been established as NP-hard.

Algorithmic Approach

The proposed TAF algorithm is structured into two main stages: an initialization stage and a refinement stage.

1. Initialization Stage: TAF introduces an innovative orthogonality-promoting initialization method. This approach contrasts with traditional spectral initialization methods by leveraging the orthogonality properties inherent in high-dimensional random vectors instead of the strong law of large numbers. The initialization aims to minimize the sum of squared normalized inner-products for a subset of vectors, which is computationally efficient even for large dimensions.
2. Refinement Stage: The refinement phase involves iterative updates achieved through scalable truncated generalized gradient iterations. Here, the primary challenge is the nonconvex and nonsmooth nature of the amplitude-based objective function. TAF employs a judiciously designed truncation rule to manage erroneously estimated signs in the gradient components. This refinement process systematically eliminates potentially misleading components, thus improving the algorithm's convergence.

Theoretical Contributions

The paper provides rigorous theoretical guarantees on the effectiveness of TAF:

• Exact Recovery: TAF is shown to exactly recover the unknown vector $\bm{x}$ with high probability when the number of measurements is on the order of the dimension of the unknown vector. This sample complexity is argued to be near-optimal.
• Linear Convergence: The refinement phase achieves linear convergence to a global minimum, ensuring computational efficiency. This translates to a convergence rate independent of the problem’s dimension, emphasizing the method's scalability.
• Numerical Stability: In noisy settings, TAF exhibits robustness, maintaining acceptable performance levels and demonstrating stability under bounded noise.

Empirical Evaluation

The algorithm's empirical performance is validated against state-of-the-art methods like Truncated Wirtinger Flow (TWF). TAF shows superior sample complexity and success rates, particularly achieving a significant performance improvement when the measurement-to-variable ratio is low. Notably, TAF successfully operates near the theoretical limit, signaling its practical utility in realistic scenarios.

Implications and Future Directions

The TAF algorithm brings forward considerable improvements in solving nonconvex quadratic systems. Its implications are especially significant for fields requiring phase retrieval, such as optics, crystallography, and imaging technologies. The paper opens avenues for further research into enhancing initialization techniques and developing gradient regularization rules that could generalize to broader nonconvex optimization problems.

Future inquiries might explore further reducing the computational overhead or adapting the TAF framework to more structured data scenarios, such as in blind deconvolution or matrix completion. Researchers could also investigate extending the application of orthogonal properties to other domains of large-scale optimization.

Overall, the TAF method represents a substantial contribution to solving systems of random quadratic equations, both in terms of theoretical innovation and practical application.