Relative-Error CUR Matrix Decompositions (0708.3696v1)

Abstract: Many data analysis applications deal with large matrices and involve approximating the matrix using a small number of components.'' Typically, these components are linear combinations of the rows and columns of the matrix, and are thus difficult to interpret in terms of the original features of the input data. In this paper, we propose and study matrix approximations that are explicitly expressed in terms of a small number of columns and/or rows of the data matrix, and thereby more amenable to interpretation in terms of the original data. Our main algorithmic results are two randomized algorithms which take as input an $m \times n$ matrix $A$ and a rank parameter $k$. In our first algorithm, $C$ is chosen, and we let $A'=CC^+A$, where $C^+$ is the Moore-Penrose generalized inverse of $C$. In our second algorithm $C$, $U$, $R$ are chosen, and we let $A'=CUR$. ($C$ and $R$ are matrices that consist of actual columns and rows, respectively, of $A$, and $U$ is a generalized inverse of their intersection.) For each algorithm, we show that with probability at least $1-\delta$: $$ ||A-A'||_F \leq (1+\epsilon) ||A-A_k||_F, $$ where $A_k$ is thebest'' rank-$k$ approximation provided by truncating the singular value decomposition (SVD) of $A$. The number of columns of $C$ and rows of $R$ is a low-degree polynomial in $k$, $1/\epsilon$, and $\log(1/\delta)$. Our two algorithms are the first polynomial time algorithms for such low-rank matrix approximations that come with relative-error guarantees; previously, in some cases, it was not even known whether such matrix decompositions exist. Both of our algorithms are simple, they take time of the order needed to approximately compute the top $k$ singular vectors of $A$, and they use a novel, intuitive sampling method called ``subspace sampling.''

• The paper introduces randomized CUR and CX matrix decompositions with relative-error guarantees for low-rank matrix approximation.
• It employs innovative subspace sampling based on singular vectors to select informative rows and columns from the input matrix.
• The approach yields interpretable approximations with strong numerical guarantees, enabling efficient applications in fields like genomics and recommendation systems.

Overview of "Relative-Error CUR Matrix Decompositions"

The paper "Relative-Error CUR Matrix Decompositions," authored by Petros Drineas, Michael W. Mahoney, and S. Muthukrishnan, presents algorithms for computing low-rank matrix approximations that are articulated in terms of actual rows and columns of the input matrix. Unlike traditional methods that utilize combinations of rows and columns, the CUR matrix decompositions proposed here are argued to be more interpretable through the explicit use of original data matrix features.

Summary of Contributions

The paper introduces two main randomized algorithms that provide relative-error guarantees for low-rank matrix approximations. These approximations involve:

1. CX Matrix Decomposition: This approach samples columns from the matrix to provide a basis for approximation. The algorithm relies on a novel sampling method known as "subspace sampling," where columns are selected based on the Euclidean norms of the rows of the matrix's top singular vectors.
2. CUR Matrix Decomposition: Beyond selecting columns (C), this method also samples rows (R) and computes an intermediate matrix (U) to ensure the decomposition of the form $A \approx CUR$. This decomposition method aims to maintain relative-error guarantees like the CX decompositions, offering a more interpretable matrix representation.

Both algorithms are significant for being the first to achieve relative-error bounds using polynomial time solutions, marking a departure from prior methods restricted to additive errors.

Numerical Results and Claims

The algorithms provide strong numerical guarantees. For matrix $A$ and a rank parameter $k$, the algorithms ensure that:

• $\|A - CUR\|_F \leq (1 + \epsilon) \|A - A_k\|_F$,

where $A_k$ is the best rank-$k$ approximation based on the singular value decomposition (SVD) of $A$. The sampling complexity is governed by polylogarithmic dependencies on $k$, $1/\epsilon$, and $\log(1/\delta)$, ensuring efficiency and scalability.

Technical Highlights

The key advancement facilitating these results is the "subspace sampling" technique. By utilizing properties of the matrix's singular vectors, the algorithms target columns and rows that effectively capture the significant components of the matrix. This method enhances prior work, where sampling occurred with less precise formulations, and demonstrates superiority both in the theoretical bounds and practical sampling efficiency.

Implications and Future Directions

Practically, these decomposition techniques have applications in diverse computational areas like genomics, recommendation systems, and natural language processing due to their ability to interpret matrices in terms of original elements, thereby enhancing interpretability and facilitating tasks such as classification and reconstruction.

Theoretically, the paper opens directions for exploring deterministic algorithms for similar tasks, extensions to other norms, and further optimization of sampling complexities. Additionally, those in the fields of numerical linear algebra and theoretical computer science might explore extensions that incorporate constraints like sparsity or non-negativity, commonly required in applications involving large and complex datasets in industry and academia.

Summary

The paper's introduction of relative-error CUR matrix decompositions represents a substantial step forward in low-rank approximation methods, with the potential to impact various application domains and inspire future research within computational linear algebra and adjacent fields.