Approximating Sparse Matrices and their Functions using Matrix-vector products (2310.05625v2)
Abstract: The computation of a matrix function $f(A)$ is an important task in scientific computing appearing in machine learning, network analysis and the solution of partial differential equations. In this work, we use only matrix-vector products $x\mapsto Ax$ to approximate functions of sparse matrices and matrices with similar structures such as sparse matrices $A$ themselves or matrices that have a similar decay property as matrix functions. We show that when $A$ is a sparse matrix with an unknown sparsity pattern, techniques from compressed sensing can be used under natural assumptions. Moreover, if $A$ is a banded matrix then certain deterministic matrix-vector products can efficiently recover the large entries of $f(A)$. We describe an algorithm for each of the two cases and give error analysis based on the decay bound for the entries of $f(A)$. We finish with numerical experiments showing the accuracy of our algorithms.
- Near-Optimal Perfectly Matched Layers for Indefinite Helmholtz Problems. SIAM Rev. 2016;58(1):90–116.
- Garrappa R, Popolizio M. On the use of matrix functions for fractional partial differential equations. Math Comput Simul. 2011;81(5):1045–1056.
- Grimm V, Hochbruck M. Rational approximation to trigonometric operators. BIT. 2008 jun;48(2):215–229.
- Benzi M, Boito P. Matrix functions in network analysis. GAMM-Mitt. 2020;43(3):e202000012.
- Estrada E, Higham DJ. Network Properties Revealed through Matrix Functions. SIAM Rev. 2010;52(4):696–714.
- Goedecker S. Linear scaling electronic structure methods. Rev Mod Phys. 1999 Jul;71:1085–1123.
- Efficient Computation of Sparse Matrix Functions for Large-Scale Electronic Structure Calculations: The CheSS Library. J Chem Theory Comput. 2017;13(10):4684–4698.
- Cortinovis A, Kressner D. On randomized trace estimates for indefinite matrices with an application to determinants. Found Comput Math. 2021;22(3):875–903.
- Preconditioning for Scalable Gaussian Process Hyperparameter Optimization. In: International Conference on Machine Learning (ICML); 2022. .
- Newman MEJ. Finding community structure in networks using the eigenvectors of matrices. Phys Rev E. 2006 Sep;74:036104.
- Higham NJ. Functions of Matrices. SIAM; 2008.
- Golub GH, Van Loan CF. Matrix Computations. 4th ed. Johns Hopkins University Press; 2013.
- Güttel S. Rational Krylov approximation of matrix functions: Numerical methods and optimal pole selection. GAMM-Mitt. 2013;36(1):8–31.
- Limited-memory polynomial methods for large-scale matrix functions. GAMM-Mitt. 2020;43(3):e202000019.
- Benzi M, Razouk N. Decay bounds and O(n) algorithms for approximating functions of sparse matrices. ETNA. 2007;28:16–39. Available from: http://eudml.org/doc/130625.
- Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inf Theory. 2006;52(2):489–509.
- Candès EJ, Tao T. Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? IEEE Trans Inf Theory. 2006;52(12):5406–5425.
- Donoho DL. Compressed sensing. IEEE Trans Inf Theory. 2006;52(4):1289–1306.
- Decay rates for inverses of band matrices. Math Comp. 1984;43(168):491–499.
- Benzi M, Golub GH. Bounds for the Entries of Matrix Functions with Applications to Preconditioning. BIT. 1999 sep;39(3):417–438.
- Analysis of Probing Techniques for Sparse Approximation and Trace Estimation of Decaying Matrix Functions. SIAM J Matrix Anal Appl. 2021;42(3):1290–1318.
- Coleman TF, Moré JJ. Estimation of Sparse Jacobian Matrices and Graph Coloring Blems. SIAM J Numer Anal. 1983;20(1):187–209.
- On the Estimation of Sparse Jacobian Matrices. IMA J Appl Math. 1974 02;13(1):117–119. Available from: https://doi.org/10.1093/imamat/13.1.117.
- Saad Y. Analysis of Some Krylov Subspace Approximations to the Matrix Exponential Operator. SIAM J Numer Anal. 1992;29(1):209–228.
- Computing Aαsuperscript𝐴𝛼A^{\alpha}italic_A start_POSTSUPERSCRIPT italic_α end_POSTSUPERSCRIPT, log(A)𝐴\log(A)roman_log ( italic_A ), and Related Matrix Functions by Contour Integrals. SIAM J Numer Anal. 2008;46(5):2505–2523.
- Block krylov subspace methods for functions of matrices. ETNA. 2018;27:100–126.
- Divide-and-Conquer Methods for Functions of Matrices with Banded or Hierarchical Low-Rank Structure. SIAM J Matrix Anal Appl. 2022;43(1):151–177.
- Halikias D, Townsend A. Matrix recovery from matrix-vector products. arXiv preprint arXiv:221209841. 2022;.
- Levitt J, Martinsson PG. Linear-Complexity Black-Box Randomized Compression of Hierarchically Block Separable Matrices. arXiv preprint arXiv:220502990. 2022;.
- Querying a Matrix through Matrix-Vector Products. ACM Trans Algorithms. 2021 oct;17(4).
- An estimator for the diagonal of a matrix. Applied Numerical Mathematics. 2007;57(11):1214–1229. Numerical Algorithms, Parallelism and Applications (2).
- Herman MA, Strohmer T. High-Resolution Radar via Compressed Sensing. IEEE Trans Signal Process. 2009;57(6):2275–2284.
- Identification of Matrices Having a Sparse Representation. IEEE Trans Signal Process. 2008;56(11):5376–5388.
- Sketching Sparse Matrices, Covariances, and Graphs via Tensor Products. IEEE Trans Inf Theory. 2015;61(3):1373–1388.
- Candès EJ, Tao T. Decoding by linear programming. IEEE Trans Inf Theory. 2005;51(12):4203–4215.
- Greedy algorithms for compressed sensing. In: Eldar YC, Kutyniok G, editors. Compressed Sensing: Theory and Applications. Cambridge University Press; 2012. p. 348–393.
- Blanchard JD, Tanner J. Performance comparisons of greedy algorithms in compressed sensing. Numer Linear Algebra Appl. 2015;22(2):254–282.
- Blumensath T, Davies ME. Normalized Iterative Hard Thresholding; Guaranteed Stability and Performance. IEEE J Sel Topics Signal Process. 2010;4(2):298–309.
- Foucart S. Hard Thresholding Pursuit: an Algorithm for Compressive Sensing. SIAM J Numer Anal. 2011;49(6):2543–2563.
- Needell D, Tropp JA. CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmon Anal. 2009;26(3):301–321.
- Foucart S, Rauhut H. A Mathematical Introduction to Compressive Sensing. Applied and Numerical Harmonic Analysis. Springer New York; 2013. Available from: https://books.google.co.uk/books?id=zb28BAAAQBAJ.
- Rokhlin V, Tygert M. A Fast Randomized Algorithm for Overdetermined Linear Least-Squares Regression. Proc Natl Acad Sci USA. 2008;105(36):13212–13217.
- Adaptive algorithm for sparse signal recovery. Digital Signal Processing. 2019;87:10–18.
- On the Power of Adaptivity in Sparse Recovery. In: 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science; 2011. p. 285–294.
- Krahmer F, Ward R. Stable and Robust Sampling Strategies for Compressive Imaging. IEEE Trans Image Process. 2014;23(2):612–622.
- Improved Algorithms for Adaptive Compressed Sensing. In: Chatzigiannakis I, Kaklamanis C, Marx D, Sannella D, editors. 45th International Colloquium on Automata, Languages, and Programming (ICALP 2018). vol. 107 of Leibniz International Proceedings in Informatics (LIPIcs). Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik; 2018. p. 90:1–90:14.
- Benzi M, Simoncini V. Decay Bounds for Functions of Hermitian Matrices with Banded or Kronecker Structure. SIAM J Matrix Anal Appl. 2015;36(3):1263–1282.
- Pozza S, Simoncini V. Inexact arnoldi residual estimates and decay properties for functions of non-Hermitian matrices. BIT. 2019;59(4):969–986.
- LeVeque RJ. Finite Difference Methods for Ordinary and Partial Differential Equations. Steady State and Time Dependent Problems. SIAM; 2007.
- Loan CFV. The ubiquitous Kronecker product. J Comput Appl Math. 2000;123(1):85–100. Numerical Analysis 2000. Vol. III: Linear Algebra.
- Davis TA, Hu Y. The University of Florida Sparse Matrix Collection. ACM Trans Math Softw. 2011 dec;38(1). Available from: https://doi.org/10.1145/2049662.2049663.
- Ranking hubs and authorities using matrix functions. Linear Algebra Appl. 2013;438(5):2447–2474.
- Estrada E, Hatano N. Communicability in complex networks. Phys Rev E. 2008 Mar;77:036111.
- Decay Properties of Spectral Projectors with Applications to Electronic Structure. SIAM Rev. 2013;55(1):3–64.
- Analysis of stochastic probing methods for estimating the trace of functions of sparse symmetric matrices. arXiv preprint arXiv:200911392. 2023;.