Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
173 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A randomized algorithm for simultaneously diagonalizing symmetric matrices by congruence (2402.16557v3)

Published 26 Feb 2024 in math.NA and cs.NA

Abstract: A family of symmetric matrices $A_1,\ldots, A_d$ is SDC (simultaneous diagonalization by congruence, also called non-orthogonal joint diagonalization) if there is an invertible matrix $X$ such that every $XT A_k X$ is diagonal. In this work, a novel randomized SDC (RSDC) algorithm is proposed that reduces SDC to a generalized eigenvalue problem by considering two (random) linear combinations of the family. We establish exact recovery: RSDC achieves diagonalization with probability $1$ if the family is exactly SDC. Under a mild regularity assumption, robust recovery is also established: Given a family that is $\epsilon$-close to SDC then RSDC diagonalizes, with high probability, the family up to an error of norm $\mathcal{O}(\epsilon)$. Under a positive definiteness assumption, which often holds in applications, stronger results are established, including a bound on the condition number of the transformation matrix. For practical use, we suggest to combine RSDC with an optimization algorithm. The performance of the resulting method is verified for synthetic data, image separation and EEG analysis tasks. It turns out that our newly developed method outperforms existing optimization-based methods in terms of efficiency while achieving a comparable level of accuracy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Beyond Pham’s algorithm for joint diagonalization. In Proceedings of ESANN, Bruges, Belgium, April 2019.
  2. Joint diagonalization on the oblique manifold for independent component analysis. In Proceedings of ICASSP, volume V, pages 945–948, 2006.
  3. Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ, 2008.
  4. B. Afsari. Sensitivity analysis for the problem of matrix joint diagonalization. SIAM J. Matrix Anal. Appl., 30(3), 2008.
  5. A method of moments for mixture models and hidden markov models. In Proceedings of COLT, volume 23 of Proc. Mach. Learn. Res., pages 33.1–33.34, 2012.
  6. Tensor decompositions for learning latent variable models. J. Mach. Learn. Res., 15:2773–2832, 2014.
  7. A spectral algorithm for latent Dirichlet allocation. Algorithmica, 72(1):193–214, 2015.
  8. LAPACK Users’ Guide. Society for Industrial and Applied Mathematics, third edition, 1999.
  9. pyriemann/pyriemann: v0.3, July 2022. URL https://pyriemann.readthedocs.io/en/latest/.
  10. Online denoising of eye-blinks in electroencephalography. Neurophysiol Clin., 47(5):371–391, 2017.
  11. A blind source separation technique using second-order statistics. IEEE Trans. Signal Process., 45(2):434–444, 1997.
  12. N. Bosner. Efficient algorithms for joint approximate diagonalization of multiple matrices. Preprint, 2023. URL https://doi.org/10.21203/rs.3.rs-2581723/v1.
  13. Riemannian optimization and approximate joint diagonalization for blind source separation. IEEE Trans. Signal Process., 66(8):2041–2054, 2018.
  14. Approximate joint diagonalization with Riemannian optimization on the general linear group. SIAM J. Matrix Anal. Appl., 41(1):152–170, 2020.
  15. N. Boumal. An introduction to optimization on smooth manifolds. Cambridge University Press, 2023.
  16. Solving the problem of simultaneous diagonalization of complex symmetric matrices via congruence. SIAM J. Matrix Anal. Appl., 41(4):1616–1629, 2020.
  17. J.-F. Cardoso. On the performance of orthogonal source separation algorithms. In Proceedings of EUSIPCO, volume 94, pages 776–779, 1994.
  18. Joint matrices decompositions and blind source separation: A survey of methods, identification, and applications. IEEE Signal Process. Mag., 31(3):34–43, 2014.
  19. An introduction to eeg source analysis with an illustration of a study on error-related potentials. In Guide to Brain-Computer Music Interfacing, pages 163–189. Springer, 2014.
  20. A reordered Schur factorization method for zero-dimensional polynomial systems with multiple roots. In Proceedings of ISSAC, pages 133–140. ACM, 1997.
  21. L. De Lathauwer. A link between the canonical decomposition in multilinear algebra and simultaneous matrix diagonalization. SIAM J. Matrix Anal. Appl., 28(3):642–666, 2006.
  22. J. D. Dixon. Estimating extremal eigenvalues and condition numbers of matrices. SIAM J. Numer. Anal., 20(4):812–814, 1983.
  23. A randomized multivariate matrix pencil method for superresolution microscopy. Electron. Trans. Numer. Anal., 51:63–74, 2019.
  24. Canonical polyadic decomposition via the generalized Schur decomposition. IEEE Signal Process. Lett., 29:937–941, 2022a.
  25. A recursive eigenspace computation for the canonical polyadic decomposition. SIAM J. Matrix Anal. Appl., 43(1):274–300, 2022b.
  26. Matrix computations. Johns Hopkins Studies in the Mathematical Sciences. JHU Press, Baltimore, MD, fourth edition, 2013.
  27. H. He and D. Kressner. Randomized joint diagonalization of symmetric matrices. SIAM J. Matrix Anal. Appl., 45(1):661–684, 2024.
  28. R. Jiang and D. Li. Simultaneous diagonalization of matrices and its applications in quadratically constrained quadratic programming. SIAM J. Optim., 26(3):1649–1668, 2016.
  29. Automatic selection of relevant attributes for multi-sensor remote sensing analysis: A case study on sea ice classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 14:9025–9037, 2021.
  30. B. Kågström and D. Kressner. Multishift variants of the QZ algorithm with aggressive early deflation. SIAM J. Matrix Anal. Appl., 29(1):199–227, 2006.
  31. D. Kressner. Numerical methods for general and structured eigenvalue problems, volume 46 of Lecture Notes in Computational Science and Engineering. Springer-Verlag, Berlin, 2005.
  32. Simultaneous diagonalization via congruence of Hermitian matrices: some equivalent conditions and a numerical solution. SIAM J. Matrix Anal. Appl., 43(2):882–911, 2022.
  33. Robustifying independent component analysis by adjusting for group-wise stationary noise. J. Mach. Learn. Res., 20:Paper No. 147, 50, 2019.
  34. D. T. Pham. Joint approximate diagonalization of positive definite Hermitian matrices. SIAM J. Matrix Anal. Appl., 22(4):1136–1152, 2001.
  35. Blind separation of instantaneous mixtures of nonstationary sources. IEEE Trans. Signal Process., 49(9):1837–1848, 2001.
  36. T. Steel. Fast Algorithms for Generalized Eigenvalue Problems. PhD thesis, KU Leuven, 2023.
  37. Matrix perturbation theory. Computer Science and Scientific Computing. Academic Press, Inc., Boston, MA, 1990.
  38. J.-G. Sun. Stability and accuracy: Perturbation analysis of algebraic eigenproblems. Technical Report UMINF 98-07, Department of Computing Science, University of Umeå, Umeå, Sweden, 1998. Revised 2002. Available from https://people.cs.umu.se/jisun/Jiguang-Sun-UMINF98-07-rev2002-02-20.pdf.
  39. B. Sutton. Simultaneous diagonalization of nearly commuting Hermitian matrices: do-one-then-do-the-other. IMA J. Numer. Anal, page drad033, 2023.
  40. P. Tichavský and A. Yeredor. Fast approximate joint diagonalization incorporating weight matrices. IEEE Trans. Signal Process., 57(3):878–891, 2009.
  41. Solving joint diagonalization problems via a Riemannian conjugate gradient method in Stiefel manifold. Proc. Ser. Brazil. Soc. Comput. Appl. Math, 6(2), 2018.
  42. Leveraging joint-diagonalization in transform-learning NMF. IEEE Trans. Signal Process., 70:3802–3817, 2022.
  43. A fast algorithm for joint diagonalization with non-orthogonal transformations and its application to blind source separation. J. Mach. Learn. Res., 5:777–800, 2003/04.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now