2000 character limit reached
A subspace method for large-scale trace ratio problems (2402.02920v2)
Published 5 Feb 2024 in math.NA and cs.NA
Abstract: A subspace method is introduced to solve large-scale trace ratio problems. This approach is matrix-free, requiring only the action of the two matrices involved in the trace ratio. At each iteration, a smaller trace ratio problem is addressed in the search subspace. Additionally, the algorithm is endowed with a restarting strategy, that ensures the monotonicity of the trace ratio value throughout the iterations. The behavior of the approximate solution is investigated from a theoretical viewpoint, extending existing results on Ritz values and vectors, as the angle between the search subspace and the exact solution approaches zero. Numerical experiments in multigroup classification show that this new subspace method tends to be more efficient than iterative approaches relying on (partial) eigenvalue decompositions at each step.
- Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ, USA, 2009.
- The Davidson method. SIAM J. Sci. Comput., 15(1):62–76, 1994.
- The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl., 20(2):303–353, 1998.
- On the trace ratio method and Fisher’s discriminant analysis for robust multigroup classification. preprint arXiv:2211.08120, 2022.
- J.H. Friedman. Regularized discriminant analysis. J. Am. Stat. Assoc., 84(405):165–175, 1989.
- A generalized Foley–Sammon transform based on generalized Fisher discriminant criterion and its application to face recognition. Pattern Recognit. Lett., 24(1-3):147–158, 2003.
- The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York, NY, USA, 2009.
- Z. Jia and F. Lai. A convergence analysis on the iterative trace ratio algorithm and its refinements. CSIAM Trans. Appl. Math., 2(2):297–312, 2021.
- Z. Jia and G. W. Stewart. An analysis of the Rayleigh–Ritz method for approximating eigenspaces. Math. Comput., 70(234):637–647, 2000.
- Applied Multivariate Statistical Analysis. Prentice Hall, Upper Saddle River, NJ, USA, 6th edition, 2007.
- A. V. Knyazev. Toward the optimal preconditioned eigensolver: Locally optimal block preconditioned conjugate gradient method. SIAM J. Sci. Comp, 23(2):517–541, 2001.
- Principal angles between subspaces in an A-based scalar product: algorithms and perturbation estimates. SIAM J. Sci. Comp., 23(6):2008–2040, 2002.
- R. Mathias. Quadratic residual bounds for the Hermitian eigenvalue problem. SIAM J. Matrix Anal. Appl., 19(2):541–550, 1998.
- R. B. Morgan. Davidson’s method and preconditioning for generalized eigenvalue problems. J. Comput. Phys., 89(1):241–245, 1990.
- R. B. Morgan. Computing interior eigenvalues of large matrices. Linear Algebra Appl., 154:289–309, 1991.
- Generalizations of Davidson’s method for computing eigenvalues of sparse symmetric matrices. SIAM J. Sci. Stat. Comput., 7(3):817–825, 1986.
- The trace ratio optimization problem. SIAM Rev., 54(3):545–569, 2012.
- Robust and sparse multigroup classification by the optimal scoring approach. Data Min. Knowl. Discov., 34(3):723–741, 2020.
- E. Romero and J. E. Roman. A parallel implementation of davidson methods for large-scale eigenvalue problems in SLEPc. ACM Trans. Math. Softw., 40(2):1–29, 2014.
- Y. Saad. Numerical Methods for Large Eigenvalue Problems. SIAM, Philadelphia, PA, 2011.
- A geometric revisit to the trace quotient problem. In MTNS 2010, pages 1–7, 2010.
- G. L. G. Sleijpen and H. A. Van der Vorst. A Jacobi–Davidson iteration method for linear eigenvalue problems. SIAM Rev., 42(2):267–293, 2000.
- The German traffic sign recognition benchmark: a multi-class classification competition. In Proc. Int. Jt. Conf. Neural Netw., pages 1453–1460. IEEE, 2011.
- Dynamic thick restarting of the Davidson, and the implicitly restarted Arnoldi methods. SIAM J. Sci. Comput., 19(1):227–245, 1998.
- G. W. Stewart. Error and perturbation bounds for subspaces associated with certain eigenvalue problems. SIAM Rev., 15(4):727–764, 1973.
- G. W. Stewart. A generalization of Saad’s theorem on Rayleigh–Ritz approximations. Linear Algebra Appl., 327(1-3):115–119, 2001.
- G. W. Stewart. Matrix Algorithms. Vol. II. SIAM, Philadelphia, PA, 2001.
- Matrix Perturbation Theory. Academic Press Inc., Boston, MA, 1990.
- G.W. Stewart. A Krylov–Schur algorithm for large eigenproblems. SIAM J. Matrix Anal. Appl., 23(3):601–614, 2002.
- Trace-ratio vs. ratio-trace for dimensionality reduction. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pages 1–8, 2007.
- K. Wu and H. Simon. Thick-restart Lanczos method for large symmetric eigenvalue problems. SIAM J. Matrix Anal. Appl., 22(2):602–616, 2000.
- Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. preprint arXiv:1708.07747, 2017.
- Maximization of the sum of the trace ratio on the stiefel manifold, ii: Computation. Sci. China Math., 58(7):1549–1566, 2015.
- Fast algorithms for the generalized Foley–Sammon discriminant analysis. SIAM J. Matrix Anal. Appl., 31(4):1584–1605, 2010.
- Y. Zhou and Y. Saad. Block Krylov–Schur method for large symmetric eigenvalue problems. Numer. Algorithms, 47:341–359, 2008.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
