Generalized Matrix Eigenvalue Problem

• The generalized matrix eigenvalue problem is defined by finding nontrivial pairs (λ, x) such that Ax=λBx, extending classical eigenproblems with nontrivial inner products.
• Iterative, optimization-based, and transform-domain methods are developed to efficiently and robustly solve GEPs in high-dimensional and ill-conditioned settings.
• GEP applications span quantum physics, machine learning, and structural engineering, where tailored algorithms enable precise spectral analysis and practical computational insights.

The generalized matrix eigenvalue problem (GEP) is a fundamental construct in applied mathematics, physics, statistics, and engineering, formalized as the computation of nontrivial pairs (λ, x) such that $A x = \lambda B x$, for given matrices $A$ and $B$. In diverse applications—ranging from quantum many-body physics to machine learning, control theory, numerical discretizations of PDEs, and lattice gauge theory—this generalized formulation extends the classic eigenproblem ($B=I$) to a richer structure, accommodating nontrivial inner products, operator discretizations, and model constraints. This article provides a comprehensive account of GEP, covering its formulation, solution principles, perturbation theory, algorithmic approaches, controllable approximations, and selected domain-specific innovations.

1. Mathematical Formulation and Theoretical Foundations

The GEP seeks scalars $\lambda$ and nonzero vectors $x$ such that $A x = \lambda B x$. Here, $A$ and $B$ are typically square matrices, with $B$ often assumed symmetric positive definite or Hermitian and invertible, though extensions relax these conditions. The spectral properties of GEPs are deeply influenced by algebraic and topological features—such as the definiteness of $B$, the symmetry of $A$, and the structure of the operator pencil $(A,B)$.

A canonical variational representation is the maximization (or minimization) of the Rayleigh quotient subject to a $B$-weighted constraint: $\max_{x\ne 0} \frac{x^\top A x}{x^\top B x},\quad \text{subject to %%%%14%%%%.}$ The stationary condition of this quadratic constrained problem yields the GEP via Lagrangian duality (Ghojogh et al., 2019). In the matrix case, simultaneous optimization over a basis $X$ and diagonal $\Lambda$ gives $A X = B X \Lambda$.

Analytically, every GEP with $B$ invertible can be transformed into the standard eigenproblem $B^{-1} A x = \lambda x$, but such reduction may be ill-conditioned or numerically unstable when $B$ is nearly singular or highly ill-conditioned (Ghojogh et al., 2019).

2. Solution Methods and Algorithmic Developments

Solution strategies for GEPs depend on the properties of $A$ and $B$, problem size, and desired spectral features.

Direct and Transform-based Approaches

• Eigen-decomposition/Whitening: For $B$ symmetric positive definite, compute its eigendecomposition $B = Q \Lambda Q^\top$, apply the change of variable $y = \Lambda^{-1/2} Q^\top x$, and reduce the problem to standard form $A' y = \lambda y$ with $A' = \Lambda^{-1/2} Q^\top A Q \Lambda^{-1/2}$ (Ghojogh et al., 2019).
• Shift-and-Invert Spectral Transformation: For symmetric semidefinite $B$, the shifted formulation $(A - \sigma B) x = 0$ and corresponding factorization enable backward-stable algorithms for dense and sparse cases, producing computed eigenpairs that are, to first order, exact eigenpairs of perturbed $(A+E,B+F)$ with normwise perturbations $O(u)$ (Stewart, 5 Nov 2024).

Iterative and Optimization-based Algorithms

• Generalized Power and Subspace Methods: Power iteration, subspace iteration, and Chebyshev-Davidson variants solve $A x = \lambda B x$ via repeated application of $B^{-1}A$ or filtered polynomial accelerations. The Chebyshev–RQI subspace algorithm combines Chebyshev filtering and inexact Rayleigh quotient iteration, augmenting efficiency and robustness for large-scale Hermitian GEPs (Wang et al., 2022).
• Transform-domain Mirror Descent: Optimization-based formulations minimize structured objectives, such as $f(x)=x^\top B x - \sqrt{x^\top A x}$, rather than maximizing the Rayleigh quotient. Accelerated preconditioned mirror descent (PMD) in a transformed variable $y=Px$ yields significantly faster convergence than fixed-step gradient descent, and with suitable preconditioner choices, recovers or improves on the classical GEP power method (Liu et al., 3 Jul 2025).
• Split-Merge and Surrogate-based Schemes: The Split–Merge algorithm extends to GEPs, employing second-order surrogates of the objective based on richer (Hessian) information and adaptive quadratic approximations, yielding convergence improvements over both power and Lanczos methods for generalized problems with clustered spectra (Liu et al., 3 Jul 2025).

Sparse and High-dimensional GEPs

• Sparse GEP Optimization: For high-dimensional statistics, sparse principal component analysis (PCA), and discriminant analysis, sparsity is imposed via $\ell_0$-norm penalties on the eigenvector. Iteratively reweighted quadratic minorization (IRQM) and block-decomposition strategies reduce the combinatorial sparse GEP to a sequence of regular GEPs or small quadratic fractional subproblems, with preconditioned ascent and combinatorial search for active sets (Song et al., 2014, Yuan et al., 2018).
• Distributed and Verification Frameworks: Verified computation methods (complex moments with Rayleigh–Ritz reduction) rigorously identify eigenvalues and eigenvectors of Hermitian GEPs within prescribed regions and error bounds, with practical advantages in efficiency and error enclosure, including scenarios with clustered or nearly multiple eigenvalues (Imakura et al., 2021).

3. Perturbation Theory, Error Analysis, and Inclusion Bounds

The sensitivity of GEP eigenvalues to perturbations in $A$ and $B$ is governed by analytical and geometric considerations.

• Quadratic Perturbation Bounds: When $A$ is Hermitian and $B$ is positive definite, and a spectral gap exists, the error in an eigenvalue under perturbation $E$ scales quadratically as $O(\|E\|^2/\delta^2)$, with $\delta$ the spectral gap, improving over classical linear or Bauer–Fike-type first-order estimates (Nakatsukasa, 2010, Nakatsukasa, 2010).
• Gerschgorin-type Disks and Forward Error: Generalizations of the Gerschgorin theorem to matrix pencils $A-\lambda B$ define inclusion disks in the complex plane and yield practical forward error bounds for diagonalizable pencils. Disjointness of these disks ensures unique localization of simple eigenvalues, with error proportional to off-diagonal computed residuals; diagonal scaling sharpens bounds further to quadratic order in the local separation $\delta$ (Nakatsukasa, 2010).

4. Applications in Physics, Engineering, and Data Science

GEPs permeate countless applied contexts, often as the decisive step in extracting physically or statistically relevant information.

• Lattice Gauge Theory and Spectroscopy: The GEVP is central to extracting hadron energies from Euclidean time correlation matrices in lattice QCD. Given a basis of interpolating operators $O_1, \ldots, O_N$, one solves

$$C(t)\, v_n(t,t_0) = \lambda_n(t,t_0) C(t_0)\, v_n(t,t_0)$$

where $C_{ij}(t) = \langle O_i(t) O_j^\dagger(0) \rangle$. The effective energies are computed via $$E_n^{\rm eff}(t,t_0) = -\partial_t \ln\lambda_n(t, t_0)$$. The method permits systematic separation of ground and excited states even when operator overlap is nontrivial (0808.1017).

• Machine Learning and Signal Processing: Generalized eigenvalue decompositions underpin Fisher Discriminant Analysis, Canonical Correlation Analysis, kernel methods, and supervised PCA. The GEP arises from variational characterizations where quadratic forms are optimized subject to $B$-weighted normalization or orthogonality constraints (Ghojogh et al., 2019).
• Quantum Chemistry and Quantum Computing: Subspace expansion approaches for excited state algorithms (QSE, qEOM) on quantum computers require solving GEPs with overlap matrices constructed from measured expectation values. High condition numbers of overlap matrices render such problems extremely sensitive to sampling noise; methods with orthonormal bases (e.g., q-sc-EOM) alleviate this instability by working with identity overlap (Kwao et al., 12 Mar 2025).
• Structural Engineering and Spectral Optimization: Minimization of the maximum GEP eigenvalue subject to structural constraints optimizes eigenfrequencies (vibration characteristics) of truss structures. The log–sum–exp smoothing technique and explicit Clarke subdifferential calculations yield efficient projected gradient algorithms with proven $O(1/\sqrt{k})$ convergence to global minima. The maximum eigenvalue is shown to be pseudoconvex under affine $A(x)$, $B(x)$, and $B(x)\succ 0$ conditions (Nishioka et al., 2023).
• Sparsity-constrained and Parametric Models: The Ritz method for parametric GEPs with affine dependence on parameters combines an eigenvector splitting approach (projecting onto an average-matrix eigenspace and a correction operator) with sparse polynomial interpolation, achieving uniform accuracy across parameter sets and robustly handling eigenvalue crossings and multiplicities (Bisch et al., 11 Mar 2025).

5. Noise, Stability, and Regularization

Statistical and computational noise, especially in quantum or sampling-based settings, can destabilize GEP solutions through ill-conditioning of $B$ or the overlap matrix.

• Noise Amplification and Bayesian Trimmed Sampling: Stochastic or noisy matrix entries in $A$ or $B$ (as in quantum Monte Carlo or generator coordinate methods) lead to GEPs with unstable inversion of $B$. The trimmed sampling algorithm leverages Bayesian inference: samples of $(H,N)$ are drawn from distributions informed by estimated uncertainties, with physics-based constraints (e.g., $N$ positive definite, smooth eigenvalue convergence) imposed in the likelihood. The resulting posterior quantifies observable uncertainties robustly, outperforming conventional regularization by ridge regression with ad hoc parameters (Hicks et al., 2022).
• Thresholding and Loss of Spectral Completeness: In quantum subspace methods, thresholding small singular values of the overlap matrix $S$ during GEP solution avoids catastrophic noise amplification but at the cost of losing completeness in the excited state spectrum—specifically, entire states may be omitted when $S$ becomes nearly singular (Kwao et al., 12 Mar 2025).

6. Special Structure and Fast Algorithms

Exploiting structure in $(A,B)$ enables algorithmic acceleration.

• Divide-and-Conquer with Spectral Shattering: For definite Hermitian pencils $A,B$, structured divide-and-conquer algorithms using structured perturbations (e.g., GUE or diagonal random) achieve pseudospectral “shattering”—splitting the real spectrum into well-separated intervals. The inverse-free, highly parallel eigensolver then recursively splits the spectrum via sign function extraction and randomized rank-revealing factorizations, preserving definiteness and reducing computational complexity relative to unstructured approaches (Demmel et al., 28 May 2025).
• Analytical Solution for Structured Matrices: For discretizations leading to Toeplitz-plus-Hankel or block-diagonal matrices (as in FEM or IGA), explicit formulas for eigenvalues and eigenvectors can be given in terms of trigonometric functions, and classical identities (like the eigenvector–eigenvalue product rule) extend to the GEP setting (Deng, 2020).

7. Open Problems and Research Directions

Challenges in GEPs include:

• Efficient algorithms for large-scale, indefinite, or highly non-normal pencils.
• Rigorous error control under severe noise or sampling constraints, especially for excited state quantum algorithms.
• Global convergence guarantees for nonlinear, nonsmooth, or pseudoconvex GEP minimization.
• Uniform subspace constructions that handle eigenvalue crossings and multiplicities in parametric/stochastic settings.
• Scalability and parallelism in modern hardware architectures, leveraging structure- and spectrum-aware algorithm design.

Ongoing developments in the analysis of smoothing methods, mirror descent, and surrogate-accelerated optimization, as well as the identification of universal split spaces for parameterized GEPs, continue to broaden the theoretical and algorithmic foundation of the field. The diversity of techniques reflected across domains—from variational quantum eigensolvers and Bayesian uncertainty quantification to structure-preserving eigensolvers and sparse optimization—ensures that GEP remains central to scientific computing's evolving landscape.