Parameterized Generalized Inverse Eigenvalue Problem

• PGIEP is a problem of identifying parameter-dependent matrix pencils to match a prescribed spectrum under structural and spectral constraints.
• The approach uses a product-manifold framework and a parameterized Stiefel multilayer perceptron (P-SMLP) to enforce hard orthogonality without soft penalties.
• Empirical results demonstrate rapid convergence and high accuracy across small- to large-scale problems, including defective and singular pencil cases.

The Parameterized Generalized Inverse Eigenvalue Problem (PGIEP) concerns the identification of matrix pencils with parameter-dependent structure such that their spectrum matches a prescribed set of eigenvalues. The recent development of hard orthogonality-constrained neural architectures and product-manifold optimization enables direct, end-to-end solutions for PGIEPs that were previously computationally intractable or required alternating between distinct constraint sets. This article provides a technical synthesis covering formal definitions, product-manifold modeling, the structure of the parameterized Stiefel multilayer perceptron (P-SMLP), algorithmic procedures, convergence theory, and empirical benchmarks.

1. Formal Definition and Product-Manifold Formulation

The PGIEP is posed as follows: Given a family of real matrices depending affinely on Euclidean parameters,

$$A(c) = A_0 + \sum_{i=1}^k c_i A_i, \quad B(c) = B_0 + \sum_{i=1}^k c_i B_i, \quad c \in \mathbb{R}^k,$$

and a desired real spectrum $\{\lambda_1, \dots, \lambda_n\}$ (with an extended treatment for possibly infinite eigenvalues $\lambda_n = \infty$), find parameters $c^*$ so that the pencil $(A(c), B(c))$ matches the assigned spectrum.

The key advance in (Zhang et al., 25 Jan 2026) is modeling all optimization variables jointly on a product manifold: $$\mathcal{M} = \mathbb{R}^k \times \mathrm{O}(n) \times \mathrm{O}(n),$$ where $Q, Z$ are orthogonal matrices ($Q^\top Q = Z^\top Z = I_n$), facilitating simultaneous optimization of Euclidean and Stiefel (orthogonality) parameters. The generalized real Schur decomposition ensures that

$Q^\top A(c) Z = S, \quad Q^\top B(c) Z = T,$

with $S, T$ upper triangular.

2. Loss Function and Structured Constraints

The objective function on the product manifold enforces both spectral and structural properties by masking: $$\begin{aligned} F(c, Q, Z) =\, &\tfrac{1}{2} \|\Lambda \odot (Q^\top B(c) Z) - I_n \odot (Q^\top A(c) Z)\|_F^2 \ &+ \tfrac{1}{2} \| P \odot (Q^\top A(c) Z) \|_F^2 \ &+ \tfrac{1}{2} \| P \odot (Q^\top B(c) Z) \|_F^2, \end{aligned}$$ where $$\Lambda = \operatorname{diag}(\lambda_1, \dots, \lambda_n)$$ and $P \in \{0,1\}^{n\times n}$ is the strict upper triangle mask ($P_{ij}=1$ iff $i>j$). The Hadamard products enforce diagonal relations (eigenvalue assignment) and force upper-triangularity of the reduced matrices; these constraints reduce PGIEP to a nonlinear least squares problem over the Stiefel–Euclidean product manifold.

For singular pencils or semi-definite constraints, masking is extended and regularization applied to ensure feasibility in degenerate cases, with analytic handling of extreme eigenvalues.

3. Parameterized Stiefel Multilayer Perceptron (P-SMLP) Architecture

The P-SMLP implements the product-manifold model with end-to-end differentiable layers:

• Input Seed: A fixed orthogonal seed $X \in \mathbb{R}^{n \times n}$.
• Hidden Layers: For $l=1,\dots,L-1$, $H_l = \Phi(W_{l-1} H_{l-1} + b_{l-1})$, with $\Phi$ (e.g., ReLU) activations.
• Output Layer: Produces $$f(X) = W_L H_L + b_L \in \mathbb{R}^{n \times 2n + k}$$, split into two $n \times n$ blocks (for $Q$, $Z$) and a vector $c$.
• Stiefel (Orthogonality) Constraints: For each block, apply a Stiefel operator $\Psi: \mathbb{R}^{n \times n} \to \mathrm{O}(n)$, realized by either:
  • SVD projection: $M = U \Sigma V^\top \implies Q = U V^\top$,
  • QR projection: $M = Q R$ provides $Q$.
  • For distinct $Q, Z$, use paired projections or column permutations.

These hard-constraints (rather than soft penalties) guarantee that $Q, Z$ remain orthogonal at all times. The Euclidean parameter $c$ is unconstrained.

4. End-to-End Algorithmic Procedure

The P-SMLP optimization proceeds as follows (see Section 3 of (Zhang et al., 25 Jan 2026)):

1. Forward Pass: From the seed $X$, propagate through the network, extract $Q, Z$ by projection, and form $c$.
2. Loss Evaluation: Compute $F(c, Q, Z)$ as above.
3. Backpropagation: Use automatic differentiation for unconstrained parameters; for SVD/QR, frameworks with differentiable matrix decompositions (e.g., PyTorch, TensorFlow) support backprop through orthogonal projections.
4. Parameter Update: Apply Adam or related optimizers to all weights $(W_l, b_l)$ and $c$.
5. Repeating: Iteratively train until $F$ falls below tolerance.

No alternating or block-wise optimization is required; the procedure updates all variables jointly, a capability unique to the product-manifold approach.

5. Theoretical Results and Convergence Properties

The product-manifold construction guarantees that the gradient of $F$ is globally Lipschitz on any domain where parameters and orthogonal factors are bounded: $$\|\nabla F(c,Q,Z)\;-\;\nabla F(\hat c, \hat Q, \hat Z)\|_F \leq L \left( \|c-\hat{c}\|_2 + \|Q-\hat{Q}\|_F + \|Z-\hat{Z}\|_F \right).$$ This property holds for all bounded $A_i$, $B_i$, $\Lambda$ (operator norm) and compact parameter sets, enabling convergence proofs via standard stochastic gradient theory (see references in (Zhang et al., 25 Jan 2026)). While global optimality cannot be ensured due to nonconvexity, experiments consistently reach loss values at machine precision for small $n$ and $10^{-6}$–$10^{-4}$ for $n$ up to 40.

6. Empirical Evaluation and Computational Aspects

Empirical results validate the robustness and competitiveness of P-SMLP for various PGIEPs:

• Small-scale PGIEP (n = 2, 5): All tested Stiefel strategies achieve eigenvalue errors near machine precision or $10^{-4}$ over $2 \times 10^5$ epochs.
• Large-scale PGIEP (n = 10, 20, 40): The approach scales capably to $n=40$, attaining errors $10^{-4}$–$10^{-6}$.
• Defective/singular pencils: Properly handles pencils with $B(c)$ singular and $\lambda_n=\infty$, delivering correct rank and eigenvalues.
• Efficiency: Each epoch requires one SVD or QR on dense $n \times n$ blocks (cost $O(n^3)$). Batch training on modern GPUs makes even $n=40$ practical.

In comparison to traditional methods (Newton’s method, Cayley transforms, alternating projections), P-SMLP removes the need for repeated eigenproblem computations—computationally dominant for large instances.

7. Extensions, Related Developments, and Applications

The P-SMLP framework generalizes seamlessly to broader inverse and structured eigenvalue problems, including orthogonal dictionary learning, PCA regression on manifolds, parameter estimation for control systems with rotation constraints, and Grassmann-valued or pose-constrained optimization.

The key distinction compared to prior SMLP approaches (Zhang et al., 2024) is that parameters on both Stiefel and Euclidean components are learned synchronously via hard manifold constraints, and orthogonality is never enforced by a penalty, resulting in numerically stable and robust solutions. This paradigm may be applied to neural architectures requiring guaranteed spectral or geometric properties (e.g., orthogonal autoencoders, rotation-equivariant policies, spectral regularization in deep learning).

---

In summary, the parameterized generalized inverse eigenvalue problem is now directly addressable by hard constrained neural architectures operating on the product manifold of orthogonal and Euclidean spaces. This development enables robust, efficient, end-to-end learning of inverse spectra problems with guaranteed structure, rapid convergence, and broad applicability (Zhang et al., 25 Jan 2026).