Linear Systems and Eigenvalue Problems: Open Questions from a Simons Workshop

Abstract: This document presents a series of open questions arising in matrix computations, i.e., the numerical solution of linear algebra problems. It is a result of working groups at the workshop \emph{Linear Systems and Eigenvalue Problems}, which was organized at the Simons Institute for the Theory of Computing program on \emph{Complexity and Linear Algebra} in Fall 2025. The complexity and numerical solution of linear algebra problems %in matrix computations and related fields is a crosscutting area between theoretical computer science and numerical analysis. The value of the particular problem formulations here is that they were produced via discussions between researchers from both groups. The open questions are organized in five categories: iterative solvers for linear systems, eigenvalue computation, low-rank approximation, randomized sketching, and other areas including tensors, quantum systems, and matrix functions.

• The paper identifies key challenges in iterative solvers and eigenvalue methods, emphasizing the need for refined condition number and convergence analyses.
• It rigorously examines multigrid convergence, preconditioning effects, and finite precision issues, bridging theoretical insights with practical computational challenges.
• The workshop synopsis calls for novel approaches in randomized algorithms and low-rank approximations, advancing both numerical analysis and computational complexity.

Open Problems in Linear Systems and Eigenvalue Computation: A Simons Workshop Synopsis

Introduction

This document, "Linear Systems and Eigenvalue Problems: Open Questions from a Simons Workshop" (2602.05394), synthesizes a collection of precise, technically formulated open problems in numerical linear algebra and theoretical computer science. The covered domains—iterative solvers, eigenvalue computation, low-rank approximations, sketching, and emerging directions such as tensor methods and quantum systems—highlight both algorithmic frontiers and complex interactions between computation, precision, and structure. The problems reflect the need for closer synergy between numerical analysis and computational complexity communities, targeting both empirically relevant and theoretically tractable challenges.

Iterative Solvers for Linear Systems

Benchmark Design and Robust Scalability

A central theme is the lack of comprehensive benchmark families for linear system solvers that bridge theoretical and numerical communities. There is a strong call for software and model problems parameterized by physical and discretization properties, especially for PDE-induced systems such as highly anisotropic diffusion, advection-dominated phenomena, and indefinite Helmholtz problems. The challenges involve designing fast algorithms with nearly linear complexity, amenable to both theoretical guarantees and robust empirical performance.

Multigrid Methods and Algebraic Analysis

The document identifies the formal proof of optimal multigrid convergence for wide classes of polynomially bounded symmetric weakly-chained diagonally dominant M-matrices (SWCDDM) as a fundamental unresolved problem. Existing convergence proofs are primarily rooted in geometric settings and strong SDDM structure. Extending algebraic multigrid analysis, especially to broadband and non-Laplacian matrix classes, remains open, with potential impact on both the TCS and NLA communities.

Complexity and Spectral Structure

Critical attention is given to spectral distribution and its nuanced effect on iterative solver complexity. Notably, the document highlights that condition number alone can provide a poor complexity estimate in the presence of clustered or outlier singular values/eigenvalues. There are sharp distinctions between complexity in the direct access model (where entire matrix entries are available) versus the matrix-vector-product model, particularly for systems with outlying or clustered spectrum [derezinski2025approaching].

Key open problems include establishing the precise arithmetic and bit complexity for classes of $(n, k, l, \kappa)$ matrices (with $k$ large, $l$ outlying singular values), and clarifying whether averaged condition numbers or alternate spectral summaries yield tighter bounds for convergence, including in finite precision.

Preconditioning and Non-Normality

The document challenges the widespread heuristic that preconditioning should focus solely on reducing condition number. Instead, it emphasizes the nuanced, sometimes non-monotonic relationship between spectrum, eigenvector conditioning, and iterative convergence—especially for nonsymmetric and highly nonnormal systems, where field-of-values or pseudospectral characteristics may be more indicative. There is a demand for more descriptive, perhaps problem-specific, convergence criteria for methods such as GMRES, with rigorous ties to eigenstructure and preconditioner effects.

Finite Precision and Residual Behavior

Several problems probe the finite-precision behavior of CG and Lanczos-type methods, including precise residual norm analysis, backward/forward stability, necessary bit precision, and distinctions among various implementations (plain versus block, and CG versus Lanczos). New stable or interpretable analyses—possibly using more abstract matrix perturbation models—are sought, as are characterizations for block methods in finite arithmetic.

The Forsythe Conjecture

An explicit challenge is posed by the classical Forsythe conjecture on the asymptotic cycling of restarted gradient-type methods. Despite numerical evidence and special-case results, the general conjecture remains unresolved, especially for restart lengths $s > 2$.

Eigenvalue Computation

Randomization, Conditioning, and Derandomization

A major theoretical bottleneck concerns the need for random perturbations (pseudospectral shattering) in achieving polynomial condition numbers for Schur and diagonalization algorithms [Banks-et-al-23]. While randomized algorithms can ensure $\kappa_{eig}(A+E)$ is polynomially bounded in $n/\delta$ with high probability, the existence of polynomial-time deterministic algorithms achieving similar guarantees is unknown. Derandomizing these regularization methods—via either explicit constructions or deterministic "shattering" matrices—constitutes a prominent open direction.

For Hermitian tridiagonal matrices, there is a related problem of constructing deterministic diagonal perturbations achieving spectral gaps analogous to the Minami bound.

Bit Complexity and Inversion-Free Diagonalization

Known randomized eigenvalue algorithms require bit precision growing as $O(\log^4(n/\delta)\log n)$ for backward stability. Improving this—ideally to the $O(\log(n/\delta))$ bits attained in optimized Hermitian algorithms [shah2025hermitian]—remains an explicit open problem, with implications for both numerical practice and algorithmic optimality.

Ritz Values, Invariant Subspaces, and Arnoldi Process Analysis

Several open problems focus on the fidelity and localization of Ritz values—in both deterministic and probabilistic settings—when approximate or random subspaces are used. There is insufficient understanding of the distribution of Ritz values within the numerical range for non-Hermitian problems and the precise bounds on eigenvector conditioning in compressed scenarios.

Tridiagonal Eigenvalue Algorithms (MRRR)

Despite widespread use of the Multiple Relatively Robust Representations (MRRR) algorithm for tridiagonal eigenproblems, a complete TCS-style floating-point analysis guaranteeing correctness and convergence for all inputs is lacking. Extending or adapting MRRR to bidiagonal SVD problems—while retaining orthogonality, accuracy, and operation count—remains an ongoing challenge, especially in the presence of tight spectral clusters.

Spectral Radius and Rightmost Eigenvalues

The existence of gap-free algorithms for computing leading eigenvalues (magnitude and rightmost real part) of general nonnormal matrices, with polylogarithmic iteration complexity in $n$ and $\kappa_V$, is not yet demonstrated. Formalizing and improving the practical guarantees of power/inverse iteration methods, especially for defective or nearly defective matrices, is a highlighted gap.

Low-Rank Approximation and Column Subset Selection

Quasi-Optimal Algorithmic Guarantees

While greedy and randomized column selection algorithms (e.g., CPQR, GECP, and RRQR) function well empirically, theoretical guarantees often exhibit exponential dependencies on rank in the worst case, whereas observed behavior is substantially better. The challenge is to characterize structural (e.g., positivity, smoothness, diagonal dominance) or problem-dependent conditions under which quasi-optimal performance—such as polynomial or logarithmic factors in bounds—can be ensured.

Block, Row, and Partition Selection in Structured Matrices

For matrices with underlying geometric or physical structure (e.g., arising in hierarchical solvers, kernel learning, or PDE discretizations), efficient block or partition-adaptive low-rank approximations are essential. Open questions pertain to the existence and implementation of algorithms yielding near-optimal approximations for broad PDE families, and the optimal dependence of accuracy on partitioning parameters.

Laplacian Inverse and Submodularity

Recently, the submodularity of the nuclear norm Nyström approximation error for Laplacian inverses has been established, with immediate implications for Markov chain compression and graph reduction. Extending this result to SDDM, SDD, or less structured matrices could broaden the range of greedy and volume sampling-based methods with formal performance guarantees.

Volume Sampling Tightness

An explicit challenge is to quantify the tightness of volume sampling-based error bounds relative to worst-case optimal subset selection, particularly for trace/nuclear objectives and fixed spectrum cases. This impacts foundational sampling theory in low-rank matrix approximation.

Randomized Sketching and Embeddings

Subspace Injections Versus Embeddings

The subspace embedding (OSE) paradigm is often stricter than necessary for practical accuracy in sketched least squares, regression, and low-rank approximation. The more permissive subspace injection (OSI) property can enable sketch dimension or sparsity improvements, and open problems involve establishing when OSI suffices for relative-error bounds (not just constant-factor), including for $\ell_p$ problems and randomized SVD.

Sparse Embeddings: Optimality and Phase Transitions

The Nelson–Nguyen conjecture on optimal tradeoffs between dimension and sparsity for very sparse embeddings (e.g., SparseStack) is central. While recent breakthroughs reduced subpolylogarithmic gaps [chenakkod2026optimal], closing them fully and identifying exact thresholds for constant sparsity and dimensions remains unresolved. Empirical success at constant sparsity below established OSE barriers suggests that OSI-based analyses may unlock further progress [Tro25].

Fast Structured Transforms

The need for tight bounds for rerandomized structured transforms (e.g., two- or three-level randomized Hadamard transforms) is highlighted by both practical usage and empirical evidence for $k=O(r)$. The theoretical picture is still evolving, especially for OSI properties and for multiple rounds of randomization in large-scale, GPU- or hardware-accelerated settings.

Tensors, Quantum Problems, and Matrix Functions

Tensor Train and Hierarchical Tensor Decompositions

Theoretical bounds for the approximation error of tree-based and tensor train decompositions are limited to those implied by sequential SVDs, typically with an $(m-1)\epsilon$ bound for an $m$-node tree. Whether strictly better polynomial algorithms exist, or whether this bound is tight under mild assumptions, is a significant open question. There are special cases (Tucker format), and randomized [amsel2025quasioptimalhierarchicallysemiseparablematrix] and structured [ceruti2025low] matrix analogs, but comprehensive analysis is missing.

Nonlinear Eigenvalue Problems from Quantum Systems

New NEPv instances motivated by quantum Hamiltonians—especially those with tensor product structure—pose the dual challenge of algorithmic scalability (iteration per $d$ factors in the system, not full dimension) and local superlinear convergence. Despite the existence of Newton-type methods [bai2018robust], the step complexity relative to system size and structure is open.

Matrix Sign Function with Matrix Multiplication Constraints

There is a renewed focus on approximating the matrix sign function using a fixed number of matrix-matrix multiplications, a critical building block in recursive eigensolvers, density matrix computation, and other high-performance contexts. Tight estimates for the minimal approximation error as a function of allowed matrix multiplications, structural properties (e.g., composition of low-degree polynomials), and connections to classical approximation theory are open.

Conclusion

This document articulates open problems at the interface of numerical linear algebra and theoretical computer science, emphasizing challenges at the algorithmic, structural, and complexity-theoretic levels. Progress on these issues will likely require new theoretical frameworks, advanced randomization/derandomization techniques, deeper understanding of matrix and operator structure, and attention to practicality in high-dimensional and hardware-oriented environments. The cross-disciplinary nature of these problems offers substantial avenues for breakthroughs in computational mathematics, data science, simulation, and emerging areas such as quantum and tensor computation.