Double Descent in Quantum Kernel Ridge Regression

Abstract: Various classical machine learning models, including linear regression, kernel methods, and deep neural networks, exhibit double descent, in which the test risk peaks near the interpolation threshold and then decreases in the overparameterized regime. However, this phenomenon has received less attention in the quantum setting. In this work, we investigate the double descent phenomenon in quantum kernel ridge regression (QKRR). By applying deterministic equivalents from random matrix theory (RMT), we derive an asymptotic expression for the test risk of QKRR in the high-dimensional limit. Our analysis rigorously characterizes the interpolation peak and reveals how explicit regularization can effectively suppress it. We corroborate our theoretical results with numerical simulations, demonstrating close agreement even for finite-size quantum systems.

• The paper establishes a closed-form asymptotic expression for the test risk in QKRR, capturing double descent behavior with explicit regularization.
• The paper leverages advanced random matrix theory and quantum feature map design to rigorously analyze the interpolation peak and benign overfitting regime.
• The paper validates its theoretical framework with simulations on synthetic and real-world data, confirming the applicability of RMT in quantum learning.

Double Descent in Quantum Kernel Ridge Regression

Introduction

The manuscript "Double Descent in Quantum Kernel Ridge Regression" (2604.17202) investigates the emergence and analytical characterization of double descent phenomena in Quantum Kernel Ridge Regression (QKRR). While double descent—where test risk as a function of model complexity exhibits a peak at the interpolation threshold and decreases for further overparameterization—has been rigorously studied in classical machine learning (CML), its rigorous extension to quantum machine learning (QML), and especially to quantum kernel methods, has remained largely unexplored. The paper leverages advanced random matrix theory (RMT) techniques to derive deterministic equivalents for the test risk in QKRR and provides numerical validation for these asymptotic results, even for finite quantum circuits.

Context: Double Descent in Classical and Quantum Learning

In the classical regime, double descent arises ubiquitously in high-dimensional linear models, kernel machines, and deep networks, marking a transition from the standard U-shaped bias-variance curve predicted by classical generalization theory to a more complex non-monotonic risk landscape as the number of model parameters $p$ exceeds the number of training samples $N_{\mathrm{tr}}$.

Figure 1: Conceptual illustration of double descent. The classic U-shaped risk (red, dotted) gives way to a second descent following an interpolation peak (blue, solid) at $p = N_{\mathrm{tr}}$.

The blue solid curve in Figure 1, derived from RMT, contrasts with conventional bounds by predicting a decreased test risk in the highly overparameterized regime. This reveals benign overfitting—models can perfectly interpolate noisy training data while maintaining nontrivial generalization.

Figure 2: Concrete example of double descent in polynomial regression, illustrating underfitting ($p=1$), near-correct specification ($p=3$), high-variance interpolation ($p=20$), and benign overfitting ($p=1000$) for $N_{\mathrm{tr}}=20$.

While theoretical advances in CML have precisely characterized double descent and the efficacy of implicit and explicit regularization, the QML setting—where features are quantum states produced by parameterized quantum circuits (PQCs)—introduces correlated and highly non-Gaussian feature distributions. This raises both a technical challenge and an opportunity to probe the universality of double descent in quantum models.

Theoretical Framework: Quantum Kernel Models and RMT

The authors rigorously map quantum kernel models onto the RMT framework by considering feature vectors derived from quantum states encoded via PQCs. These quantum feature maps are shown to be Lipschitz continuous with respect to the input, permitting the application of modern deterministic equivalent theory from RMT.

Notably, the ambient quantum feature dimension $p'$ is $4^n$ for $N_{\mathrm{tr}}$0 qubits (due to the Pauli expansion), and the effective feature dimension $N_{\mathrm{tr}}$1 is controlled by the expressivity of the ansatz (full-rank for highly expressive circuits, $N_{\mathrm{tr}}$2 for tensor product circuits). The authors build an effective quantum feature map that diagonalizes the population covariance, facilitating theoretical analysis and confirming that the performance curve is ultimately governed by the nonzero spectrum of the kernel operator.

The logical structure of the approach is diagrammed succinctly:

(Figure 3—not displayed, but described in-text)

Figure 3: Theoretical framework: from quantum feature map generation (concentrated vectors), through bias-variance decomposition, to RMT deterministic equivalent calculations yielding the main test risk expression.

Main Results: Asymptotic Test Risk in QKRR

The core analytical result is a closed-form asymptotic expression for the test risk in QKRR as $N_{\mathrm{tr}}$3 at fixed ratio $N_{\mathrm{tr}}$4, incorporating the quantum-specific population covariance $N_{\mathrm{tr}}$5, the regularization parameter $N_{\mathrm{tr}}$6, and the effective degrees of freedom $N_{\mathrm{tr}}$7.

The test risk deterministic equivalent is

$N_{\mathrm{tr}}$8

where

• $N_{\mathrm{tr}}$9 solves the self-consistent equation

$p = N_{\mathrm{tr}}$0

• $p = N_{\mathrm{tr}}$1,
• and $p = N_{\mathrm{tr}}$2 encodes the target projection onto the kernel eigenbasis.

Crucially:

• The divergence of the variance term $p = N_{\mathrm{tr}}$3 at $p = N_{\mathrm{tr}}$4 is the analytic signature of the interpolation peak.
• Both the depth and position of the second descent depend on the type of quantum circuit, data spectrum, and regularization.

Figure 4: Test risk of QKRR with HEA, empirical vs theoretical: double descent is suppressed as $p = N_{\mathrm{tr}}$5 increases. Highly expressive circuits with $p = N_{\mathrm{tr}}$6 qubits exhibit the predicted peak.

Figure 5: Test risk of QKRR with TPA (lower effective dimension), empirical vs theoretical: double descent and regularization sensitivity are preserved but peak amplitude and alignment are modulated by circuit expressiveness.

Figure 6: Behavior of $p = N_{\mathrm{tr}}$7 with respect to $p = N_{\mathrm{tr}}$8 and $p = N_{\mathrm{tr}}$9 across the isotropic covariance spectrum.

Numerical Validation

Simulations on both synthetic and real-world data (Fashion-MNIST) verify the robustness of the asymptotic risk expression. The deterministic equivalent accurately predicts the empirical test risk profile, including the interpolation peak amplitude and position, even for modest qubit numbers ($p=1$0). Increasing $p=1$1 consistently dampens and smooths the double descent. The agreement is tighter for higher $p=1$2, evidencing the high-dimensional validity of RMT tools in quantum settings.

Implications and Theoretical Significance

This work extends the universality of double descent into quantum kernel methods, establishing that the interplay of quantum feature generation, data spectral properties, and regularization leads to qualitatively and quantitatively similar risk landscapes as their classical analogs, even in regimes of quantum expressivity. The derived test risk encapsulates all quantum-specific effects in the nontrivial spectrum of $p=1$3, including settings where kernel bandwidth or circuit entanglement induce either flat or rapidly decaying spectra.

The results corroborate recent critiques of classical generalization bounds in quantum learning [gil-fuster2024understanding], validating that quantum models exhibit intricate double descent structure not explained by traditional VC or Rademacher complexity approaches but captured by high-dimensional asymptotics.

Explicit regularization is quantitatively shown to control the interpolation peak, providing a tunable mechanism for mitigating excess variance at the critical threshold. The theory also frames when benign overfitting is feasible in quantum models (flat spectrum) and delineates the necessity for spectrum engineering via ansatz design or kernel construction for quantum advantage.

Outlook and Future Directions

The deterministic equivalent test risk for QKRR lays groundwork for more advanced quantum learning theory. Future extensions should explicitly incorporate:

• Shot noise and kernel estimation errors—the current treatment presumes ideal kernel access and should be generalized for NISQ devices subject to measurement noise and barren plateaus [thanasilp2022exponential, larocca2024review].
• Task-dependent analysis—assessing phase transitions where the second descent outperforms the best classical curve, thus quantifying quantum advantage in overparameterized regimes.
• Generalization to variational quantum circuit learning, especially where implicit unitarity-induced regularization and non-Gaussian features further modulate risk behavior.
• Regularization- and epoch-wise double descent in quantum neural networks, leveraging explicit matrix algebra and advanced RMT results for quantum nonlinear models.

Conclusion

The paper rigorously identifies and characterizes double descent in QKRR, demonstrating that key phenomena originated in high-dimensional statistics and classical learning extend—both mathematically and empirically—to quantum kernel methods with intricate kernel-induced feature dependencies. The integration of quantum feature map theory and RMT yields a robust formula for test risk in the presence of explicit regularization, providing an actionable analytic basis for principled quantum model and kernel design in the overparameterized regime. The theoretical framework and validations presented invite further exploration of quantum statistical learning at the interplay of expressivity, regularization, and noise.