Optimal Interpolation Coordinate Descent

• OICD is a coordinate descent method that exploits interpolation to reconstruct local function landscapes from minimal samples.
• The algorithm leverages trigonometric and spectral techniques to achieve near-exact 1D updates in both convex problems and parameterized quantum circuit training.
• Empirical benchmarks show that OICD markedly improves convergence rates and statistical efficiency over traditional methods under noise.

Optimal Interpolation-Based Coordinate Descent (OICD) is a class of coordinate-descent algorithms that leverage interpolation techniques to achieve improved convergence and statistical efficiency in high-dimensional optimization. OICD frameworks have been applied in both classical convex optimization and parameterized quantum circuit (PQC) training. The core conceptual advance is the use of structural knowledge—such as trigonometric expansions in quantum circuits or spectral properties of quadratic forms—to construct exact or nearly exact local updates from a small set of samples, optimizing convergence rates, stability, and measurement efficiency (Lai et al., 6 Mar 2025, Kovalev et al., 2018).

1. Problem Formulation and General Principles

OICD approaches address optimization problems where the objective function exhibits exploitable structure—either as a quadratic form or a trigonometric polynomial—when restricted to a single coordinate. In parameterized quantum circuits, for instance, the cost function

$$f(\theta) = \langle 0| U(\theta)^\dagger M U(\theta)|0\rangle$$

with $U(\theta)$ a product of parameterized unitaries and $M$ Hermitian, reduces (via spectral decomposition) to a finite Fourier series in each parameter:

$$f_j(x) = a_0/\sqrt2 + \sum_{k=1}^{r_j} [a_k \cos(\Omega_k^j x) + b_k \sin(\Omega_k^j x)]$$

when all but one parameter are fixed. Each $f_j(x)$ can thus be exactly specified by $2r_j+1$ samples.

Classical OICD for quadratics minimizes

$f(x) = \frac12 x^T A x - b^T x$

by iteratively selecting directions (coordinates, eigenvectors, or spectral combinations), and performing exact 1D minimizations in those directions (Kovalev et al., 2018).

2. Interpolation-Based Framework and Node Selection

In OICD, the 1D subproblem for a chosen coordinate or direction $j$ involves reconstructing $f_j(x)$ using $n=2r_j+1$ sampled points, known as "interpolation nodes":

• For PQCs: nodes $x_0, ..., x_{n-1}$ are chosen, measurements $y_i = \tilde{f}_j(x_i)$ are performed under quantum shot noise, and the interpolation matrix $A$ is built with trigonometric basis functions.
• The least-squares or interpolation solution recovers the coefficients $\hat{z} = A^{-1}y$, yielding a classical surrogate $\hat{f}_j(x) = t(x)^T \hat{z}$.

The selection of interpolation nodes is crucial under noise. For general frequencies $\{\Omega_k\}$, node optimization is numerical. For equispaced (integer) frequencies, as present when Pauli generators are used, uniform nodes $x_k = s + 2\pi k / n$ minimize mean squared error (MSE), condition number of $A$, and average derivative variance simultaneously. This yields highly stable and unbiased updates (Lai et al., 6 Mar 2025).

3. Algorithmic Procedure and Update Rules

The OICD iteration consists of:

1. Randomly or cyclically choosing a coordinate $j$ (or, in classical settings, a direction from an augmented set).
2. Gathering $2r_j+1$ quantum (or function) evaluations at optimally spaced nodes.
3. Computing trigonometric (or spectral) interpolation to reconstruct the restricted cost landscape.
4. Solving for the exact global minimizer of this surrogate, using methods such as:
   • Closed-form arctangent for $r=1$
   • Eigenvalue-based root-finding of a trigonometric polynomial for $r>1$ (Boyd–Mao method).
5. Updating $\theta_j$ or $x$ and repeating.

In the quadratic case, OICD interpolates between standard Randomized Coordinate Descent (RCD), which selects coordinates, and Stochastic Spectral Descent (SSD), which selects all eigenvectors, by mixing coordinate and spectral directions according to carefully chosen probabilities (Kovalev et al., 2018).

4. Statistical Properties and Theoretical Guarantees

OICD possesses several favorable theoretical properties:

• Unbiasedness: $E[\hat{z}] = z$ and $E[\hat{f}_j(x)] = f_j(x)$ under measurement noise.
• Optimal MSE: At optimal equispaced nodes, MSE$(\hat{z}) = 2\sigma^2$ (independent of $r$), with perfect numerical conditioning ($\mathrm{cond}(A) = 1$).
• Variance of derivatives: For derivative order $d$, $$\mathrm{Var}[\hat{f}^{(d)}(x)] = (2\sigma^2/n)\sum_{k=1}^r k^{2d}$$.
• Complexity: Quantum call cost matches RCD, $2r_j$ evaluations per update, with $O(r_j^3)$ classical cost per update.
• Coordinate/Spectral rate tuning: In quadratics, the convergence rate can be explicitly characterized as a function of the number of spectral directions incorporated, interpolating between the slowest (RCD) and fastest (SSD) cases, and adapted to the spectrum structure (Kovalev et al., 2018).

5. Empirical Performance and Practical Applications

Comprehensive numerical benchmarks validate OICD's efficiency:

Problem Domain	Setup	OICD Benefits
PQCs (MaxCut, TFIM, XXZ)	Qiskit Aer, $N=4$–6 qubits	Fewer quantum evaluations, rapid converge.
Quadratics	Large sparse matrices	Accelerated convergence with spectral mixing.

In PQC training (e.g., MaxCut, transverse-field Ising, XXZ spin chain), OICD attains ground-state fidelities $>0.999$ in markedly fewer quantum circuit evaluations than SGD or RCD. Under measurement noise, only the analytically-derived equispaced nodes lead to robust convergence; suboptimal node choices fail to converge reliably (Lai et al., 6 Mar 2025).

In classical settings with clustered spectrum, even modest spectral augmentation (small $k$) in OICD reduces iteration count from $O(\mathrm{Tr}(A)/\lambda_1)$ toward $O(n)$ as in SSD. Mini-batch extensions, inexact spectral directions, and practical considerations regarding eigenvector computation are detailed in (Kovalev et al., 2018).

6. Limitations, Open Questions, and Prospects

OICD's principal limitations include:

• For quantum circuits, formal convergence guarantees under measurement noise and hardware-induced error require further study.
• Reduction of $r_j$ via Fourier sparsity, and extension to arbitrary Hamiltonians, remain open.
• For classical problems, the cost of computing eigenvectors is amortized only when spectral structure is suitably exploited; diminishing returns can set in as $k$ grows.
• Robustness to forms of noise beyond measurement shot noise and to inexact direction sets must be characterized further.

A plausible implication is that OICD's effectiveness depends critically on careful node selection and on matching the interpolation structure to the underlying function's algebraic form. Its introduction marks a significant methodological advance for high-fidelity variational quantum algorithms and large-scale quadratics where structure can be exploited for statistical and computational gains (Lai et al., 6 Mar 2025, Kovalev et al., 2018).