Quantum-Inspired Classical LS-SVM

• Quantum-Inspired Classical LS-SVM is a scalable approach that applies quantum-inspired techniques, like length-square sampling and spectral decomposition, to classical LS-SVM optimization.
• Indirect sampling via length-square rejection and submatrix sketching reduces the computational complexity of kernel matrix operations while ensuring rigorous error bounds.
• The method offers efficient distributed implementations and bridges the gap between traditional SVMs and quantum LS-SVM algorithms, enhancing performance on high-dimensional data.

Distributed Newton-type Support Vector Machines (HPSVM) refer to a class of scalable Support Vector Machine (SVM) algorithms that utilize Newton-type optimization methods in a distributed or parallel setting. This approach enables efficient training of SVMs on large-scale or high-dimensional datasets by leveraging core concepts from quantum and quantum-inspired computational paradigms, such as length-square sampling, submatrix sketching, and efficient singular value transformations, facilitating polylogarithmic runtime in favorable regimes.

1. Least Squares SVM: Formulation and Duality

The least-squares SVM (LS-SVM) operates on training data $\{(x_j, y_j)\}_{j=1}^m$ with $x_j \in \mathbb{R}^n$ and $y_j \in \{+1, -1\}$, seeking a separating hyperplane $w^T x + b = 0$. The LS-SVM optimization is: $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$ where $\gamma > 0$ is the regularization parameter. By introducing Lagrange multipliers and eliminating primal variables, the dual normal equations simplify (for $b = 0$) to: $(X^T X + \gamma^{-1} I_n)\alpha = y$ with $X \in \mathbb{R}^{n \times m}$ and $y \in \mathbb{R}^m$. For linear kernels, $x_j \in \mathbb{R}^n$0, yielding: $x_j \in \mathbb{R}^n$1 Prediction on a new $x_j \in \mathbb{R}^n$2 is via

$x_j \in \mathbb{R}^n$3

(Ding et al., 2019).

2. Indirect Sampling and Sketching Techniques

Direct formation or inversion of kernel matrices $x_j \in \mathbb{R}^n$4 is computationally prohibitive for large $x_j \in \mathbb{R}^n$5 due to $x_j \in \mathbb{R}^n$6 costs. Indirect sampling circumvents this by length-square sampling of columns (size $x_j \in \mathbb{R}^n$7) and rows (size $x_j \in \mathbb{R}^n$8) of $x_j \in \mathbb{R}^n$9 to form smaller matrices $y_j \in \{+1, -1\}$0 and $y_j \in \{+1, -1\}$1. For column sampling: $y_j \in \{+1, -1\}$2 Bounds on approximations are given in terms of matrix norms, with samples $y_j \in \{+1, -1\}$3 and $y_j \in \{+1, -1\}$4 required to control the error with high probability (Ding et al., 2019, Jethwani et al., 2019). Length-square rejection sampling and trace inner-product estimation (via randomized algorithms) are used to efficiently estimate quantities appearing in the LS-SVM solution.

3. Quantum-Inspired SVM Algorithms and SVT Framework

Inspired by quantum linear system and SVM solvers, such as amplitude encoding plus the Harrow–Hassidim–Lloyd (HHL) algorithm, classical quantum-inspired approaches replace quantum memory access with length-square sampling and submatrix spectral decompositions (Ding et al., 2019). The Quantum-Inspired Support Vector Machine (QI-SVM) method operates as follows:

1. Subsample columns/rows to sketch the kernel.
2. Compute an $y_j \in \{+1, -1\}$5 spectral decomposition.
3. Use indirect estimators for traces and inner products (length-square and rejection sampling).
4. Output approximate dual variables $y_j \in \{+1, -1\}$6.
5. Classify new points using $y_j \in \{+1, -1\}$7.

In the more general singular value transformation (SVT) framework, for a function $y_j \in \{+1, -1\}$8 and SVD $y_j \in \{+1, -1\}$9, one constructs $w^T x + b = 0$0. For ridge regression and LS-SVM, $w^T x + b = 0$1 so that

$w^T x + b = 0$2

Efficient randomized sketching, polynomial (Chebyshev) approximations, and sampled post-processing enable approximate solutions with rigorous error guarantees (Jethwani et al., 2019).

4. Extensions to Nonlinear Kernels

For kernels beyond the linear case, the mapping is generalized: the data is mapped by $w^T x + b = 0$3, forming $w^T x + b = 0$4 and $w^T x + b = 0$5. Polynomial kernels are handled by $w^T x + b = 0$6-fold rejection sampling of $w^T x + b = 0$7. For shift-invariant kernels (e.g., RBF), approximate random Fourier features of dimension $w^T x + b = 0$8 can be sampled in the same length-square manner (Ding et al., 2019).

5. Complexity and Success Probabilities

Under low-rank ($w^T x + b = 0$9), favorable condition number ($$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$0), and high-dimensional structure, the total runtime for these quantum-inspired algorithms is

$$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$1

where $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$2 hides $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$3 factors and $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$4 is the target failure probability. When $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$5, this yields polylogarithmic scaling in the data dimensions, closely matching the scaling of quantum LS-SVM via HHL, but entirely within the classical computational model. Accurate parameter choices for sample sizes $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$6 and trace estimation repetitions guarantee final failure probabilities $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$7 (Ding et al., 2019).

6. Data-Access and Distributed Implementation Models

QI-SVMs and quantum-inspired SVT approaches assume efficient $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$8-sampling access to the data and labels, supported in $$\min_{w, b, e}\quad \frac{1}{2}\|w\|^2 + \frac{\gamma}{2}\sum_{j=1}^m e_j^2 \quad \text{subject to}\quad y_j(w^T x_j + b) = 1 - e_j,\quad j=1, \dots, m$$9 time via weighted $\gamma > 0$0-trees. This supports querying, updating, row/column norm computation, and randomized index access for efficient distributed or parallel implementation (Jethwani et al., 2019). The double-sampling and sketching steps map directly to distributed frameworks, enabling scalable training of SVMs with Newton-type updates on massive datasets.

7. Comparative Perspective: Quantum, Quantum-Inspired, and Distributed Newton-type SVMs

Table: High-level comparison of approaches

Method	Kernel Matrix Inversion	Data Scaling
Quantum LS-SVM [HHL]	Amplitude encoding/HHL	$\gamma > 0$1 for $\gamma > 0$2
Quantum-Inspired LS-SVM	Length-square sampling, sketching, SVD	$\gamma > 0$3 for $\gamma > 0$4
Classical SVM	Matrix inversion	$\gamma > 0$5

Quantum-inspired LS-SVMs “dequantize” the speedup claims of quantum algorithms for SVMs, providing fully classical algorithms whose asymptotic data-size dependence is indistinguishable (up to polynomial factors in $\gamma > 0$6) from that of their quantum counterparts. Unlike fully quantum algorithms, quantum-inspired SVMs require only randomized access, with no Hilbert-space or amplitude encoding overhead (Ding et al., 2019, Jethwani et al., 2019).

The distributed implementation potential is a consequence of the sketch-and-solve paradigm and explicit data-access models, establishing these methods as central to modern scalable SVM optimization.