Low-rank Support Quaternion Matrix Machine

• The paper introduces LSQMM, employing quaternion nuclear norm regularization to capture inter-channel spectral correlations for improved color image classification.
• LSQMM leverages an ADMM solver with quaternion singular value thresholding to attain convergence and competitive computational efficiency compared to traditional models.
• Empirical evaluations demonstrate LSQMM's superior accuracy, robustness to noise, and efficiency over standard SVM and tensor-based classifiers on multiple color image datasets.

The Low-rank Support Quaternion Matrix Machine (LSQMM) is a discriminative learning framework for color image classification in which input features and classifier parameters are encoded as pure quaternion matrices. LSQMM advances conventional real-valued vector and matrix models by treating the three color channels (R, G, B) as coherent components of a pure quaternion variable, thereby exploiting inter-channel spectral correlation via quaternion algebra. Central to LSQMM is the incorporation of the quaternion nuclear norm as a regularizer, promoting low-rank structure in the quaternion domain to leverage channel correlations. The resulting optimization is tackled via an alternating direction method of multipliers (ADMM) solver with proven convergence guarantees and competitive computational complexity. Empirically, LSQMM yields superior accuracy, robustness, and efficiency over established SVM and matrix/tensor-based classifiers on multiple color image datasets (Chen et al., 9 Dec 2025).

1. Quaternion Representation and Motivation

LSQMM models each input color image as a pure quaternion matrix $X \in \mathbb{Q}^{m \times n}$, where each pixel is encoded as $x_{ij} = r_{ij}i + g_{ij}j + b_{ij}k$ (with $r_{ij}$, $g_{ij}$, $b_{ij}$ real-valued intensities). This holistic representation preserves the intrinsic inter-channel coupling that is otherwise lost if channels are treated independently. The decision function of LSQMM is parameterized by a quaternion weight matrix $W \in \mathbb{Q}^{m \times n}$, allowing the classifier to act on all spectral and spatial dependencies.

Preserving and exploiting channel relationships is fundamental in color image processing, as RGB channels of natural images typically exhibit strong linear or nonlinear dependencies. Traditional machine learning models operate on concatenated features or stacked matrices in the real field, ignoring these spectral structures. Quaternionic modeling introduced in LSQMM is motivated by demonstrated success in quaternion-based color image recovery, denoising, and restoration tasks, where similar channel coupling strategies have been shown to yield state-of-the-art performance.

2. Quaternion Nuclear Norm Regularization

Let $A \in \mathbb{Q}^{m \times n}$ be a quaternion matrix of (right) rank $r$. The quaternion singular value decomposition (QSVD) yields

$A = U\Sigma V^*,$

where $U, V$ are unitary quaternion matrices and $$\Sigma = \operatorname{diag}(\sigma_1, \ldots, \sigma_r, 0, \dots, 0)$$ contains the real, nonnegative singular values $\sigma_1 \geq \cdots \geq \sigma_r > 0$. The quaternion nuclear norm is defined as

$\|A\|_{Q,*} := \sum_{i=1}^r \sigma_i.$

This norm is the tightest convex relaxation of rank in the quaternion context, inheriting all majorization and variational properties from the real and complex settings. Via a real embedding isomorphism $$\Psi: \mathbb{Q}^{m \times n} \to \mathbb{R}^{4m \times 4n}$$, there exists a scaling constant $c$ such that $\|A\|_{Q,*} = c \|\Psi(A)\|_*$, linking quaternion and real nuclear norms fundamentally (Chen et al., 9 Dec 2025).

The low-rank bias enforced by $\|W\|_{Q,*}$ encourages the classifier to leverage the joint spatial-spectral structure of the color input space, naturally controlling overfitting in high-dimensional regimes common to image classification.

3. LSQMM Objective Function and Constraints

The LSQMM model seeks a low-complexity classifier by minimizing a regularized hinge-loss functional: $$\begin{aligned} \min_{W \in \mathbb{Q}^{m \times n},\, b \in \mathbb{R},\, \{\xi_i\}} &\quad \frac{1}{2}\|W\|_F^2 + \lambda\,\|W\|_{Q,*} + C\sum_{i=1}^N \xi_i\ \text{s.t.} &\quad y_i\left(\Re\,\langle W, X_i\rangle + b\right) \ge 1-\xi_i, \quad \xi_i \ge 0,\quad i=1,\dots,N. \end{aligned}$$ Here, $\|W\|_F^2$ is the Frobenius (Hilbert–Schmidt) norm, $\|W\|_{Q,*}$ is the quaternion nuclear norm promoting low rank, and slack variables $\{\xi_i\}$ control misclassification. The quaternion-valued inner product is $\langle W, X\rangle = \operatorname{Tr}(W^* X)$, with $\Re$ taking its real part. Parameters $C>0$ and $\lambda>0$ trade off between loss, slack, and regularization. This objective generalizes the support matrix machine (SMM) by replacing the real matrix norm with its quaternion counterpart (Chen et al., 9 Dec 2025).

4. ADMM Optimization Framework

To address the non-smooth, coupled nature of the objective, LSQMM introduces an auxiliary variable $Z$ to decouple the nuclear and Frobenius regularizers, enforcing the constraint $W = Z$. The augmented Lagrangian becomes

$$L_\rho(W, b, Z, U) = \frac{1}{2}\|W\|_F^2 + \lambda\|Z\|_{Q,*} + C\sum_i h\big(1 - y_i(\Re\langle W, X_i \rangle + b)\big) - \Re\langle U, W - Z\rangle + \frac{\rho}{2}\|W - Z\|_F^2,$$

where $U$ is the quaternion Lagrange multiplier, $h(t)=\max\{0, t\}$ is the hinge loss, and $\rho > 0$ the penalty parameter.

The ADMM update sequence is:

1. Update $W$, $b$: Solve a convex quadratic problem in $W$ and $b$, yielding

$$W^{k+1} = \frac{1}{1+\rho}(\rho Z^k + U^k + \sum_{i=1}^N \alpha_i^* y_i X_i),$$

where the $\alpha_i^*$ are SVM-like Lagrange multipliers.

2. Update $Z$: Apply quaternion singular value thresholding (QSVT). If $$W^{k+1} - U^k / \rho = \widetilde{U} \operatorname{diag}(\{\sigma_i\})\widetilde{V}^*$$ is the QSVD,

$$Z^{k+1} = \widetilde{U} \operatorname{diag}\big\{\max(\sigma_i - \lambda/\rho, 0)\big\} \widetilde{V}^*.$$

3. Multiplier update:

$U^{k+1} = U^k - \tau \rho (W^{k+1} - Z^{k+1}),$

with relaxation $\tau \in (0, 1.618]$.

The stopping criterion is based on the normalized primal residual $$\|W^k - Z^k\|_F / \max\{\|W^k\|_F, \|Z^k\|_F\} < 10^{-3}$$ or a fixed iteration limit.

Convergence is guaranteed via mapping to the equivalent real-matrix ADMM and invoking two-block convex convergence results. Each update is computationally dominated by the quaternion SVD, with $\mathcal{O}(mn \max\{m, n\})$ cost per iteration (Chen et al., 9 Dec 2025).

5. Theoretical and Empirical Evaluation

The quaternion nuclear norm regularization in LSQMM achieves several effects:

• Low-rank Promotion: $\|W\|_{Q,*}$ induces low rank in the quaternion weight matrix, capturing the highly correlated structure of RGB color channels.
• Improved Classification Accuracy: Across multiple binary color image classification tasks, LSQMM matches or outperforms real-valued SMM, support tensor machines, and standard LIBSVM, with absolute classification accuracy boosts of several percentage points in some settings.
• Robustness to Noise: When training or test data are corrupted with Gaussian noise (signal-to-noise ratios up to 1.0), LSQMM degrades in accuracy more slowly compared to competitors, attributed to the enforced joint spatial-spectral structure.
• Computational Efficiency: Despite the extra cost of computing a quaternion SVD in each iteration, LSQMM remains competitive in runtime, particularly relative to tensor-based models, and does not suffer from excessive computational overhead given the classification accuracy gained (Chen et al., 9 Dec 2025).

6. Position Relative to Broader Quaternion Low-Rank Learning

LSQMM extends the suite of quaternion low-rank methods—such as quaternion matrix/tensor completion, robust principal component analysis (RPCA), and image recovery—to the classification domain. The core methodological principle, i.e., the use of the quaternion nuclear norm as a convex surrogate for rank, appears throughout the quaternion signal processing literature, including in regression models for color face recognition (Miao et al., 2020), matrix completion (Yang et al., 2021), robust denoising (Huang et al., 19 Oct 2024, Guo et al., 30 Apr 2025, Guo et al., 12 Sep 2024), and weighted/truncated nuclear norm models (Zhang et al., 2023). Empirical superiority of LSQMM for small-sample, high-dimensional color-image classification mirrors the superior reconstruction performance of quaternion nuclear-norm-regularized models in image recovery, supporting the general efficacy of exploiting spatial-spectral low-rankness in multichannel data analysis (Chen et al., 9 Dec 2025).

7. Implementation and Practical Considerations

Effective deployment of LSQMM relies on efficient QSVD computation, which can be achieved via isomorphic embeddings into larger real or complex matrices. Selection of hyperparameters ($\lambda$, $C$, and ADMM scheduling parameters) is commonly performed via cross-validation. The ADMM framework is modular and compatible with further extensions, such as alternative loss functions (e.g., logistic, squared hinge), more intricate regularizations (e.g., group-sparsity or robust losses), and kernelization for nonlinear decision boundaries, following similar techniques developed in the real and complex matrix machine literature.

LSQMM thus constitutes a general, theoretically grounded approach for multichannel image classification tasks where spatial and spectral correlations are dominant, providing robustness, accuracy, and algorithmic clarity (Chen et al., 9 Dec 2025).