GRKneg: Negative-Augmented OC-SVM Kernel

• GRKneg is a kernel technique that integrates actual and generated negatives to improve the accuracy of one-class support vector machines.
• It constructs a centered kernel matrix using spectral thresholding of the pseudoinverse derived from combined real and synthetic negative samples.
• Empirical results demonstrate significant gains in outlier detection, especially when negative samples are scarce, compared to traditional OC-SVM.

A Generalized Reference Kernel with Negative Samples (GRKneg) is a kernel engineering technique for one-class support vector machines (OC-SVM) designed to leverage small sets of labeled negative samples to enhance outlier detection accuracy. Unlike classical OC-SVM, which constructs kernels solely from positive class data, GRKneg constructs a kernel that incorporates both actual and generated negative examples, thereby regularizing and reshaping the feature space in a principled manner while retaining compatibility with standard OC-SVM solvers. This approach demonstrates significant empirical improvements in low-negative-sample regimes, offering a drop-in replacement for the base RBF kernel in one-class classification tasks (Raitoharju, 17 Jun 2025).

1. Mathematical Formulation

Let $P = \{p_1, ..., p_P\} \subset \mathbb{R}^d$ denote the positive (inlier) training set, $N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$ the available negative (outlier) training set, and $\widetilde{k}(x, y)$ the base kernel, frequently chosen as RBF. The GRKneg approach forms an augmented reference vector set

$$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$

where $M = \{m_1, ..., m_G\}$ are $G$ generated negatives sampled independently from a Gaussian $\mathcal{N}(\mu_{\text{neg}}, \Sigma_{\text{neg}})$, with $\mu_{\text{neg}} = \mathrm{mean}(N)$ and $$\Sigma_{\text{neg}} = \mathrm{diag}(\mathrm{std}(N)^2)$$.

The centered reference-reference and reference-positive kernel matrices,

$\widetilde{K}_{RR},\ \widetilde{K}_{RP},$

are computed and centered via

$N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$0

to obtain $N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$1 and $N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$2.

The GRKneg kernel matrix on the positive set is

$N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$3

where $N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$4 is the Moore–Penrose pseudoinverse, computed using eigen-decomposition with only nonzero eigenvalues retained.

For any $N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$5, the GRKneg kernel is given by

$N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$6

with $N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$7 after centering. This construction is a valid Mercer kernel and defines an implicit mapping

$N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$8

No additional explicit regularization is introduced apart from spectral thresholding in the pseudoinverse (Raitoharju, 17 Jun 2025, Raitoharju et al., 2022).

2. Reference Vector Generation and Selection

Nine reference vector selection/generation strategies were evaluated. The optimal variant concatenates the original $N = \{n_1, ..., n_N\} \subset \mathbb{R}^d$9 negative training samples and $\widetilde{k}(x, y)$0 generated negatives from $\widetilde{k}(x, y)$1. The process is:

1. Compute $\widetilde{k}(x, y)$2 from negatives.
2. Generate $\widetilde{k}(x, y)$3 synthetic negatives $\widetilde{k}(x, y)$4.
3. Concatenate $\widetilde{k}(x, y)$5 to form $\widetilde{k}(x, y)$6 ($\widetilde{k}(x, y)$7).
4. Construct and center the kernel matrices as previously specified.
5. Perform eigen-decomposition and compute the pseudoinverse.
6. Form the final GRKneg kernel matrix.

Using only positives as references was found to be ineffective for OC-SVM, while using a mixture of true and generated negatives maximized kernel informativeness, particularly when real negatives are scarce (Raitoharju, 17 Jun 2025).

3. Integration with OC-SVM Framework

GRKneg serves as a direct replacement for the conventional kernel in standard OC-SVM pipelines. The OC-SVM dual maximization for $\widetilde{k}(x, y)$8-SVM remains: $\widetilde{k}(x, y)$9 subject to $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$0, $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$1.

All other algorithmic details, including parameter $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$2, thresholding, and solver mechanics, are unchanged from standard OC-SVM usage; only the kernel is replaced (Raitoharju, 17 Jun 2025).

4. Algorithmic Workflow

The GRKneg process comprises four main stages:

• Preprocessing: Standardize all positive samples and negative samples using positive-set statistics.
• Reference Kernel Construction:

1. Estimate $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$3.
2. Generate $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$4 synthetic negatives.
3. Build $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$5, form and center the $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$6 and $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$7 base kernel matrices.
4. Perform spectral decomposition and compute pseudoinverse to construct $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$8.

• OC-SVM Model Training: Use $$R_{\text{ref}} = N \cup M, \quad |R_{\text{ref}}| = N + G,$$9 and parameter $M = \{m_1, ..., m_G\}$0 in standard OC-SVM dual optimization to obtain $M = \{m_1, ..., m_G\}$1, threshold $M = \{m_1, ..., m_G\}$2.
• Testing: For a test point $M = \{m_1, ..., m_G\}$3, compute the centered $M = \{m_1, ..., m_G\}$4 vector, assemble $M = \{m_1, ..., m_G\}$5, and apply the OC-SVM scoring rule: $M = \{m_1, ..., m_G\}$6; classify as an outlier if $M = \{m_1, ..., m_G\}$7 (Raitoharju, 17 Jun 2025).

5. Hyperparameterization and Implementation

Principal parameters and settings include:

• Base-kernel bandwidth $M = \{m_1, ..., m_G\}$8, with $M = \{m_1, ..., m_G\}$9 the mean squared distance among training samples, and $G$0 swept over $G$1.
• OC-SVM parameter $G$2.
• Number of generated negative samples $G$3.
• Eigenvalue truncation: retain nonzero eigenvalues (as in LIBSVM pseudo-inverse defaults).
• Parameter selection via 5-fold cross-validation within training data.

Implementation is realized in MATLAB and LIBSVM via custom precomputed kernels (option $G$4) (Raitoharju, 17 Jun 2025).

6. Empirical Evaluation

Experiments were performed across 14 one-class benchmarks (UCI Iris, Seeds, Ionosphere, Sonar, Bankruptcy, Happiness) at negative counts $G$5. The geometric mean (Gmean $G$6) was the principal metric.

Key findings, averaged across tasks, include:

Negatives	SVM (Binary)	OC-SVM (RBF)	GRKneg (Variant 7)	GRK-Only Negatives (Var 3)
5	70.0 ± 12.7	73.7 ± 6.1	79.5 ± 9.0	74.6 ± 7.4
10	76.4 ± 5.9	75.0 ± 4.7	81.9 ± 4.6	79.2 ± 6.7
20	84.1 ± 3.2	75.9 ± 4.0	83.7 ± 4.1	82.8 ± 5.7

For very small negative sample sizes ($G$7 or $G$8), GRKneg outperformed both baseline OC-SVM and classical binary SVM. For larger negative sets ($G$9), binary SVM achieved the highest Gmean, but GRKneg remained competitive (Raitoharju, 17 Jun 2025). These patterns are consistent with general findings for negative-augmented GRK variants (Raitoharju et al., 2022).

7. Interpretation, Limitations, and Future Directions

GRKneg enables OC-SVM to exploit small amounts of negative-label information to engineer the kernel space, resulting in significant accuracy gains in data-scarce, imbalance-prone outlier detection scenarios. The kernel implicitly regularizes the model via the spectrum of the reference Gram matrix and may be viewed as an extension of the generalized reference kernel (GRK) framework with negative augmentation (Raitoharju et al., 2022).

Limitations include:

• Computational overhead due to the eigen-decomposition of the $\mathcal{N}(\mu_{\text{neg}}, \Sigma_{\text{neg}})$0 reference Gram matrix.
• Reliance on a Gaussian approximation for the negative distribution, which may not capture complex outlier structures.
• Introduction of hyperparameters such as the number of generated negatives and the base kernel scale.

A plausible implication is that further advances can be achieved by employing more sophisticated negative-sample generation techniques (e.g., mixture models, adversarial sampling), adapting reference selection for multi-view or deep-feature scenarios, and benchmarking against other semi-supervised or class-imbalance–aware SVM variants.

GRKneg unifies and extends prior kernel approximation and regularization paradigms—Nyström approximation, the Non-linear Projection Trick, and random mapping—into a kernel construction paradigm that directly incorporates side information, notably negative samples, to robustly enhance one-class classification performance in practical settings where negative data are limited (Raitoharju, 17 Jun 2025, Raitoharju et al., 2022).