One-Class Slab SVM (OCSSVM)

• OCSSVM is a kernel-based one-class classification method that defines a bounded slab using two parallel hyperplanes to capture the normal region of the target class.
• It optimizes both lower- and upper-tail errors through a dual quadratic programming formulation with slack variables, enhancing novelty detection over traditional OCSVM.
• Empirical evaluations show that OCSSVM, trained via an efficient SMO procedure, significantly reduces false positives and outperforms conventional methods on benchmark datasets.

The One-Class Slab Support Vector Machine (OCSSVM) is a kernel-based one-class classification framework that constructs two parallel hyperplanes in the feature space to define a “normal” region for the target class, thereby enabling robust novelty detection. By controlling both lower- and upper-tail outliers relative to the projected score distribution, OCSSVM addresses key limitations of traditional one-class SVM (OCSVM) approaches, notably reducing false positive rates arising from high-scoring negatives and increasing detection accuracy for novel instances (Fragoso et al., 2016, Kumar et al., 2020).

1. Motivation and Problem Formulation

Traditional OCSVM seeks a single separating hyperplane in feature space, admitting points satisfying $\langle w, \Phi(x) \rangle \geq \rho$ and rejecting those below as novel. This mechanism only bounds the lower tail. In many settings, negative samples may reside above the main mass of target scores, leading to high rates of false positives. OCSSVM defines a bounded “slab”

$\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$

with the same normal $w$ and offsets $\rho_1, \rho_2$. Instances falling outside this slab are deemed outliers, directly controlling both tails of the score distribution (Fragoso et al., 2016, Kumar et al., 2020).

2. Mathematical Derivation and Optimization

Given a set of $m$ positive-class training points $\{x_i\}_{i=1}^m$ mapped into a reproducing kernel Hilbert space by $\Phi(\cdot)$, OCSSVM introduces two sets of slack variables: $\xi_i \geq 0$ for lower-bound violations and $\bar{\xi}_i \geq 0$ for upper-bound violations. With control parameters $\nu_1, \nu_2 \in (0,1]$ and a weight $\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$0, the primal quadratic program is: $\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$1 Dualization leads to the convex quadratic program: $\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$2 where $\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$3 (Fragoso et al., 2016, Kumar et al., 2020).

The model parameters are reconstructed by

$\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$4

with offsets $\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$5 computed via

$\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$6

and likewise for $\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$7, but over support vectors with $\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$8 (Fragoso et al., 2016, Kumar et al., 2020).

The decision score is

$\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2$9

with classifier

$w$0

declaring $w$1 if $w$2, $w$3 otherwise.

3. Algorithmic Solution and SMO Training

The dual convex program admits solution by any off-the-shelf quadratic programming method. Due to scalability constraints ($w$4 memory, $w$5 time for generic QP), a modified Sequential Minimal Optimization (SMO) procedure is effective (Kumar et al., 2020). By posing the dual in terms of $w$6, the update at each iteration solves an analytic two-variable problem with box and slab constraints: $w$7 The SMO update takes the form: $w$8 along with appropriate clipping (Kumar et al., 2020).

The KKT-violation-based working set selection prioritizes variables corresponding to maximal slab constraint violation. Each outer iteration involves $w$9 computation. Empirically, convergence is achieved in tens to hundreds of steps, with rapid subquadratic scaling in both time and memory (Kumar et al., 2020).

4. Theoretical Properties

The OCSSVM objective is convex in $\rho_1, \rho_2$0, guaranteeing a unique solution. The generalization capacity is linked to the slab's margin in the projected 1-D score space; arguments using Rademacher complexity or covering numbers extend from the OCSVM literature, with a wide slab corresponding to small $\rho_1, \rho_2$1 and thus lower complexity for fixed empirical errors (Fragoso et al., 2016).

Computationally, the complexity per iteration matches OCSVM ($\rho_1, \rho_2$2); the number of variables doubles, but not the asymptotic cost. The SMO solver eliminates the infeasible $\rho_1, \rho_2$3 scaling for large datasets (Fragoso et al., 2016, Kumar et al., 2020).

5. Empirical Evaluation and Benchmarks

OCSSVM has been evaluated on multiple datasets:

Dataset	OCSVM MCC	OCSSVM MCC	Notes
Letter (RBF kernel)	≈0.07	≈0.39	26 classes, 16,000 train, 4,000 test
Letter (linear)	≈0.02	≈0.14
Pascal VOC 2012 (additive kern.)	0.04–0.02	0.26–0.18	20 object classes, HOG features

OCSSVM consistently outperforms OCSVM and is competitive with or superior to SVDD, KPCA, and KDE where applicable. On synthetic data, SMO-OCSSVM scales from 0.35s (500 samples) to 5.91s (5,000 samples), with MCC improving with data volume (0.07 to 0.33) (Fragoso et al., 2016, Kumar et al., 2020).

6. Applications and Practical Recommendations

OCSSVM is recommended in scenarios where traditional one-class novelty detection is affected by high false-positive rates from right-tail outliers, which is prevalent in open-set visual recognition and industrial anomaly detection tasks (Fragoso et al., 2016). Default parameters ($\rho_1, \rho_2$4, $\rho_1, \rho_2$5, $\rho_1, \rho_2$6) are effective starting points, with additional tuning via class-restricted validation. RBF kernels are a robust default, while histogram features benefit from intersection, Hellinger, or $\rho_1, \rho_2$7 kernels. Data sizes below 2,000 permit primal-dual QP methods; larger sets necessitate SMO (Fragoso et al., 2016, Kumar et al., 2020).

7. Implementation Outline

OCSSVM is constructed by assembling the kernel matrix $\rho_1, \rho_2$8, solving the dual for variables $\rho_1, \rho_2$9, $m$0 (or $m$1), extracting the primal weight $m$2 and offsets $m$3, $m$4 via KKT, and applying the two-hyperplane decision. The SMO solver is directly adaptable from standard OCSVM literature with the box-sum modifications above (Fragoso et al., 2016, Kumar et al., 2020).