---
title: One-Class Slab SVM (OCSSVM)
url: https://www.emergentmind.com/topics/one-class-slab-svm-ocssvm
type: topic
---

# One-Class Slab SVM (OCSSVM)

The One-Class Slab Support Vector Machine (OCSSVM) is a kernel-based one-class classification framework that constructs two parallel hyperplanes in the feature space to define a “normal” region for the target class, thereby enabling robust novelty detection. By controlling both lower- and upper-tail outliers relative to the projected score distribution, OCSSVM addresses key limitations of traditional one-class SVM (OCSVM) approaches, notably reducing false positive rates arising from high-scoring negatives and increasing detection accuracy for novel instances [1608.01026][2011.03243].

## 1. Motivation and Problem Formulation

Traditional OCSVM seeks a single separating hyperplane in feature space, admitting points satisfying $\langle w, \Phi(x) \rangle \geq \rho$ and rejecting those below as novel. This mechanism only bounds the lower tail. In many settings, negative samples may reside above the main mass of target scores, leading to high rates of false positives. OCSSVM defines a bounded “slab”
\[
\rho_1 \leq \langle w, \Phi(x) \rangle \leq \rho_2
\]
with the same normal $w$ and offsets $\rho_1, \rho_2$. Instances falling outside this slab are deemed outliers, directly controlling both tails of the score distribution [1608.01026][2011.03243].

## 2. Mathematical Derivation and Optimization

Given a set of $m$ positive-class training points $\{x_i\}_{i=1}^m$ mapped into a reproducing kernel Hilbert space by $\Phi(\cdot)$, OCSSVM introduces two sets of slack variables: $\xi_i \geq 0$ for lower-bound violations and $\bar{\xi}_i \geq 0$ for upper-bound violations. With control parameters $\nu_1, \nu_2 \in (0,1]$ and a weight $\varepsilon > 0$, the primal quadratic program is:
\[
\begin{aligned}
\min_{w, \rho_1, \rho_2, \xi, \bar{\xi}}\quad & \frac{1}{2}\|w\|^2 + \frac{1}{\nu_1 m} \sum_{i = 1}^m \xi_i - \rho_1 + \frac{\varepsilon}{\nu_2 m} \sum_{i = 1}^m \bar{\xi}_i + \varepsilon \rho_2 \\
\text{subject to} \quad & \langle w, \Phi(x_i) \rangle \geq \rho_1 - \xi_i, \,\,\,\, \xi_i \geq 0 \\
                        & \langle w, \Phi(x_i) \rangle \leq \rho_2 + \bar{\xi}_i, \,\,\,\, \bar{\xi}_i \geq 0,\quad \forall i
\end{aligned}
\]
Dualization leads to the convex quadratic program:
\[
\begin{aligned}
\min_{\alpha, \bar{\alpha}} \quad & \frac{1}{2}(\alpha - \bar{\alpha})^\top K (\alpha - \bar{\alpha}) \\
\text{subject to} \quad & 0 \leq \alpha_i \leq \frac{1}{\nu_1 m},\,\,\, \sum_{i = 1}^m \alpha_i = 1 \\
& 0 \leq \bar{\alpha}_i \leq \frac{\varepsilon}{\nu_2 m},\,\,\, \sum_{i = 1}^m \bar{\alpha}_i = \varepsilon
\end{aligned}
\]
where $K_{ij} = k(x_i, x_j)$ [1608.01026][2011.03243].

The model parameters are reconstructed by
\[
w = \sum_{i=1}^m (\alpha_i - \bar{\alpha}_i) \Phi(x_i)
\]
with offsets $\rho_1, \rho_2$ computed via
\[
\rho_1 = \frac{1}{N_1}\sum_{i:0<\alpha_i<1/(\nu_1 m)}\sum_{j=1}^m (\alpha_j-\bar{\alpha}_j)K_{ij}
\]
and likewise for $\rho_2$, but over support vectors with $0 < \bar{\alpha}_i < \varepsilon/(\nu_2 m)$ [1608.01026][2011.03243].

The decision score is
\[
s(x) = \langle w, \Phi(x) \rangle = \sum_{i=1}^m (\alpha_i - \bar{\alpha}_i) k(x_i, x)
\]
with classifier
\[
f(x) = \operatorname{sgn}\left( (s(x) - \rho_1)(\rho_2 - s(x)) \right)
\]
declaring $+1$ if $\rho_1 \leq s(x) \leq \rho_2$, $-1$ otherwise.

## 3. Algorithmic Solution and SMO Training

The dual convex program admits solution by any off-the-shelf quadratic programming method. Due to scalability constraints ($O(m^2)$ memory, $O(m^3)$ time for generic QP), a modified Sequential Minimal Optimization (SMO) procedure is effective [2011.03243]. By posing the dual in terms of $\gamma_i = \alpha_i - \bar{\alpha}_i$, the update at each iteration solves an analytic two-variable problem with box and slab constraints:
\[
-\frac{\varepsilon}{\nu_2 m} \leq \gamma_i \leq \frac{1}{\nu_1 m}, \qquad \sum_{i} \gamma_i = 1 - \varepsilon
\]
The SMO update takes the form:
\[
\gamma_b^{\text{new}} = \gamma_b^* + \eta \sum_{j=1}^m \gamma_j^* (k_{aj} - k_{bj}), \qquad \eta = \frac{1}{k_{aa} + k_{bb} - 2k_{ab}}
\]
along with appropriate clipping [2011.03243].

The KKT-violation-based working set selection prioritizes variables corresponding to maximal slab constraint violation. Each outer iteration involves $O(m)$ computation. Empirically, convergence is achieved in tens to hundreds of steps, with rapid subquadratic scaling in both time and memory [2011.03243].

## 4. Theoretical Properties

The OCSSVM objective is convex in $(w, \rho_1, \rho_2, \xi, \bar{\xi})$, guaranteeing a unique solution. The generalization capacity is linked to the slab's margin in the projected 1-D score space; arguments using Rademacher complexity or covering numbers extend from the OCSVM literature, with a wide slab corresponding to small $\|w\|$ and thus lower complexity for fixed empirical errors [1608.01026].

Computationally, the complexity per iteration matches OCSVM ($O(m)$); the number of variables doubles, but not the asymptotic cost. The SMO solver eliminates the infeasible $O(m^3)$ scaling for large datasets [1608.01026][2011.03243].

## 5. Empirical Evaluation and Benchmarks

OCSSVM has been evaluated on multiple datasets:

| Dataset                        | OCSVM MCC   | OCSSVM MCC | Notes                                  |
|------------------------------- | ----------- | ---------- | ----------------------------------------|
| Letter (RBF kernel)            | ≈0.07       | ≈0.39      | 26 classes, 16,000 train, 4,000 test   |
| Letter (linear)                | ≈0.02       | ≈0.14      |                                        |
| Pascal VOC 2012 (additive kern.) | 0.04–0.02   | 0.26–0.18  | 20 object classes, HOG features        |

OCSSVM consistently outperforms OCSVM and is competitive with or superior to SVDD, KPCA, and KDE where applicable. On synthetic data, SMO-OCSSVM scales from 0.35s (500 samples) to 5.91s (5,000 samples), with MCC improving with data volume (0.07 to 0.33) [1608.01026][2011.03243].

## 6. Applications and Practical Recommendations

OCSSVM is recommended in scenarios where traditional one-class novelty detection is affected by high false-positive rates from right-tail outliers, which is prevalent in open-set visual recognition and industrial anomaly detection tasks [1608.01026]. Default parameters (\(\nu_1=0.1\), \(\nu_2=0.01\), \(\varepsilon=2/3\)) are effective starting points, with additional tuning via class-restricted validation. RBF kernels are a robust default, while histogram features benefit from intersection, Hellinger, or $\chi^2$ kernels. Data sizes below 2,000 permit primal-dual QP methods; larger sets necessitate SMO [1608.01026][2011.03243].

## 7. Implementation Outline

OCSSVM is constructed by assembling the kernel matrix $K$, solving the dual for variables $\alpha_i$, $\bar{\alpha}_i$ (or $\gamma_i$), extracting the primal weight $w$ and offsets $\rho_1$, $\rho_2$ via KKT, and applying the two-hyperplane decision. The SMO solver is directly adaptable from standard OCSVM literature with the box-sum modifications above [1608.01026][2011.03243].

Source: https://www.emergentmind.com/topics/one-class-slab-svm-ocssvm