Calibrated OC-SVM Techniques

• Calibrated OC-SVM is a method that decouples offset calibration from SVM optimization to precisely capture high-density regions.
• It aggregates outputs from multiple train-test splits to reduce variance and reliably estimate minimum volume sets.
• The approach employs unsupervised hyperparameter tuning, balancing kernel bandwidth selection with effective change-point detection in time series.

A calibrated One-Class Support Vector Machine (OC-SVM) refers to a collection of methods and hyperparameter tuning strategies designed to address key limitations of the classic OC-SVM framework, specifically its sensitivity to kernel and offset selection, and its lack of unsupervised calibration for controlling false-positive rate or detecting change-points in time series data. Calibrated OC-SVMs introduce procedures such as offset adjustment, aggregation over multiple data splits, and meta-parameter optimization via black-box heuristic search methods to achieve robust minimum volume set (MV-set) estimation, anomaly detection, or change-point localization with improved reliability over standard OC-SVMs (Thomas et al., 2015, Jin et al., 2019).

1. Classical OC-SVM Formulation and Limitations

The OC-SVM algorithm implements unsupervised anomaly detection or high-density region estimation by seeking a function in a reproducing kernel Hilbert space (RKHS) such that most data points lie on one side of a decision boundary. For i.i.d. samples $\{X_i\}_{i=1}^n\subset \mathbb{R}^d$ and with feature map $\Phi$ (kernel $k_\sigma$), the primal and dual optimization problems for the $\nu$-formulation are:

• Primal:

$$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$

subject to $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$, $\xi_i\geq 0$, $\nu \in (0,1)$.

• Dual:

$$\min_{\gamma\in\mathbb{R}^n} \ \tfrac{1}{2} \sum_{i,j=1}^n \gamma_i\gamma_j k(X_i,X_j)$$

subject to $0 \leq \gamma_i \leq \frac{1}{\nu n}$, $\Phi$0.

The data region $\Phi$1 approximates a minimum-volume set of the nominal class, but the selection of kernel width $\Phi$2 (or $\Phi$3), offset $\Phi$4, and the outlier parameter $\Phi$5 are highly sensitive and difficult to tune in unsupervised settings. In practice, this can lead to poor estimation of high-density regions or change-points, and overfitting to noise or to the finite-sample idiosyncrasies of the training set (Thomas et al., 2015, Jin et al., 2019).

2. Offset Calibration via Empirical Mass Control

Classical OC-SVMs attempt to set $\Phi$6 to force at least an $\Phi$7 fraction of data inside $\Phi$8. However, for finite $\Phi$9, this can mis-estimate the actual coverage of the high-density region. The calibrated OC-SVM method decouples the offset calibration from the SVM optimization:

• Fix a generous $k_\sigma$0 (e.g., $k_\sigma$1) to allow learning of the density tail.
• Randomly partition the data into $k_\sigma$2.
• Train OC-SVM on $k_\sigma$3 yielding $k_\sigma$4.
• Compute the empirical $k_\sigma$5-quantile of scores $k_\sigma$6 on $k_\sigma$7 and set $k_\sigma$8 such that

$k_\sigma$9

The set $\nu$0 is then empirically calibrated to include fraction $\nu$1 of the test data.

This procedure enables direct control of the mass of the estimated set and allows post hoc construction of nested sets for different target masses $\nu$2 from the same trained $\nu$3 (Thomas et al., 2015).

3. Aggregation and Variance Reduction over Splits

Variance in the estimated offset due to random train/test splits can be considerable. Aggregation is performed as follows:

• Repeat the above train/calibrate procedure for $\nu$4 independent random splits.
• For each split $\nu$5, get $\nu$6 and $\nu$7.
• Define the averaged scoring function:

$\nu$8

• The final estimated set is $\nu$9.

This aggregation yields more stable set estimates, reduces splitting-induced variance, and, coupled with different $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$0-offsets, ensures the nestedness property: $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$1 for $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$2 (Thomas et al., 2015).

4. Bandwidth Selection and Mass–Volume Tradeoff

Hyperparameter tuning in calibrated OC-SVM is performed unsupervisedly by minimizing the area under the Mass–Volume (AMV) curve:

• For a kernel grid $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$3, and quantile window $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$4, the volume of $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$5 (for $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$6) is estimated via Monte Carlo.
• The AMV for each bandwidth is

$$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$7

where $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$8 is typically the Lebesgue measure in the data space.

• The optimal bandwidth is chosen as $$\min_{w,\rho,\xi}\ \frac{1}{2}\|w\|^2 - \rho + \frac{1}{\nu n} \sum_{i=1}^n \xi_i;$$9.

This approach balances the inclusion of high-density data (mass) against restricting the region to minimal volume, thereby mitigating both over-smoothing and under-smoothing (Thomas et al., 2015).

5. Calibration for Time Series Change Point Detection

For time series scenarios where the location of regime change (change-point) is unknown and only implicitly labeled, a distinct calibration procedure is employed:

• For each series $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$0, parameterize the healthy fraction $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$1, and treat these fractions, along with kernel width $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$2, as optimization variables.
• For each hypothesis $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$3, label training data up to $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$4 as normal and the rest as outlier.
• Train the OC-SVM using only the hypothesized normal data.
• Use a loss function (e.g., log-loss or 0–1 loss based on prediction-calibrated probabilities) measuring consistency between hypothesized change points and OC-SVM predictions.
• Minimize this loss over hypotheses via Differential Evolution (DE), using population-based random search within defined hyperparameter bounds.

On NASA C-MAPSS FD004, this approach achieves competitive change-point detection with as few as $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$5 calibration series, demonstrating significant data efficiency relative to deep learning-based methods that require orders of magnitude more training samples (Jin et al., 2019).

6. Empirical Findings and Evaluation Metrics

Experiments on both synthetic and real datasets demonstrate:

• Calibrated OC-SVM, with aggregation and bandwidth selection, exhibits superior coverage/volume efficiency as measured by symmetric difference to ground-truth MV-sets and AMV curves.
• The method is robust to the presence of uniform outliers and performs competitively in moderate to high dimensions compared to kernel density estimators (KDEs).
• In time series change-point detection, calibrated OC-SVMs approach or match performance of WTTE-RNN on real engine data while using substantially less training data.

Metrics used for evaluation include:

• Lebesgue measure of symmetric difference $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$6,
• Area under the Mass–Volume curve (AMV),
• For sequences: detection delay, false alarm rate, miss rate—where change-points are identified as the time of first label transition (Thomas et al., 2015, Jin et al., 2019).

7. Practical Guidelines and Implementation Considerations

Best practices for calibrated OC-SVMs include:

• Set $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$7 at a “large” value (often $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$8) to capture the density’s tail structure for MV set estimation.
• Use $\langle w, \Phi(X_i)\rangle \geq \rho-\xi_i$9–$\xi_i\geq 0$0 random train/test splits and aggregate calibrated offsets.
• Choose kernel bandwidth grids (or $\xi_i\geq 0$1) logarithmically spaced to span effective data diameter.
• For change-point detection, perform hyperparameter optimization (e.g., DE) over kernel width and per-series healthy fractions within domain-informed bounds.
• In time series cases, limit the search range for $\xi_i\geq 0$2 away from 0 and 1 if prior knowledge exists about healthy/faulty durations.
• Volume estimation by Monte Carlo remains the main computational bottleneck in high dimensions.
• Use aggregation and offset-calibration also for other kernels beyond Gaussian if desired.

Extensions include adapting the loss function for specific downstream metrics if real labels are available, and jointly optimizing over $\xi_i\geq 0$3 if not fixed. If access to more labeled data is possible, increasing $\xi_i\geq 0$4 improves calibration. With fewer samples, stronger priors on hyperparameter search ranges are advised (Thomas et al., 2015, Jin et al., 2019).