---
title: Ellipsoidal Separation Machine
url: https://www.emergentmind.com/topics/ellipsoidal-separation-machine-esm
type: topic
---

# Ellipsoidal Separation Machine

Searching arXiv for the named topic and closely related usages of “ESM”.
The **Ellipsoidal Separation Machine (ESM)** is a binary classifier derived from **Separation by a Convex Body (SCB)**: instead of separating two classes with a hyperplane, it separates them with an **ellipsoid**. In its canonical formulation, the separator is a bounded convex set chosen so that the segment joining a positive point and a negative point intersects the ellipsoid. The training problem is structurally similar to the Support Vector Machine (SVM) one, is convex, and can be formulated as a **Semidefinite Program (SDP)**; the practical contribution is a full implementation that replaces the intractable large-scale SDP with smaller optimization subproblems and approximate solvers. A defining property of ESM is that it necessarily creates **indeterminate points**, making it particularly suited to **Classification with Rejection (CwR)** [2507.20698].

## 1. Separation by a convex body and the emergence of ESM

ESM instantiates the SCB principle with an ellipsoid. In SCB, a set \(\mathcal S\) separates a positive point \(x^+\) and a negative point \(x^-\) if
\[
\operatorname{conv}(\{x^+,x^-\}) \cap \mathcal S \neq \emptyset.
\]
For two sets \(X^+\) and \(X^-\), the SCB requirement is imposed for all cross-class pairs. ESM chooses \(\mathcal S\) to be an ellipsoid because ellipsoids admit semidefinite representations and therefore connect the geometry of bounded convex separators to tractable convex optimization [2507.20698].

The resulting construction differs fundamentally from hyperplane-based classification. A hyperplane extends unboundedly and assigns every point to one side or the other. An ellipsoid is bounded and local. This changes the semantics of classification: the ellipsoid is not merely a decision surface, but a geometric object that certifies cross-class separation through segment intersection. The 2025 formulation is presented as the first fully functional implementation of the SCB approach for classification with an ellipsoid, thereby turning an earlier convex-geometric idea into a workable training and inference pipeline [2507.20698].

## 2. Ellipsoidal geometry, scores, and the decision rule

The separator is an ellipsoid
\[
\mathcal S=\{x\in\mathbb R^n:(x-\gamma)^\top S(x-\gamma)\le 1\},
\]
with \(S\succeq 0\) symmetric positive semidefinite and \(\gamma\in\mathbb R^n\) the center. The eigenstructure of \(S\) determines the axes and shape of the ellipsoid. In the training objective, smaller ellipsoids are preferred, but the ellipsoid must still separate the two classes in the SCB sense [2507.20698].

A distinctive aspect of ESM is that points inside the ellipsoid are removed from the reliable reference sets:
\[
\bar X^+ = X^+ \setminus \mathcal S,\qquad \bar X^- = X^- \setminus \mathcal S.
\]
For an unlabeled point \(z\), ESM counts how many segments from \(z\) to training points of each class intersect the ellipsoid:
\[
n^+(z)=\left|\left\{x^+\in\bar X^+:\mathcal S\cap[z,x^+]\neq\emptyset\right\}\right|,
\]
\[
n^-(z)=\left|\left\{x^-\in\bar X^-:\mathcal S\cap[z,x^-]\neq\emptyset\right\}\right|.
\]
These counts define normalized scores
\[
r^+(z)=\frac{n^-(z)}{|\bar X^-|}-\frac{n^+(z)}{|\bar X^+|},\qquad
r^-(z)=\frac{n^+(z)}{|\bar X^+|}-\frac{n^-(z)}{|\bar X^-|},
\]
with \(r^+(z)+r^-(z)=0\). The classification rule is thresholded by \(\varepsilon\in(0,1]\): if \(r^+(z)\ge\varepsilon\), assign \(X^+\); if \(r^-(z)\ge\varepsilon\), assign \(X^-\); otherwise the point is **rejected** or declared **indeterminate** [2507.20698].

This decision rule yields a geometry that differs sharply from SVM. In ESM, indeterminacy is not just a narrow strip around a local boundary. It can include points inside \(\mathcal S\), points very far from both classes, and points similarly separated from both classes. In the formulation described for ESM, a point inside the ellipsoid typically satisfies \(r^+(z)=r^-(z)=0\), which directly exhibits the abstention mechanism [2507.20698].

## 3. Training problem, convex reformulations, and scalable optimization

The naive training model minimizes an ellipsoid size functional under the requirement that every cross-class segment intersects the ellipsoid. With \(P=I^+\times I^-\) and interpolation points
\[
x_{ij}=\alpha_{ij}x_i^+ + (1-\alpha_{ij})x_j^- ,\qquad \alpha_{ij}\in[0,1],
\]
one writes
\[
\min\; \operatorname{size}(S)
\]
subject to
\[
(x_{ij}-\gamma)^\top S(x_{ij}-\gamma)\le 1,\qquad (i,j)\in P.
\]
To handle nonseparable data, ESM introduces slack variables and regularization:
\[
\min\; \operatorname{size}(S) + C_1\sum_{(i,j)\in P}\beta_{ij}
+ C_2\Big(\sum_{i\in I^+}\nu_i^+ + \sum_{j\in I^-}\nu_j^-\Big)
+ C_3\|S\|_F^2,
\]
where \(\beta_{ij}\) penalizes violated cross-segment intersection constraints and \(\nu_i^+,\nu_j^-\) penalize training points that fall inside the ellipsoid. The size term is written either as \(\log\det(S^{-1})\) or, in an auxiliary SDP representation, as \(\operatorname{tr}(T)\) [2507.20698].

The central technical difficulty is bilinearity: \(\gamma\) couples with \(S\), and the points \(x_{ij}\) couple with \(S\) because they depend on \(\alpha_{ij}\). The paper resolves this through a **homogeneous ellipsoid reformulation** in \(\mathbb R^{n+1}\),
\[
\tilde S=\begin{pmatrix} s & t^\top \\ t & F \end{pmatrix},
\]
with normalization \(s=1\), so that ellipsoid membership is encoded by
\[
(1,x)^\top \tilde S (1,x)\le 1.
\]
This removes the explicit center \(\gamma\) and enables a convex SDP reformulation via auxiliary variables and Schur-complement-type constraints [2507.20698].

The full convex SDP remains expensive because there is one constraint for each pair \((i,j)\in X^+\times X^-\), the matrix variables live in dimension \(n+1\), and volume-related terms such as \(\log\det\) add complexity. The practical solver therefore uses a nonconvex reformulation amenable to a **Block-Gauss-Seidel** approach alternating between a much smaller SDP and a simple separable **Second-Order Cone Program (SOCP)**. Even the reduced SDP is further relaxed in a Lagrangian way, and the multipliers are updated by fast subgradient-type approaches. In the classification setting, this sacrifices exact optimality for speed while preserving the basic ellipsoidal-separation structure [2507.20698].

## 4. Classification with rejection and the role of indeterminate points

The reject option is not an auxiliary add-on to ESM; it is intrinsic to the geometry of the model. The ellipsoid defines a bounded separator, and the score construction only yields a class assignment when the evidence is sufficiently asymmetric. Hence ESM necessarily defines **indeterminate points**, namely points that cannot be reliably classified as belonging to either set [2507.20698].

This makes ESM particularly suitable for **Classification with Rejection**. In margin-based SVM rejection, abstention is usually implemented by rejecting points close to the separating hyperplane. ESM behaves differently. Its rejection is geometric and global: a point may be rejected because it lies inside the ellipsoid, because it is similarly separated from both classes, or because it is far from the data region and thus not convincingly associated with either side. This suggests a notion of abstention tied to **insufficient evidence** rather than merely to **small local margin** [2507.20698].

The practical significance is application-dependent. The formulation identifies ESM as especially attractive when false positives are costly, abstention is valuable, outlier detection matters, or a bounded separator is more appropriate than a hyperplane. The same geometry that yields indeterminacy also differentiates ESM from reparameterized SVM variants: the paper explicitly emphasizes that ESM’s rejection zone behaves quite differently from SVM margins [2507.20698].

## 5. Empirical behavior and comparison with SVM

The reported evaluation uses both synthetic and real binary datasets. The synthetic experiments are generated with `make_classification`. The real datasets are taken from the LIBSVM repository: **Australian**, **Breast**, **Diabetes**, **Fertility**, **Heart**, **Ionosphere**, and **Liver**. Performance is reported with the metrics **AR** (accuracy with rejection), **A** (accuracy on classified points), **WC** (percentage well classified), **MC** (percentage misclassified), and **R** (rejection rate). The AR score is
\[
\frac{|X_{wc}|-\xi|X_{rej}|}{|X|},
\]
with rejection cost \(\xi\in[0,0.5)\) [2507.20698].

The baseline comparison is against SVM with **polynomial**, **Gaussian/RBF**, and **sigmoid** kernels, where rejected points are those inside the SVM margin. The empirical conclusions are stated qualitatively rather than as a single universal ranking. ESM is generally **competitive** with the best SVM variant. On synthetic datasets, it often behaves similarly to sigmoid SVM and tends to reject fewer points than polynomial or Gaussian SVM. On the real datasets, it often trades blows with the best kernel choice and sometimes achieves the best AR. The paper therefore presents ESM as a genuinely different model rather than a mere reformulation of existing margin-based classifiers [2507.20698].

The same experiments also expose the main limitations. Training is more complex than SVM, the full convex formulation does not scale well, and practical use requires hyperparameter tuning over several constants. The classification rule is also more elaborate than the sign of a hyperplane. These limitations are structural consequences of replacing a linear separator with a bounded convex body and of insisting on a principled abstention region [2507.20698].

## 6. Related uses of ellipsoidal separation and ambiguity of the acronym

The term **ESM** is not unique to the classifier defined in 2025. In adjacent literatures, “separator,” “ellipse,” “ellipsoid,” and even the acronym “ESM” are used in technically distinct senses.

| Usage | Domain | Distinction |
|---|---|---|
| **Ellipsoidal Separation Machine** | Classification | Ellipsoid-based SCB classifier with indeterminate points [2507.20698] |
| **Optimal separator for an ellipse; Application to localization** | Interval analysis | Minimal contractor and minimal separator for a planar ellipse, not a machine-learning separator [2305.10842] |
| **ESM data** | Surveillance | “ESM” means Electronic Support Measures, not ellipsoidal classification [1607.01355] |

The interval-analysis work on planar ellipses develops a **minimal contractor** and **minimal separator** for an ellipse in the plane, exploits the **hyperoctahedral group** of symmetries, and applies the construction to guaranteed localization with multiple sonars. Its separator is a set-valued operator yielding inner and outer approximations of a feasible set; it is explicitly not a machine-learning separator and is firmly in validated interval set inversion [2305.10842].

A different but conceptually adjacent line appears in nonlinear predictive control for obstacle avoidance, where vehicle and obstacles are modeled as **convex superellipsoids** and collision avoidance is encoded through the **separating hyperplane theorem**, support functions, and a separating axis constrained inside an NMPC problem. This is naturally viewed as an **ESM-style separation problem embedded in predictive control**, but it is a control formulation rather than a classifier [2404.14257].

Further afield, an elliptically wound microfluidic device has been described as **ESM-like** because a periodically varying curvature drives bifurcations in inertial particle equilibria, producing **longitudinal clustering** and size-based separation along the duct [2606.12894]. A plausible conceptual relation also exists to HYPERION, a transmitted-THz blind source-separation method that uses the **Löwner–John ellipsoid** as a geometric preconditioner so that source signatures become vertices of a regular simplex in the preconditioned space [2109.05425]. These neighboring usages share ellipsoidal geometry, but they are not the same object as the Ellipsoidal Separation Machine in binary classification.

In the strict sense established by the classification paper, the Ellipsoidal Separation Machine is therefore a specific SCB-based model: an ellipsoid is optimized so that opposite-class segments intersect it, prediction is made through segment-intersection counts, and rejection is intrinsic rather than heuristic [2507.20698].

Source: https://www.emergentmind.com/topics/ellipsoidal-separation-machine-esm