SMOTE-N: Oversampling for Mixed Imbalanced Data

• SMOTE-N is an extension of SMOTE that generates synthetic minority data through convex combinations, addressing imbalances in datasets with both continuous and nominal features.
• It employs k-nearest neighbor methods using Euclidean and specialized nominal distance metrics to interpolate between minority instances, ensuring valid synthetic sample creation.
• Empirical results show that SMOTE-N enhances classifier performance with improved ROC curves and AUC by up to 10–15% compared to traditional under-sampling and cost-sensitive methods.

SMOTE-N is an extension of the Synthetic Minority Over-sampling TEchnique (SMOTE), designed to address the challenges of imbalanced binary classification, particularly when feature spaces include both continuous and nominal (categorical) variables. SMOTE-N synthesizes minority class data by interpolating new examples in feature space, expanding the minority manifold beyond simply duplicating minority samples. Its operation—like that of canonical SMOTE—is founded on constructing synthetic samples between existing minority instances using convex combinations. The "N" suffix denotes adaptations for mixed or purely nominal feature spaces, although the fundamental methodology is tied to the general SMOTE framework and its parametrization (Chawla et al., 2011).

1. Framework for Imbalanced Data and Motivation

Many real-world classification problems present substantial class imbalance: one class ("majority") dominates, while the other ("minority") is severely under-represented. Standard classifier learning can yield poor generalization for the minority class, particularly when the cost of misclassification is asymmetric. To ameliorate this, data-level strategies are employed:

• Over-sampling the minority class by synthetic instance generation (as SMOTE/SMOTE-N)
• Under-sampling the majority class by random deletion

SMOTE-N integrates these principles, seeking to extend the minority class population in a way that prompts classifiers to learn broader, less specific decision regions for under-represented classes, especially in datasets with both continuous and nominal attributes (Chawla et al., 2011).

2. Nearest-Neighbor Construction in Mixed Feature Space

SMOTE-N identifies the $k$-nearest neighbors for each minority class instance using feature-space distance. For continuous variables, Euclidean distance is used:

$$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$

For nominal features, an appropriate generalized distance metric is adopted to meaningfully couple nominal similarity with continuous distances (the extension for SMOTE-N is to handle these mixed types). The $k$ neighbors establish the local neighborhood for sampling.

The value of $k$ controls the locality of synthetic sample creation:

• Larger $k$ introduces greater global diversity and broader decision regions
• Smaller $k$ confines synthetic instances to local clusters, suitable for sparser manifolds

3. Synthetic Minority Example Generation

SMOTE-N procedurally generates synthetic minority points as follows:

• For each minority instance $x_i$, randomly select one of its $k$ nearest neighbors, denoted $x_{nn}$.
• For continuous features, construct a synthetic point by linear interpolation via a random $\delta$ drawn from the uniform interval $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$0:

$$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$1

• For nominal attributes (critical in SMOTE-N), synthetic values are often assigned based on majority voting among nearest neighbors or other suitable categorical sampling—this step ensures generated samples are valid with respect to the intrinsic attribute semantics.

This approach “fills in” the interstitial space between minority examples, supporting learning of broader minority regions without overfitting.

4. Parameterization and Effects on Sample Distribution

The two primary control parameters are:

• $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$2 (number of nearest neighbors): Larger $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$3 encourages exploration of more diverse regions of minority feature space, promoting global coverage but risking inter-class leakage for large $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$4. Smaller $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$5 preserves local structure at the expense of coverage.
• $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$6-distribution: Uniform sampling in $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$7 leads to homogeneous spread along interpolated lines. Biasing $$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$8 toward endpoints clusters synthetic samples near existing points or their neighbors, while bias toward the midpoint ($$\text{dist}(x_i, x_j) = \|x_i - x_j\|_2 = \sqrt{\sum_{f=1}^d (x_i[f] - x_j[f])^2}$$9) centers the synthetic generation.

Parameter selection thus mediates the “shape” and “spread” of synthetic points, trading off local vs. global exploration and the aggressiveness of minority class expansion into majority territory (Chawla et al., 2011).

5. Combined Over-Sampling and Under-Sampling Protocol

The augmented learning protocol incorporating SMOTE-N consists of:

• Over-sampling the minority class (at rates from 50% to 500%, i.e., $k$0 to $k$1 its original size).
• Under-sampling the majority class by random instance deletion to varying fractions, allowing control over the minority-to-majority size ratio.

Classifier training (using C4.5, Ripper, Naive Bayes) is then performed on the composite, resampled set. Repeating this process for distinct over-/under-sampling ratios yields an operating frontier in ROC space.

6. Evaluation and Empirical Performance Implications

Performance is quantified using:

• 10-fold cross-validation
• Reporting of $k$2TruePositive (sensitivity) and $k$3FalsePositive (1–specificity)
• ROC curves, Area Under the Curve (AUC), and the ROC convex hull

Key empirical findings include:

• SMOTE-augmented over-/under-sampling results in ROC curves that are consistently higher and farther left than those from plain under-sampling or cost-sensitive classifier parameterization.
• A greater proportion of SMOTE-induced classifier operating points fall on the ROC convex hull, indicating superior or Pareto-dominant tradeoffs between sensitivity and specificity over a range of cost/prior settings.
• Over-sampling by simple replication fails to generalize, yielding overly specific, overfitted minority regions, while SMOTE-N’s synthetic interpolation produces broader, more robust minority class coverage.
• Across diverse benchmarks—mammography, oil-spill detection, forest cover, Pima diabetes, phoneme, and adult income datasets—SMOTE-style sampling achieved AUC improvements of up to 10–15% compared to plain under-sampling and generally outperformed cost-tuning alternatives (Chawla et al., 2011).

7. Extensions and Relation to Broader Research

SMOTE-N’s foundational principle—convex combination of minority examples—supports adaptation to mixed continuous/nominal attributes and forms the basis for numerous variant methodologies, including application to domain-specific tasks (e.g., text retrieval, SMOTE-NB hybrids).

A typical taxonomy related to SMOTE-N in terms of resampling methods:

Approach	Candidate Features	Synthetic Minority Generation
SMOTE	Continuous	Linear interpolation
SMOTE-N	Mixed (Continuous/Nominal)	Interpolation + Nominal assignment
SMOTE-NB Hybrid	Application-specific	SMOTE + domain-specific heuristics

A plausible implication is that SMOTE-N’s mixed-type sampling supports broader applicability, particularly where nominal features are predominant, and can be further tailored for domain-specific requirements. The approach is identified as a practical foundation for classifier training under severe class imbalance (Chawla et al., 2011).