SMOTE-NC: Oversampling Mixed Data

• SMOTE-NC is an oversampling technique that generates synthetic minority samples for datasets containing both continuous and categorical features.
• It interpolates continuous attributes using linear combinations and assigns categorical features by majority voting among k-nearest neighbors.
• Although widely used for rare-event detection, its limitations in preserving feature coherence have led to the development of advanced methods like MGS-GRF and SMOTE-ENC.

SMOTE-NC (Synthetic Minority Over-sampling Technique for Nominal and Continuous features) is an extension of the SMOTE family of imbalanced classification algorithms, specifically designed to generate synthetic samples in datasets containing both continuous and categorical variables. It performs data-level augmentation by interpolating within the feature space of the minority class while accounting for the presence of mixed data types. SMOTE-NC remains a default solution for rare-event detection in tabular binary classification where features are heterogeneous, but its limitations have spurred further developments in mixed-feature oversampling (Sakho et al., 26 Mar 2025, Mukherjee et al., 2021).

1. Algorithmic Foundations and Structure

SMOTE-NC operates on a set of minority-class instances defined as $X = (X_{1:d}, X_{d+1:d+p})$, where $X_{1:d}$ are real-valued (continuous) features and $X_{d+1:d+p}$ are categorical. The algorithm extends the interpolation approach of the original SMOTE for datasets with mixed features, using the following core steps (Sakho et al., 26 Mar 2025, Mukherjee et al., 2021):

1. Neighbor Identification (Mixed Distance): Compute the $K$ nearest neighbors of each minority sample using a composite metric. For two points $X$, $X'$:

$$L_{\rm NC}(X, X') = \sqrt{ \sum_{j=1}^d (X_j'-X_j)^2 + C^2 \sum_{j=d+1}^{d+p} \mathbf{1}_{\{X_j \neq X_j'\}} }$$

where $C$ is the median standard deviation of continuous features among the minority class.

2. Continuous Feature Synthesis: Generate continuous coordinates by linear interpolation between a seed sample and one of its $K$ nearest neighbors:

$$Z_{1:d} = X^c_{1:d} + w (X^k_{1:d} - X^c_{1:d}), \qquad w \sim \mathcal{U}(0,1)$$

3. Categorical Feature Synthesis: For each categorical feature $X_{1:d}$0, determine its value in the synthetic sample via majority-vote among the $X_{1:d}$1 neighbors:

$X_{1:d}$2

A single SMOTE-NC iteration to produce one synthetic sample follows:

1. Uniformly select a seed minority sample.
2. Identify $X_{1:d}$3 nearest neighbors under $X_{1:d}$4.
3. Interpolate as above for continuous features.
4. Assign categorical features by mode voting among those $X_{1:d}$5 neighbors.
5. Concatenate and output the synthetic instance.

2. Theoretical Relationship to SMOTE

While the original SMOTE algorithm is defined only for continuous data using Euclidean distance and linear interpolation, SMOTE-NC modifies both the neighbor selection and interpolation logic for mixed data. The essential distinctions:

• Distance Metric: SMOTE applies the $X_{1:d}$6 norm; SMOTE-NC uses $X_{1:d}$7 which penalizes categorical mismatches by $X_{1:d}$8.
• Categorical Assignment: Vanilla SMOTE performs no operation; SMOTE-NC assigns categories via neighbor majority-voting.

The generator for continuous attributes is unchanged, but the adaptation for categoricals ensures that hybrid tabular datasets can be oversampled without loss of type integrity (Sakho et al., 26 Mar 2025).

3. Limitations and Assumptions

Empirical and theoretical analysis reveals several notable limitations in SMOTE-NC's approach (Sakho et al., 26 Mar 2025, Mukherjee et al., 2021):

• Non-Coherence:

SMOTE-NC votes independently on each categorical coordinate, which can result in synthetic category-tuples that do not occur in the original minority class set. Formally, the probability that the generated tuple $X_{1:d}$9 does not belong to the observed set $X_{d+1:d+p}$0 is strictly positive, i.e., $X_{d+1:d+p}$1.

• Broken Association:

The dependence between continuous and categorical features can be disrupted, as the categorical assignment relies only on marginal majority statistics, potentially destroying the joint law of $X_{d+1:d+p}$2. In high dimensions, $X_{d+1:d+p}$3-nearest neighbor voting accuracy degrades, rendering the categorical generator nearly random when many uninformative or noisy features exist.

• Uniform Penalty for Categories:

All categorical mismatches are treated equally, disregarding the statistical association between specific levels and the minority class. This undermines the potential to exploit nuanced categorical distributions.

• Requirement for Continuous Features:

The definition of $X_{d+1:d+p}$4 (median minority-class standard deviation) is undefined if the data contains only categorical features. Hence, SMOTE-NC cannot operate where all features are nominal (Mukherjee et al., 2021).

• Failure to Exploit Weak Categorical Signals:

Categorical levels with subtle minority-class affinities are not differentiated, as SMOTE-NC penalizes only presence/absence of matches rather than strength of association.

4. Empirical Evaluation

Quantitative performance of SMOTE-NC has been evaluated on a range of public and private datasets, commonly in combination with a LightGBM or random forest classifier, and assessed using ROC AUC and PR AUC metrics (Sakho et al., 26 Mar 2025, Mukherjee et al., 2021). Representative results follow:

Dataset	Strategy	PR AUC	ROC AUC	Coherence
Bank (Churn)	None	0.622	0.977	100%
	SMOTE-NC	0.595	0.975	90%
	MGS-GRF	0.664	0.984	100%

• On "BankChurners" (imbalance 1%), SMOTE-NC decreases PR AUC compared to the un-augmented dataset, and only 90% of synthetic category-tuples are coherent.
• On "BankMarketing", SMOTE-NC reduces PR AUC by 17% relative to none.
• On a private banking dataset, SMOTE-NC produces 28% lower PR AUC than no augmentation.
• The coherence of categorical tuples generated by SMOTE-NC is consistently less than 100%, and conditional association with continuous features deteriorates rapidly as categorical dimensionality increases (Sakho et al., 26 Mar 2025).

5. Comparative Analysis with Recent Methods

The introduction of more advanced mixed-feature oversampling techniques, notably MGS-GRF ("Harnessing Mixed Features for Imbalance Data Oversampling: Application to Bank Customers Scoring") (Sakho et al., 26 Mar 2025) and SMOTE-ENC (Mukherjee et al., 2021), refines the limitations of SMOTE-NC.

• MGS-GRF employs kernel density estimators (KDE) with local covariance for continuous features, and samples categorical tuples using generalized random forests. Every generated categorical tuple is drawn from the original set, ensuring coherence, and the model-based sampler preserves the joint structure between continuous and categorical features (strong association).
• SMOTE-ENC numerically encodes nominal labels proportional to their association with minority class membership and uses this encoding in both the distance metric and the interpolation scheme, enabling operations on all-categorical datasets and accounting for variable label importance.

Empirical findings indicate that both MGS-GRF and SMOTE-ENC surpass SMOTE-NC in maintaining statistical fidelity. For example, MGS-GRF achieves PR AUC of 0.664 (vs. 0.595 for SMOTE-NC) while maintaining 100% coherence and association, and SMOTE-ENC yields consistently higher F1, ROC-AUC, and PR-AUC on datasets with numerous or predictive nominal attributes (Sakho et al., 26 Mar 2025, Mukherjee et al., 2021).

6. Context, Usage, and Prospects

SMOTE-NC remains widely used as a default data-level solution for mixed-type imbalanced tables due to its straightforward extension of SMOTE, but practitioners in finance, banking, and other domains with complex mixed data should be aware of its structural deficiencies. Development of coherent and association-preserving generators represents a critical advancement in oversampling. A plausible implication is that, for high-dimensional or heavily categorical datasets, failure to enforce tuple coherence or preserve conditional relationships may seriously undermine classifier performance and regulatory acceptability. Recent evidence argues that advanced frameworks like MGS-GRF or categorical encoding strategies (SMOTE-ENC) offer both theoretical and practical superiority in high-stakes applications (Sakho et al., 26 Mar 2025, Mukherjee et al., 2021).