SEMRes-DDPM: Tabular Minority Oversampling

• The paper presents SEMRes-DDPM, which replaces a standard reverse-process MLP with the SEMST-ResNet to generate high-fidelity minority-class samples.
• Its architecture leverages residual connections, multi-head self-attention, and learned soft-thresholding to enhance denoising and capture complex tabular feature interactions.
• Empirical results on 20 datasets show significant improvements in F1, G-mean, and AUC compared to SMOTE, GAN-based methods, and simple DDPM approaches.

SEMRes-DDPM is a denoising diffusion probabilistic model for minority-class oversampling in imbalanced tabular classification. It was introduced as a response to three limitations identified in prior oversampling practice: SMOTE-type methods rely on local interpolation and may generate samples that are not realistic enough; GAN-based oversamplers can capture the true distribution of data but are affected by pattern collapse and training instability; and DDPM-based tabular generators that replace image-oriented U-Nets with simple MLP denoisers can suffer from poor noise removal. The defining feature of SEMRes-DDPM is the replacement of the reverse-process MLP with a residual tabular denoiser, SEMST-ResNet, which combines residual connections, multi-head self-attention, and a learned soft-thresholding mechanism (Zheng et al., 2024).

1. Problem setting and rationale

SEMRes-DDPM is formulated for tabular datasets in which the minority class has far fewer samples than the majority class. In this setting, standard classifiers tend to bias toward the majority class, which can yield poor minority recall, F1, and decision boundaries. The method therefore operates as a data-level intervention: it trains a generative model only on minority-class rows and then synthesizes additional minority samples until a more balanced training distribution is obtained (Zheng et al., 2024).

The method is positioned against several families of oversamplers. SMOTE, Borderline-SMOTE, ADASYN, GDO, RBO, and OREM are described as local or statistical schemes that interpolate minority samples in feature space. Their limitation is not merely computational simplicity, but the fact that they primarily exploit local neighborhoods and can ignore the global geometry of the minority distribution. In multimodal or constrained tabular manifolds, this can place synthetic points outside realistic high-density regions. GAN-based oversamplers such as CGAN and CWGAN-GP address global distributional modeling more directly, but adversarial optimization introduces training instability and mode collapse. TabDDPM avoids adversarial training, but its MLP denoiser is treated as insufficiently expressive for robust tabular denoising, especially after long reverse chains (Zheng et al., 2024).

A common misconception is that SEMRes-DDPM is primarily a segmentation model because of its name. In fact, the model introduced under that name is a tabular minority oversampler. Its central objective is not pixel-wise prediction, but synthetic sample generation for class rebalancing.

2. Diffusion formulation for minority-class tabular generation

The method begins with tabular preprocessing. Categorical features are one-hot encoded, numerical features are min-max normalized to $[0,1]$, and the normalized rows are denoted by $S^{\text{norm}}$. The original diffusion variable is

$S_0 = S^{\text{norm}},$

with $q(S_0)$ the empirical data distribution. Training is performed only on minority-class samples (Zheng et al., 2024).

The forward process follows the standard DDPM Gaussian noising chain. For $t=1,\dots,T$,

$$q(S_t \mid S_{t-1}) = \mathcal{N}\big(S_t; \sqrt{1 - \beta_t} \, S_{t-1}, \, \beta_t I \big),$$

with $\alpha_t = 1 - \beta_t$ and $\bar\alpha_t = \prod_{i=1}^t \alpha_i$. The corresponding marginal is

$$q(S_t \mid S_0) = \mathcal{N}\big(S_t; \sqrt{\bar\alpha_t} S_0,\,(1-\bar\alpha_t) I \big).$$

As $T$ becomes large and $S^{\text{norm}}$0, $S^{\text{norm}}$1 approaches a standard Gaussian (Zheng et al., 2024).

The reverse process is parameterized as

$S^{\text{norm}}$2

where the denoising network predicts the injected Gaussian noise. Using the usual reparameterization

$S^{\text{norm}}$3

training minimizes the simple DDPM objective

$S^{\text{norm}}$4

The method is therefore a likelihood-style diffusion generator for minority-class rows rather than a discriminative imbalance correction scheme.

At generation time, SEMRes-DDPM samples

$S^{\text{norm}}$5

runs reverse diffusion to obtain $S^{\text{norm}}$6, denormalizes the result, converts one-hot groups back to categorical values, and appends the synthetic minority rows to the training set. This makes the method a pure minority-class generative oversampler rather than a joint classifier-generator.

3. SEMST-ResNet denoiser

The architectural novelty of SEMRes-DDPM is SEMST-ResNet, the reverse-process denoiser designed for tabular data. The network is organized around residual blocks, with an input fully connected projection, a stack of SEMST-ResNet blocks, and a final fully connected projection producing the predicted noise $S^{\text{norm}}$7 (Zheng et al., 2024).

At the block level, the residual form is

$S^{\text{norm}}$8

which is implemented effectively as an identity mapping plus an indirect residual transform. The initial feature transformation is

$S^{\text{norm}}$9

This is followed by multi-head self-attention,

$S_0 = S^{\text{norm}},$0

and a learned soft-thresholding operator,

$S_0 = S^{\text{norm}},$1

where $S_0 = S^{\text{norm}},$2 is Sigmoid. The indirect mapping is then defined as

$S_0 = S^{\text{norm}},$3

This construction encodes three architectural biases absent from a plain MLP denoiser. First, residual connections stabilize optimization and allow the block to learn corrections rather than a full transformation. Second, multi-head self-attention models feature interactions more explicitly than standard feed-forward tabular encoders. Third, soft-thresholding inserts an explicit denoising mechanism: thresholds are learned per sample and per feature, and Sigmoid constrains them to $S_0 = S^{\text{norm}},$4, avoiding trivial zeroing. The full network is summarized as

$S_0 = S^{\text{norm}},$5

The paper conceptually includes timestep information in $S_0 = S^{\text{norm}},$6, although the exact embedding mechanism is not specified in detail. This suggests that SEMST-ResNet should be understood as a tabular denoiser embedded within a standard DDPM timestep-conditioned framework, rather than as a standalone residual network.

4. Training procedure and oversampling pipeline

The experimental protocol uses 10-fold cross-validation. For each fold, 9 folds are used for training and 1 fold for testing. Within the training partition, majority and minority rows are separated, and SEMRes-DDPM is trained only on the minority subset (Zheng et al., 2024).

The operational workflow is fixed and explicit:

1. Sample minority-class rows from the training partition.
2. Apply one-hot encoding to categorical variables and min-max normalization to all features.
3. Train SEMRes-DDPM on minority rows using the DDPM noise-prediction loss.
4. Generate synthetic minority samples from a Gaussian prior via reverse diffusion.
5. Denormalize the generated outputs and convert one-hot groups back to categorical values.
6. Combine original majority rows, original minority rows, and synthetic minority rows into a balanced training set.
7. Train a downstream classifier on that balanced set and evaluate on the untouched test fold.

For fairness against TabDDPM, the number of diffusion steps is fixed at $S_0 = S^{\text{norm}},$7, and the reported training budget is 20,000 iterations. The implementation environment is given as PyTorch 2.0.1 with GPU, CUDA 11.7 with cuDNN, and Python 3.9. The article does not provide optimizer, learning rate, or batch-size values numerically, so the method description is more precise about architectural and procedural structure than about low-level optimization hyperparameters (Zheng et al., 2024).

This pipeline clarifies that SEMRes-DDPM is decoupled from the classifier. The generator is trained first on minority rows; the classifier is trained afterward on the rebalanced dataset. The oversampling mechanism is therefore model-agnostic with respect to the final classifier.

5. Empirical evaluation

The empirical study covers 20 real imbalanced tabular datasets from KEEL and evaluates nine classifiers: Gaussian Naïve Bayes, Bernoulli Naïve Bayes, k-Nearest Neighbor, Logistic Regression, Random Forest, Decision Tree, Gradient Boosting, Support Vector Machine, and a Multilayer Perceptron classifier (Zheng et al., 2024).

The evaluation has three layers. First, denoising quality is assessed by comparing SEMST-ResNet with an MLP denoiser inside the DDPM framework. Second, distributional fidelity is assessed through density functions, cumulative distribution functions, and absolute Pearson correlations between real and generated data. Third, downstream utility is assessed through F1-score, G-mean, and AUC after oversampling and classifier training.

Selected PSNR examples reported for denoising quality are as follows:

Dataset	MLP	SEMST-ResNet
abalone9-18	8.66	11.78
yeast3	9.94	14.58
spambase	7.71	18.45

These examples are consistent with the paper’s broader statement that PSNR between denoised and original data is higher for SEMST-ResNet than for MLP across all 20 datasets. The associated qualitative scatter plots are described as showing that MLP-denoised samples remain intermingled with noise after 1000 reverse steps, whereas SEMST-ResNet-denoised samples are much closer to the original data (Zheng et al., 2024).

Distribution matching is reported through visual density and CDF comparisons and through absolute Pearson correlations. Representative values include page-blocks-1-3-vs-4, where the reported correlations are 0.8694 for CWGAN-GP, 0.8826 for TabDDPM, and 0.9823 for SEMRes-DDPM; yeast-0-5-6-7-9-vs-4, where the reported values are 0.2915, 0.2474, and 0.8718; and newthyroid2, where the reported values are 0.1120, 0.7307, and 0.9786. These comparisons are used to support the claim that SEMRes-DDPM generates data distributions closer to the real minority distribution than CWGAN-GP and TabDDPM (Zheng et al., 2024).

For downstream classification, the mean scores over 20 datasets include the following comparison with TabDDPM:

Metric	TabDDPM	SEMRes-DDPM
F1	0.6550	0.7059
G-mean	0.7703	0.8173
AUC	0.8755	0.8979

The paper also reports that, among the non-SEMRes methods, mean F1 is 0.6763 for OREM and 0.6758 for GDO, mean G-mean is 0.7899 for OREM, and mean AUC is 0.8803 for GDO; SEMRes-DDPM exceeds these reported averages. Mean ranks over 20 datasets are 7.90 for F1, 7.30 for G-mean, and 7.95 for AUC, and a Friedman test followed by a Nemenyi post-hoc test is reported as showing that the methods do not have equal performance and that SEMRes-DDPM is significantly better than 3–6 competing methods per metric (Zheng et al., 2024).

The reported dataset-level examples reinforce the same pattern. On yeast-2-vs-4, the F1 values are 0.7047 for SMOTE, 0.6634 for TabDDPM, and 0.8300 for SEMRes-DDPM; the G-mean values are 0.7660, 0.7532, and 0.9039. On ecoli1, the F1 values are 0.6554 for SMOTE, 0.6687 for TabDDPM, and 0.8345 for SEMRes-DDPM; the G-mean values are 0.7625, 0.7858, and 0.9322. On spambase, TabDDPM reports a slightly higher F1 than SEMRes-DDPM, 0.7319 versus 0.7228, while SEMRes-DDPM reports a higher G-mean, 0.7914 versus 0.7804, and an AUC equal to TabDDPM at 0.9259. This suggests that the method’s advantage is broad but not uniformly dominant on every metric for every dataset.

6. Nomenclature and relation to other diffusion models

The name SEMRes-DDPM can be confused with residual or ensemble diffusion models in image analysis, but the tabular oversampling method is distinct from those lines of work. In particular, "ResEnsemble-DDPM: Residual Denoising Diffusion Probabilistic Models for Ensemble Learning" defines a residual ensemble segmentation framework in which a diffusion model learns a mirror target around the ground truth and is combined with a frozen end-to-end segmenter; that paper explicitly contains no mention of “SEMRes-DDPM” anywhere in the paper or appendices (Zhenning et al., 2023).

A second possible source of confusion is the semi-supervised medical segmentation literature. "Robust semi-supervised segmentation with timestep ensembling diffusion models" introduces TEDM, a timestep-ensembling diffusion model that uses a pretrained DDPM as a frozen feature extractor and ensembles predictions from multiple timesteps for robust segmentation under domain shift (Rosnati et al., 2023). That approach is concerned with representation learning and output-level ensembling for medical images, not minority-class tabular oversampling.

Likewise, "SAR Despeckling using a Denoising Diffusion Probabilistic Model" applies conditional DDPMs to SAR despeckling, using a U-Net-like conditional denoiser and a cycle-spinning inference strategy for restoration quality (Perera et al., 2022). The contrast is informative: SEMRes-DDPM replaces the image-oriented U-Net with a residual-attention-soft-thresholding denoiser precisely because the target domain is tabular data rather than gridded imagery.

This separation of meanings matters because the architectural and probabilistic choices are domain-specific. In SEMRes-DDPM, the decisive design variable is not image-conditioned denoising or segmentation-time ensembling, but the ability of the reverse denoiser to model feature interactions and remove noise effectively in structured tabular rows.

7. Limitations and prospective extensions

The principal reported limitation is computational cost. The authors note that training SEMRes-DDPM takes a large amount of time, and that for small batches, or when both accuracy and time matter, simpler methods may be preferable (Zheng et al., 2024).

Additional assumptions are embedded in the preprocessing and generation pipeline. Mixed data types are handled through one-hot encoding and min-max normalization, with no specialized treatment for high-cardinality categorical variables or ordinal structure. The method also assumes that the minority class contains enough samples to support generative modeling and that the features can be encoded into a numeric space in which Gaussian noising is a reasonable diffusion mechanism (Zheng et al., 2024).

The stated future directions are to reduce training time, explore the addition of label information or other conditioning into training, and apply SEMRes-DDPM to real-world domain-specific problems beyond public datasets. A plausible implication is that the current method is best understood as a strong baseline for unconditional minority-class generation, rather than a final answer to all forms of imbalanced tabular synthesis. Its empirical contribution lies in showing that a DDPM equipped with a denoiser tailored to tabular structure can outperform both heuristic oversamplers and prior GAN- and DDPM-based baselines on F1, G-mean, and AUC across a broad benchmark suite (Zheng et al., 2024).