NSegment+: Scale-Aware Label Augmentation

• NSegment+ is a training-time augmentation framework that improves segmentation by applying label-only stochastic elastic deformation to address implicit label noise.
• It decouples image and label transformations and employs scale-aware perturbation suppression to protect small objects from being damaged by global warping.
• Empirical results demonstrate that NSegment+ consistently enhances mIoU across multiple benchmarks without incurring extra inference cost.

NSegment+ is a training-time augmentation framework for semantic segmentation that targets implicit label noise: subtle annotation imperfections arising from ambiguous object boundaries, mixed pixels, occlusion, shadows, and annotator variability. Its central design principle is to decouple image and label transformations. Instead of applying the same geometric warp to both the RGB image and its mask, NSegment+ keeps the image unchanged and perturbs only the segmentation label with a stochastic elastic deformation. The resulting supervision is intended to represent plausible alternatives to a single brittle contour. The method extends the earlier NSegment formulation by adding scale-aware perturbation suppression so that small objects are not destroyed by global warping (Kim et al., 14 Aug 2025, Kim et al., 28 Apr 2025).

1. Problem setting and conceptual framing

NSegment+ is formulated for segmentation regimes in which labels are not grossly corrupted, yet are still imperfect in mild, spatially localized, and structurally plausible ways. The paper distinguishes this from the more common explicit/severe noise setting, such as class flips, missing objects, or heavily corrupted masks. Its assumption is that datasets may be free of obvious annotation failures while still containing latent localization uncertainty around boundaries, which can degrade mIoU during ordinary supervised training (Kim et al., 14 Aug 2025).

The method is motivated by a critique of standard segmentation augmentation. Conventional pipelines apply geometric transforms identically to image and label in order to preserve exact pixel alignment. NSegment+ treats that exactness as a problematic assumption when the label itself is only one plausible delineation. In this view, synchronized augmentation preserves annotation rigidity rather than annotation uncertainty. The earlier NSegment paper made the same argument in remote sensing, where dense supervision, boundary sensitivity, shadows, mixed pixels, and subjective annotation can make even small discrepancies consequential (Kim et al., 28 Apr 2025).

A common misconception is that NSegment+ is a noisy-label correction method in the style of sample selection, label repair, or uncertainty modeling. It is not. The paper does not redesign the segmentation architecture, the optimizer, the inference procedure, or the loss family. Instead, it treats annotation ambiguity as a label-level regularization problem: the network is trained against stochastically deformed masks while the visual evidence remains photometrically and geometrically intact (Kim et al., 14 Aug 2025).

2. Mathematical formulation and label-only deformation

Given an image

$\mathcal{I} \in \mathbb{R}^{w \times h \times 3}$

and its segmentation label

$\mathcal{L}=\{S_j\}_{j=1}^{C},$

where each $S_j \in \mathbb{R}^{w \times h}$ is the binary mask for class $j$, NSegment first samples a smooth nonrigid deformation field and applies it only to the masks. The deformation parameters are sampled from

$$\mathbf{\Omega} = \{(\alpha_k,\sigma_k)\}_{k=1}^{K},$$

with $\alpha$ controlling deformation magnitude and $\sigma$ controlling spatial smoothness (Kim et al., 14 Aug 2025).

The horizontal and vertical displacement fields are generated as

$$d\mathcal{X} \gets G_{\sigma} \circledast \phi (\alpha (2 \cdot {rand}(w, h) - 1) ),$$

$$d\mathcal{Y} \gets G_{\sigma} \circledast \phi (\alpha (2 \cdot {rand}(w, h) - 1) ).$$

Here the raw random field is drawn from a uniform distribution on $[-1,1]$, zero-padded via $\mathcal{L}=\{S_j\}_{j=1}^{C},$0, and convolved with a Gaussian kernel $\mathcal{L}=\{S_j\}_{j=1}^{C},$1. The kernel size is

$\mathcal{L}=\{S_j\}_{j=1}^{C},$2

following OpenCV convention. Each class mask is then warped by the remapping

$\mathcal{L}=\{S_j\}_{j=1}^{C},$3

using bilinear interpolation and boundary clipping via $\mathcal{L}=\{S_j\}_{j=1}^{C},$4, producing a transformed label

$\mathcal{L}=\{S_j\}_{j=1}^{C},$5

The transform is stochastic at the sample level and epoch level: the same image-mask pair can receive different label perturbations across training passes (Kim et al., 14 Aug 2025).

The augmentation is applied probabilistically. Algorithm 1 states

$\mathcal{L}=\{S_j\}_{j=1}^{C},$6

so the practical deformation probability is $\mathcal{L}=\{S_j\}_{j=1}^{C},$7. The default is $\mathcal{L}=\{S_j\}_{j=1}^{C},$8, while focused ablations use $\mathcal{L}=\{S_j\}_{j=1}^{C},$9. The supervised objective itself is unchanged in form; conceptually, training becomes

$S_j \in \mathbb{R}^{w \times h}$0

Thus NSegment+ is a preprocessing transform inserted before ordinary segmentation learning rather than a new segmentation loss or decoder (Kim et al., 14 Aug 2025).

3. The “plus” extension: scale-aware perturbation suppression

The distinction between NSegment and NSegment+ is the addition of scale-aware perturbation suppression. The underlying issue is that a globally sampled deformation field can damage small objects disproportionately. A car, sign, or small fragmented region may be semantically erased or shifted too far even when the same deformation is harmless for larger objects (Kim et al., 14 Aug 2025).

To prevent this, NSegment+ inspects each segment $S_j \in \mathbb{R}^{w \times h}$1. If

$S_j \in \mathbb{R}^{w \times h}$2

then the algorithm suppresses deformation in a neighborhood around that segment. The paper uses a single threshold across all datasets,

$S_j \in \mathbb{R}^{w \times h}$3

Let $S_j \in \mathbb{R}^{w \times h}$4 denote the bounding coordinates of the segment, and define

$S_j \in \mathbb{R}^{w \times h}$5

The bounding box is expanded with clipping, and the displacement fields are then zeroed within that region: $S_j \in \mathbb{R}^{w \times h}$6 This produces an identity-like deformation locally around small objects while leaving larger regions deformable (Kim et al., 14 Aug 2025).

The default deformation pool is the Cartesian product

$S_j \in \mathbb{R}^{w \times h}$7

giving 15 candidate $S_j \in \mathbb{R}^{w \times h}$8 pairs. The paper states that this range was chosen by grid search. In practical terms, the “plus” version is therefore defined by three defaults: stochastic parameter sampling, deformation probability $S_j \in \mathbb{R}^{w \times h}$9, and small-mask suppression with $j$0 (Kim et al., 14 Aug 2025).

4. Benchmarks, architectures, and training protocol

NSegment+ is implemented as a custom transform in PyTorch/MMSegmentation, with NumPy and OpenCV used in the augmentation code. Training and evaluation protocols otherwise follow the public implementations of the underlying segmentation models, and there is no extra inference cost. The paper emphasizes that NSegment+ uses the same GFLOPs as the baseline at test time (Kim et al., 14 Aug 2025).

The evaluation spans six datasets across remote sensing and natural-scene segmentation.

Dataset	Setup	Domain
Vaihingen	33 orthophoto tiles, 16 train / 17 test, cropped to $j$1 with stride 256; clutter excluded	Remote sensing
Potsdam	38 tiles, 24 train / 14 test, cropped to $j$2 with stride 256; clutter excluded	Remote sensing
LoveDA	2522 train / 1669 validation-as-test, random crop $j$3	Remote sensing
Cityscapes	2975 train / 500 val, official 19-class setup, random crop $j$4	Natural scene
PASCAL VOC 2012	augmented training set of 10582 images, 1449 val, random crop $j$5	Natural scene
COCO-Stuff 10K	9000 train / 1000 test, random crop $j$6	Natural scene

The method is tested on 12 architectures: DeepLab V3+, ANN, DANet, APCNet, GCNet, OCRNet, SegFormer, Mask2Former, DOCNet, CAT-Seg, Golden, and LOGCAN++. Metric is always mIoU. Each experiment is repeated 5 times with different random seeds, and the paper reports mean and standard deviation. Golden is trained for 120000 iterations; all other models are trained for 80000 iterations. Batch size is 4 across datasets. Unless an interaction study explicitly adds other augmentations, no other augmentation is used for either baseline or NSegment+ (Kim et al., 14 Aug 2025).

Relative to its precursor, this represents a clear broadening of scope: the earlier NSegment paper focused on Vaihingen, Potsdam, and LoveDA in remote sensing, whereas NSegment+ extends the same label-only principle to natural-scene benchmarks as well (Kim et al., 28 Apr 2025).

5. Empirical behavior and comparative evidence

The most decisive ablation concerns what to deform. On Vaihingen with PSPNet and stochastic parameter sampling, the baseline is 77.41 mIoU. Standard synchronized elastic deformation on both image and label falls to 67.58; image-only deformation reaches 70.69; label-only deformation reaches 77.75. The earlier NSegment paper reported the same qualitative conclusion: elastic deformation is harmful when applied to images, but useful when applied only to labels (Kim et al., 28 Apr 2025).

A second ablation compares fixed deformation strengths with stochastic sampling. On Vaihingen with PSPNet, fixed $j$7 choices produce mixed behavior, including 77.70 at $j$8, 77.67 at $j$9, and 75.99 at $$\mathbf{\Omega} = \{(\alpha_k,\sigma_k)\}_{k=1}^{K},$$0. By contrast, stochastic sampling over $$\mathbf{\Omega} = \{(\alpha_k,\sigma_k)\}_{k=1}^{K},$$1 yields 77.75, a +0.34 improvement over baseline. The paper interprets this as evidence that varying deformation strength acts as stronger label-level regularization than committing to one fixed scale (Kim et al., 14 Aug 2025).

Across the full benchmark suite, NSegment+ improves average mIoU over baseline by +0.78 on Vaihingen, +0.18 on Potsdam, +0.74 on LoveDA, +0.83 on Cityscapes, +0.91 on PASCAL VOC, and +0.44 on COCO-Stuff 10K. Representative gains include OCRNet on Vaihingen, 75.39 \to 77.68 (+2.29); Golden on LoveDA, 37.56 \to 39.94 (+2.38); DANet on Cityscapes, 62.91 \to 64.66 (+1.75); and GCNet on PASCAL VOC 2012, 61.07 \to 64.46 (+3.39) (Kim et al., 14 Aug 2025).

The paper also directly compares NSegment+ with explicit-noise learning methods on DeepLab V3+ for Vaihingen and Cityscapes. Reported results are 77.53 / 56.95 for baseline, 77.68 / 57.11 for Compensation Learning, 77.72 / 56.51 for L2B, 77.91 / 57.01 for UCE, 77.73 / 57.49 for AIO2, and 78.34 / 57.58 for NSegment+. GFLOPs are 54.27 for baseline and NSegment+, 55.04 for Compensation Learning, 542.69 for UCE, and 108.54 for AIO2. Within that comparison, NSegment+ achieves the highest mIoU on both datasets without increasing test-time computational burden (Kim et al., 14 Aug 2025).

The augmentation also composes with other regularizers. Notable examples with DeepLab V3+ include LoveDA + Random Resize, 40.73 \to 45.89 (+5.16); Cityscapes + Horizontal Flip, 56.62 \to 58.35 (+1.73); Vaihingen + CutMix, 77.54 \to 78.34 (+0.80); and PASCAL VOC + Label Smoothing, 62.55 \to 62.90 (+0.35). The stated interpretation is that image-level augmentation diversifies appearance, while NSegment+ diversifies annotation geometry and boundary uncertainty (Kim et al., 14 Aug 2025).

6. Applicability, limitations, and historical position

NSegment+ is most appropriate when the dominant annotation error mode is implicit, structurally plausible localization noise rather than semantic corruption. The paper does not present it as a remedy for class-flip noise, missing-object labels, or large annotation failures. It is instead a mechanism for teaching tolerance to near-boundary variability while preserving the realism of the image input (Kim et al., 14 Aug 2025).

Its limitations are correspondingly specific. The method may not always be best served by elastic deformation itself; the paper notes that for some domains, especially building extraction from aerial imagery, morphological operations such as erosion may better model realistic mask ambiguity than elastic warping. The need for small-mask suppression already indicates a failure mode: without it, small instances may undergo semantic erosion. More broadly, if a task requires extremely precise boundaries and the annotations are already near-perfect, aggressive label perturbation could plausibly be harmful. This suggests that NSegment+ is strongest when plausible boundary variability exists but does not exceed the scale at which semantics change (Kim et al., 14 Aug 2025).

Historically, NSegment+ should be understood as the scale-aware successor to NSegment, introduced in “NSegment : Label-specific Deformations for Remote Sensing Image Segmentation” (Kim et al., 28 Apr 2025). The earlier method established the label-only deformation principle and showed that identical image-label elastic warping is much worse than label-only warping in remote sensing. NSegment+ preserves that core mechanism, adds suppression for small masks, and expands the evaluation from three remote-sensing benchmarks to six datasets spanning both overhead and natural imagery (Kim et al., 28 Apr 2025, Kim et al., 14 Aug 2025).

By topic, NSegment+ belongs to semantic image segmentation and is unrelated to identically or similarly named methods in NLP, such as the long-text processing framework “SEGMENT+: Long Text Processing with Short-Context LLMs” (Shi et al., 2024). Within vision, its contribution is narrow but well defined: a lightweight, model-agnostic augmentation that reframes a segmentation mask not as an exact target, but as one realization from a local distribution of plausible boundaries (Kim et al., 14 Aug 2025).