Iterative Pixel Stratification (IPS)

• IPS is a segmentation-specific method that replaces binary label balancing with pixel count balancing to ensure more representative train/validation/test splits.
• It employs a greedy assignment mechanism that prioritizes rare classes by tracking residual pixel quotas, thereby addressing imbalances in dense image segmentation.
• Empirical studies show IPS improves pixel-level proportionality compared to random splits, though it can suffer from sample-count imbalances and lacks global optimization.

Iterative Pixel Stratification (IPS) is a segmentation-specific adaptation of iterative stratification for multilabel data, introduced to construct train/validation/test splits or $K$-fold partitions whose label composition is more representative than what one typically obtains from naive random splitting in image segmentation (Jami et al., 25 Sep 2025). Its defining move is to replace label-occurrence balancing with pixel-count balancing: instead of treating a class as merely present or absent in an image, IPS tracks how many pixels of each semantic class are assigned to each fold. The method is therefore tailored to dense prediction settings in which an image can contain multiple classes simultaneously and the relevant distributional quantity is class-wise pixel proportion rather than binary label incidence.

1. Conceptual basis and motivation

The motivation for IPS begins from a practical difficulty in segmentation evaluation. Random splitting of segmentation datasets often produces folds whose class distributions differ noticeably from the overall dataset, especially when the dataset is small, imbalanced, or contains rare classes (Jami et al., 25 Sep 2025). This is consequential because segmentation performance is usually evaluated per class or with metrics such as F1 and IoU; if a class is underrepresented or absent in a fold, evaluation becomes unstable or even undefined for that class. The effect is particularly acute for test folds, because they are smaller and therefore more sensitive to sampling noise.

Classical stratification is comparatively straightforward in classification problems because each sample usually belongs to exactly one class, or at least to a finite label set without proportions. Segmentation does not satisfy that structure. An image may contain several classes at once, and what matters is not only whether a class occurs, but how many pixels it occupies. Two images may both contain “road” and “car,” while one is dominated by road pixels and the other contains only a very small road region. IPS is designed for precisely this setting: a split should ideally preserve pixel-level class proportions rather than merely class occurrence.

This design makes IPS segmentation-specific in a strong sense. Prior multilabel stratification methods such as Iterative Stratification (IS), PMBSRS, and SOIS treat labels as binary present/absent events. IPS extends the iterative stratification idea by reformulating demand in terms of class pixel counts. A fold in which a rare class appears in several images can still be unrepresentative if nearly all of that class’s pixels concentrate in one fold or if one fold receives only tiny regions while another receives all large regions. IPS addresses that failure mode by operating directly on mask-derived pixel statistics (Jami et al., 25 Sep 2025).

2. Formal problem statement

The paper formalizes the segmentation splitting problem on a dataset

$D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$

where $\mathbf{x}_n$ is an image and $\mathbf{y}_n$ is its segmentation mask (Jami et al., 25 Sep 2025). There are $C$ semantic classes and $K$ desired folds with target proportions

$$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$

The goal is to produce disjoint subsets

$\mathbf{S} = (S^1, \dots, S^K)$

whose sizes approximately follow $\mathbf{r}$, while preserving the label composition of the full dataset.

The key statistics are pixel counts. The paper defines $P^n$ as the total number of pixels in sample $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$0, $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$1 as the number of pixels of class $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$2 in sample $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$3, $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$4 as the total number of pixels of class $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$5 in the whole dataset, $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$6 as the desired or accumulated number of pixels of class $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$7 in fold $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$8, $D = \{(\mathbf{x}_n, \mathbf{y}_n)\}_{n=1}^{N},$9 as the number of samples that contain class $\mathbf{x}_n$0, and $\mathbf{x}_n$1 as the desired number of such samples in fold $\mathbf{x}_n$2 (Jami et al., 25 Sep 2025).

IPS tries to match per-fold class pixel quotas

$\mathbf{x}_n$3

for every class $\mathbf{x}_n$4, while also tracking desired fold sizes

$\mathbf{x}_n$5

This is the core formal distinction from multilabel iterative stratification in classification: “number of positive instances of a label” is replaced by “number of pixels of that label.” When a sample is placed in a fold, the residual demand for every class is reduced by the number of pixels of that class present in that sample (Jami et al., 25 Sep 2025).

3. Greedy assignment mechanism

IPS is a greedy sequential assignment method rather than a global optimization procedure. Its rule is simple: repeatedly select the rarest remaining class and assign samples containing that class to the fold that currently has the largest unmet demand for that class’s pixels (Jami et al., 25 Sep 2025). The appendix pseudocode specifies the procedure.

First, the algorithm computes the desired number of samples in each fold, $\mathbf{x}_n$6. For each class $\mathbf{x}_n$7, it computes the total class-pixel count $\mathbf{x}_n$8 and the desired number of pixels of class $\mathbf{x}_n$9 in each fold, $\mathbf{y}_n$0. While unassigned samples remain, it finds the rarest class among the remaining data,

$\mathbf{y}_n$1

and iterates over the subset $\mathbf{y}_n$2 of unassigned samples containing class $\mathbf{y}_n$3.

For each sample $\mathbf{y}_n$4, IPS identifies the fold or folds with the largest remaining desire for class $\mathbf{y}_n$5,

$\mathbf{y}_n$6

If the maximizer is unique, the sample is assigned there. If several folds tie, IPS prefers the one with smallest remaining desired sample count,

$\mathbf{y}_n$7

If a tie remains, it chooses a random element of $\mathbf{y}_n$8. After selecting fold $\mathbf{y}_n$9, the sample is assigned to $C$0, removed from the unassigned dataset, the remaining sample demand is decremented as $C$1, and for every class $C$2 the fold’s remaining pixel demand is reduced by the number of class-$C$3 pixels in that sample,

$C$4

The process continues until all samples are assigned (Jami et al., 25 Sep 2025).

In plain terms, IPS treats rare classes as highest priority because allocation mistakes on rare classes are hardest to repair later. It is deterministic up to tie-breaking and the iteration order over samples in the currently selected rare-class subset. The paper does not provide a formal asymptotic complexity expression. It states only that IPS is a straightforward greedy iterative procedure; the appendix runtime comparison indicates that IPS is slower than random splitting and faster than WDES, but no exact complexity formula is given (Jami et al., 25 Sep 2025).

4. Representativeness criteria and relation to WDES

The paper introduces three heterogeneity or similarity measures that clarify what IPS is intended to preserve (Jami et al., 25 Sep 2025). Sample count balance is measured by Sample Distribution (SD):

$C$5

Pixel-level class proportion matching is measured by Pixel Label Distribution (PLD):

$C$6

Overall distributional similarity can also be summarized with the Label Wasserstein Distance (LWD). Defining cumulative class distributions by

$C$7

the paper gives

$C$8

IPS does not directly minimize PLD or LWD. Rather, it greedily tries to satisfy per-class pixel quotas fold by fold, which is intended to improve representativeness indirectly. This point is central to its contrast with Wasserstein-Driven Evolutionary Stratification (WDES). IPS is a greedy single-pass heuristic; once a sample is assigned, that choice is never revisited. WDES instead is a population-based genetic algorithm whose individuals encode complete assignments and whose fitness is defined by a Wasserstein-based score over full splits (Jami et al., 25 Sep 2025).

The paper identifies three major differences between IPS and WDES: objective, search strategy, and guarantees. IPS does not optimize an explicit global objective over complete stratifications, whereas WDES evaluates complete candidate splits in parallel and explores a broader combinatorial space. No analogous convergence guarantee is given for IPS. This difference underlies the paper’s later failure analysis: IPS was designed to be a label-aware stratification method, but its single-pass, greedy allocation strategy reduces label demands immediately after each assignment and cannot reconsider earlier decisions (Jami et al., 25 Sep 2025).

5. Empirical behavior, strengths, and limitations

IPS was evaluated on five segmentation datasets spanning several domains: PascalVOC 2012, CamVid, EndoVis2018, LoveDA, and Cityscapes (Jami et al., 25 Sep 2025). The baselines were random splitting using scikit-learn KFold, IPS, and WDES. Evaluation proceeded in two stages. First, the paper computed SD, PLD, and LWD on generated folds. Second, it assessed downstream impact using 10-fold cross-validation with a UNet (ResNet34 encoder), reporting mean and standard deviation of Accuracy, F1 score, and IoU.

The fold-quality results are mixed and generally unfavorable to IPS relative to WDES. IPS often has substantially worse SD than both random splitting and WDES because it prioritizes pixel balance over exact sample-count proportionality. Reported examples are PascalVOC: IPS SD $C$9 versus random $K$0 and WDES $K$1; CamVid: IPS SD $K$2 versus random $K$3 and WDES $K$4; EndoVis: IPS SD $K$5 versus random $K$6 and WDES $K$7; LoveDA: IPS SD $K$8 versus random $K$9 and WDES $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$0; Cityscapes: IPS SD $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$1 versus random $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$2 and WDES $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$3 (Jami et al., 25 Sep 2025).

On PLD and LWD, IPS sometimes improves over random splitting but is never better than WDES in the reported means. PascalVOC is a representative case where IPS helps relative to random—PLD drops from $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$4 to $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$5, and LWD from $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$6 to $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$7—yet WDES still performs better with PLD $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$8 and LWD $$\mathbf{r} = (r^1, \dots, r^K), \qquad \sum_{k=1}^K r^k = 1.$$9. Other datasets show weaker behavior: on CamVid, IPS is worse than random in both PLD and LWD; on EndoVis, IPS is again worse than random; on LoveDA, IPS is roughly similar to random in PLD and slightly worse in LWD; and on Cityscapes, IPS trails random on both metrics (Jami et al., 25 Sep 2025).

The downstream segmentation results follow the same pattern. On PascalVOC, IPS reduces fold-to-fold variance relative to random splitting: Accuracy standard deviation decreases from 20.6 to 14.5, F1 standard deviation from 32.5 to 30.3, and IoU standard deviation from 34.3 to 31.6. WDES improves further, to 11.0, 24.0, and 24.2 respectively. On EndoVis, IPS improves accuracy variance but worsens F1 and IoU variance: Accuracy standard deviation falls from 19.4 to 14.1, while F1 rises from 27.0 to 29.3 and IoU from 28.1 to 30.7. CamVid is described as very close to random and generally slightly worse, while LoveDA and Cityscapes are often worse than random and worse than WDES (Jami et al., 25 Sep 2025).

The paper’s limitations analysis is explicit. It identifies five weaknesses: single-pass greedy assignment, path dependence, local rather than global reasoning, weak handling of label co-occurrence and contextual diversity, and sample-count imbalance (Jami et al., 25 Sep 2025). IPS may fail to preserve co-occurring label groups, scene composition, or interdependencies between frequent and infrequent classes. Because it focuses on pixel quotas, it can also violate fold-size targets substantially, which is reflected in poor SD.

6. Practical status, usage guidance, and terminological disambiguation

The implementation is described as a straightforward baseline: IPS is “implemented by adapting Iterative Stratification without additional optimizations,” and the paper presents it as “a straightforward baseline” rather than a final recommendation (Jami et al., 25 Sep 2025). Its practical role is therefore specific. It is useful when a practitioner wants a simple segmentation-aware alternative to random splitting that directly exploits class pixel counts. The evidence suggests that it may help on some datasets, particularly where rare classes need better pixel-level balancing than random splitting provides, but it does not emerge as the preferred method under any broad regime.

The paper’s guidance is correspondingly narrow. For very large datasets, random splitting may already be adequate; the appendix gives an asymptotic representativeness result for random splits in multilabel data using the law of large numbers, and the authors state that computational efficiency and simplicity make random splitting practical in that regime (Jami et al., 25 Sep 2025). For smaller datasets, especially imbalanced or low-entropy ones, the recommendation is WDES rather than IPS, because WDES consistently gives better PLD and LWD and lower variance in downstream model evaluation.

A frequent misconception is terminological rather than methodological. In other recent work, the acronym “IPS” refers to different methods. “Iterative Patch Selection” in high-resolution image recognition is patch-based rather than pixel-based, and it does not present Iterative Pixel Stratification (Bergner et al., 2022). The same acronym is used in a later generalization study of that patch-selection framework, again with no pixel-level stratification procedure (Riffi-Aslett et al., 2024). A different line of work in single-pixel imaging presents an iterative region-based refinement method driven by image maps and mean values over overlapping partitions; that method is conceptually adjacent in its selective refinement of non-empty regions, but it is not an IPS method in the segmentation sense and does not explicitly stratify pixels into hierarchical strata (Pastuszczak et al., 1 Sep 2025).

In its canonical meaning, Iterative Pixel Stratification denotes the greedy segmentation-data splitting algorithm that replaces label-occurrence balancing with pixel-count balancing. Its historical significance lies less in empirical dominance than in formalizing a segmentation-aware extension of multilabel iterative stratification. Within the paper that introduced it, IPS functions as a natural baseline: label-aware, straightforward to implement from class pixel counts, and conceptually well matched to dense prediction, but limited by irreversible local decisions and generally outperformed by the Wasserstein-based evolutionary alternative (Jami et al., 25 Sep 2025).