Dynamic Region Merging: Image Segmentation

Updated 8 May 2026

Dynamic Region Merging is an image segmentation strategy that starts with over-segmentation and iteratively fuses similar regions using statistical hypothesis tests.
It employs methods ranging from classical testing to deep learning techniques, optimizing the merge order and stopping criteria for precise segmentation.
Applications in natural image processing and high-resolution remote sensing yield improved object boundary alignment and reduced over-segmentation.

Dynamic Region Merging (DRM) is a class of image segmentation algorithms that start from a fine, over-segmented decomposition of an image into small regions (super-pixels) and progressively merge regions based on local similarity and statistical criteria. Unlike static or one-shot methods, DRM incorporates adaptive and data-driven strategies for both the ordering of merges and the stopping criterion, aiming to achieve segmentations that align with object boundaries while avoiding both over- and under-segmentation. Region merging strategies now span a range from classical hypothesis-testing frameworks to advanced deep learning-based approaches, with extensive applications in natural images and high-resolution remote sensing domains (Peng et al., 2010, Lv et al., 2023, Tang et al., 2020).

1. Foundational Principles and Problem Formulation

Dynamic Region Merging is predicated on two main steps: (1) an initial over-segmentation (e.g., via watershed or mean-shift) yielding super-pixels, and (2) a sequence of merges governed by a similarity predicate that balances local evidence with global consistency. Formally, let the initial segmentation be $S_0 = \{R_1, \ldots, R_N\}$ , and the Region Adjacency Graph (RAG) $G=(V,E)$ , where each node $v_i$ corresponds to region $R_i$ . Edges $(v_i, v_j)$ exist if $R_i$ and $R_j$ are spatially adjacent.

The core inference problem is to assign semantic labels $\ell_i$ to regions such that regions corresponding to the same object share a label. Merging decisions between adjacent regions $(R_i, R_j)$ are treated as statistical hypothesis tests:

$H_0$ : $G=(V,E)$ 0 and $G=(V,E)$ 1 arise from different distributions,
$G=(V,E)$ 2: $G=(V,E)$ 3 and $G=(V,E)$ 4 arise from the same distribution.

The objective is a segmentation minimizing a global cost, often formulated in terms of edge (boundary) evidence or region homogeneity, while rigorously managing merge order and stopping conditions (Peng et al., 2010).

2. Classical Formulations: Statistical Predicates and Dynamic Programming

A definitive instance of DRM uses a sequential merging predicate based on both mutual similarity and statistical consistency. Two adjacent regions $G=(V,E)$ 5 are candidates for merging if:

They are mutual nearest neighbors in the RAG (i.e., $G=(V,E)$ 6 is the minimal dissimilarity in each other's adjacency set),
They pass Wald’s Sequential Probability Ratio Test (SPRT) on pooled boundary data:

$G=(V,E)$ 7

where $G=(V,E)$ 8, $G=(V,E)$ 9 are likelihoods under $v_i$ 0, $v_i$ 1, and $v_i$ 2, $v_i$ 3 are maximum likelihood parameters. Acceptance of $v_i$ 4 (merge) or $v_i$ 5 (do not merge) depends on $v_i$ 6 crossing upper/lower thresholds determined by desired error rates.

The overall segmentation process is equivalent to dynamically programming the globally optimal sequence of merges as shortest paths in a region-labeling graph, driven by the above merge predicate. Key theoretical properties include:

No under-merging: If adjacent regions remain unmerged, there is insufficient evidence for merging under the predicate.
No over-merging: No finer segmentation also satisfies the predicate everywhere.

Efficiency is greatly improved via nearest-neighbor graphs (NNG), selecting only mutual-nearest pairs and updating locally after each merge. Complexity is reduced from $v_i$ 7 per scan to $v_i$ 8 per iteration (Peng et al., 2010).

3. Algorithmic Developments: Dam Burst and Adaptive Heuristics

Alternative DRM variants such as “Dam Burst” [Editor's term for (Tang et al., 2020)] propose heuristics motivated by “flooding” analogies. Here, the process is governed by gradient strength and adaptive thresholds:

Gradient images are computed with Haar–box filters to suppress texture noise.
The Canny detector provides binary edge maps for boundary reinforcement.
The merging order is set by region mean‐gradient values, simulating weaker areas “flooding” first.
The primary merge criterion incorporates dam strength, region strength indices, and adaptively updated appearance thresholds. Weak dams ( $v_i$ 9) and low region heterogeneity ( $R_i$ 0) favor merging, while strong regions or high contrast inhibit it.

This approach achieves single-level segmentation with reduced over-segmentation, quantitative region count reductions (e.g., from ~1150 to ~376 regions on example images), and flexible adaptation to both edge and content cues, though parameter sensitivity remains an open consideration (Tang et al., 2020).

4. Learning-Based Dynamic Region Merging

Recent advances integrate learning-based criteria into DRM. DeepMerge (Lv et al., 2023) exemplifies this shift by leveraging deep similarity learning via a Transformer-based architecture (“S2Former”) in conjunction with region adjacency graphs:

Super-pixels are obtained via multiresolution segmentation (MRS).
Nodes in the RAG encode both learned feature vectors (from multiple shift-scale patches extracted via a binary-tree sampling scheme) and 18 hand-crafted descriptors (e.g., mean, std, compactness).
A shift-scale attention mechanism with 3D relative position bias fuses features across context windows at three spatial scales.
Merge decisions are based on the Euclidean distance between S2Former feature vectors:

$R_i$ 1

Training uses a margin-based contrastive loss:

$R_i$ 2

with $R_i$ 3, $R_i$ 4 for positive/negative pairs.

The merge threshold $R_i$ 5 separates positive (to merge) from negative (not to merge) examples in an interpretable and non-tunable manner, as negative pairs are trained to be separated from positive pairs by at least this margin. The final feature representation for merged nodes is updated via area-weighted averaging, allowing efficient incremental operation (Lv et al., 2023).

5. Quantitative Evaluation and Comparative Results

DRM methods are evaluated on datasets such as BSDS500 and high-resolution aerial mosaics, using boundary precision, recall, F-measure, and over-/under-segmentation errors:

Classical DRM (with watershed or mean-shift initialization) achieves F-measure up to 0.66 on BSDS500, outperforming traditional edge and early graph-based algorithms (Peng et al., 2010).
Dam Burst reduces region counts by 50–70% compared to hierarchical baselines, indicating significant mitigation of over-segmentation (Tang et al., 2020).
DeepMerge achieves high segmentation accuracy on 0.55 m RGB mosaics over 5,660 km²: F=0.9550, TE=0.0895, surpassing all baselines including FHS (F=0.8465), and region-based CNNs (UNet, DeepLab; F≈0.03–0.38), which are reported to suffer from severe under-segmentation under region-overlap metrics.

Scale sensitivity analysis in DeepMerge indicates that the F-measure peaks exactly at $R_i$ 6, reaffirming the loss-motivated interpretability of the merge threshold (Lv et al., 2023).

6. Applications, Strengths, Limitations, and Extensions

Dynamic Region Merging is central to unsupervised and semi-supervised segmentation in both natural and high-resolution remote sensing imagery. Strengths across classical and modern DRM approaches include:

Theoretical guarantees against under- and over-merging.
Interpretability and adaptability through statistical or learned similarity predicates.
Efficiency via NNG acceleration or incremental feature fusion schemes.
Flexibility: Dam Burst and related heuristics provide practical, generic modules for various initializations.

Limitations persist in the form of parameter sensitivity (box size, thresholds), dependence on initialization quality, and the possibility of residual over- or under-segmentation if cues are ambiguous. DeepMerge further demonstrates that deep learning, when fused with classical RAG frameworks and scale-adaptive attention, can yield interpretable and high-performing region-merging algorithms. Potential enhancements outlined include the incorporation of advanced descriptors, adaptive region sizing, and integration with learned or structured edge detectors (Lv et al., 2023, Tang et al., 2020).

7. Representative Comparison Table

Method	Merge Predicate	Global Guarantee	Key Dataset Results
DRM (Peng et al., 2010)	Mutual NN + SPRT	No under/over-merging	BSDS500: F=0.66
Dam Burst (Tang et al., 2020)	Edge-adapted gradient	Adaptive single-level	Region count drop 50–70%
DeepMerge (Lv et al., 2023)	Deep S2Former+features	Interpretable threshold	0.55 m mosaic: F=0.9550, TE=0.0895

These approaches collectively define Dynamic Region Merging as a rigorous, flexible family of segmentation algorithms, encompassing both foundational statistical formulations and recent advances in deep similarity learning.