Differential Morphological Profile (DMP)

• Differential Morphological Profile (DMP) is a multi-scale shape descriptor computed from absolute differences of morphological openings and closings on grayscale images.
• It enhances semantic segmentation in remote sensing by explicitly encoding scale-aware geometric features via direct-input and hybrid dual-stream network strategies.
• Empirical results show that careful selection of structuring elements and scale sets improves accuracy for geometry-dominated classes in large, tiled imagery.

Searching arXiv for the specified paper and closely related DMPNet work. arXiv search query: "Differential Morphological Profile Neural Networks for Semantic Segmentation" Differential Morphological Profile (DMP) is a multi-scale shape descriptor built from grayscale morphological operators. In overhead remote sensing, it is used to provide an explicit, scale-aware representation of object shape that complements learned features, particularly under extreme scale variation, foreground-background imbalance, and very large image sizes. In "Differential Morphological Profile Neural Networks for Semantic Segmentation" (Huangal et al., 4 Sep 2025), DMP is incorporated into semantic segmentation by extending prior DMPNet work beyond classification and object detection and integrating DMP features into three modern backbones—SegFormer-B1, SegNeXt-S, and EfficientViT-B2—through both a direct-input strategy and a hybrid dual-stream strategy.

1. Definition in grayscale morphology

The paper adopts flat structuring elements and defines DMP on a single-band image through grayscale morphology. The two primitive operators are dilation and erosion, followed by opening and closing. The formalism is given as follows (Huangal et al., 4 Sep 2025):

$$\begin{align} \delta^{SE}(I) = \delta(I) \vee \mathtt{SE} \ \epsilon^{SE}(I) = \epsilon(I) \wedge \mathtt{SE} \end{align}$$

where $I$ is a single-banded image, $SE$ is the structuring element, $\vee$ is a set-wise maximum, and $\wedge$ is a set-wise minimum. Using these two base operations, the paper defines opening $\gamma$ and closing $\varphi$ as

$$\begin{align} \gamma^{SE}(I) = \delta^{SE}(\epsilon^{SE}(I)) \ \varphi^{SE}(I) = \epsilon^{SE}(\delta^{SE}(I)) \end{align}$$

Let $L$ be a strictly increasing set of structuring-element sizes. The opening and closing profiles over scale are

$$\begin{align} \Pi\gamma(I) = \{\Pi\gamma_l : \gamma^{SE=l}(I), \forall l \in L\} \ \Pi\varphi(I) = \{\Pi\varphi_l : \varphi^{SE=l}(I), \forall l \in L\} \end{align}$$

The Differential Morphological Profile is then defined as absolute forward differences between adjacent scales, using forward differences only, for all $I$0 with $I$1:

$I$2

Within this notation, $I$3 denotes opening and $I$4 denotes closing. The paper does not define or use the morphological gradient $I$5 or top-hat transforms. This formulation makes DMP a representation of how image structure changes as the size of the structuring element increases. A plausible implication is that the descriptor is particularly suited to dense prediction problems in which class identity is strongly coupled to geometry across multiple object scales.

2. Structuring elements, scale sets, and feature construction

The study evaluates two flat structuring-element shapes: square and disk. No oriented line or anisotropic structuring elements are used (Huangal et al., 4 Sep 2025). The scale configurations are organized into the Original DMPNet set, an Improved DMPNet multi-scale set, and two ECA-discovered sets.

Set	Differentials	Stated emphasis
Original DMPNet set	[5−3], [7−5], [9−7]	Thin/small features
Improved DMPNet multi-scale set	[5−3], [7−5], [9−7], [15−9], [21−15], [27−21], [35−27]	Adds larger objects
ECA-discovered Set 1	[29−5], [23−5], [19−13], [17−13], [17−9], [15−11], [13−7]	RarePlanes object detection fitness; task-transfer to segmentation
ECA-discovered Set 2	[29−5], [23−9], [23−5], [19−13], [17−13], [15−11], [13−7]	RarePlanes object detection fitness; task-transfer to segmentation

The rationale given for these choices is explicit. Small differentials capture thin structures and boundaries, including crosswalk lines, road markings, and vehicles. Larger differentials capture whole objects and broader context, including aircraft, storage tanks, and soccer fields. The experiments indicate that disks tended to outperform squares, which the paper associates with a better isotropic response to shapes common in overhead imagery such as roundabouts, pools, and tanks.

DMP feature computation is performed as preprocessing on RGB overhead imagery. The segmentation experiments cast the input to grayscale before morphology, although the exact grayscale formula is not specified in this work; prior DMPNet work used ITU-R 601-2 LUMA for grayscale casting. Morphology is then applied with flat structuring elements at increasing sizes, and absolute forward differences across scales produce DMP channels for opening and closing streams. Border handling and efficiency are not detailed. DMP is precomputed as a non-learnable, non-differentiable feature extractor, and no surrogate gradients or differentiable morphology are used.

The channel count follows directly from the number of structuring-element differentials. If $I$6 is the number of differential pairs, then DMP opening yields $I$7 channels, DMP closing yields $I$8 channels, and grayscale adds one channel. The Original set has $I$9, giving $SE$0 channels total. The Improved and both ECA sets have $SE$1, giving $SE$2 channels. Normalization is not explicitly stated; DMP is concatenated with grayscale and fed to the network, and no quantization is described.

3. Architectural integration into segmentation networks

The study implements two integration strategies: a direct-input depth-extended approach and a hybrid dual-stream approach (Huangal et al., 4 Sep 2025). Both are evaluated on SegFormer-B1, SegNeXt-S, and EfficientViT-B2.

In the direct-input approach, DMP closing channels, grayscale, and DMP opening channels are concatenated along the channel dimension and passed directly into the encoder. The procedure is specified as: image received as input; image cast to grayscale; DMP generated from the grayscale image using both opening and closing profiles; closing profile, grayscale, and opening profile concatenated into a $SE$3-channel tensor; encoder processes the tensor; decoder produces segmentation masks. The first convolution or patch embedding is modified to accept 7 or 15 channels depending on the structuring-element set. The paper identifies a drawback: this disrupts pretrained weights for the first layer. No special positional encoding changes are reported; for SegFormer’s MiT, the first projection is simply adapted to the new input dimensionality.

In the hybrid dual-stream approach, color information is preserved by using separate encoders for RGB and grayscale+DMP. The steps are: image received as input; original RGB fed to one encoder; grayscale plus DMP channels fed to a second encoder of the same architecture; weights are not shared; feature maps are concatenated along the channel dimension; the fused features are passed to the segmentation decoder. Both streams use the same backbone family, fusion is channel-wise concatenation, no attention or cross-attention is used, and decoders remain as in the original architectures.

The paper also provides concise computation and forward-pass descriptions. DMP computation per image consists of converting $SE$4 to $SE$5, computing $SE$6 and $SE$7 for $SE$8, then constructing forward absolute differences over $SE$9, and finally returning $\vee$0. The direct-input forward pass is $\vee$1, $\vee$2, $\vee$3. The hybrid forward pass is $\vee$4, $\vee$5, $\vee$6, $\vee$7.

A central conclusion of the architectural study is that input-level replacement of RGB with grayscale+DMP is not sufficient, whereas feature-level fusion through a dedicated DMP encoder is more reliable. This suggests that DMP functions best as a complementary modality rather than as a substitute for spectral information.

4. Experimental protocol on iSAID

The empirical study uses the iSAID semantic segmentation benchmark, which contains 15 object classes and very large source images with widths from 800 to 13,000 pixels (Huangal et al., 4 Sep 2025). To make training tractable, images are tiled into $\vee$8 crops with stride 512, producing 33,978 training images and 11,644 validation images.

Encoders use ImageNet pretrained weights, while decoders are randomly initialized with Kaiming initialization. Optimization runs for 160,000 iterations with batch size 16, using AdamW with initial learning rate 0.00006 and a polynomial learning-rate schedule. Augmentations consist of horizontal and vertical flips and random photometric distortions. A fixed random seed is used to ensure fair comparison across models.

The paper does not specify the loss function and states only that standard segmentation losses are used per architecture’s implementation. Evaluation uses mIoU as the primary metric, with mPrecision, mRecall, and mF1 also reported as per-class averaged metrics. No statistical significance tests or confidence intervals are reported.

The protocol is aligned with the paper’s remote-sensing motivation. Large imagery, heavy tiling, and multiple object scales make the benchmark appropriate for testing whether a scale-aware morphological descriptor contributes signal beyond standard RGB encoders.

5. Empirical performance and class-wise behavior

Across all three backbones, non-DMP baselines generally outperform direct-input DMP variants, while hybrid DMP consistently outperforms direct-input and in one case surpasses the non-DMP baseline on mIoU, mF1, and mRecall (Huangal et al., 4 Sep 2025).

For SegFormer-B1, the baseline achieves mIoU 66.51, mF1 78.79, mPrec 86.96, and mRec 73.16. The best direct-input DMP configuration is Evo. 2 with disk structuring elements, yielding mIoU 63.19, mF1 75.85, mPrec 85.95, and mRec 69.70. The hybrid DMP configuration with Evo. 2 and disk structuring elements improves over direct-input to mIoU 65.84, mF1 78.23, mPrec 86.54, and mRec 72.52, but remains below the baseline.

For SegNeXt-S, the baseline records mIoU 67.72, mF1 79.59, mPrec 85.66, and mRec 75.36. The best direct-input DMP setting is the Original set with disk structuring elements, yielding mIoU 66.26, mF1 78.19, mPrec 84.90, and mRec 73.48. The hybrid DMP configuration with the same Original disk setting reaches mIoU 67.97, mF1 79.76, mPrec 85.23, and mRec 75.69. This is the only DMP variant reported to surpass the baseline across mIoU, mF1, and mRecall, with slightly lower precision.

For EfficientViT-B2, the baseline obtains mIoU 67.12, mF1 79.18, mPrec 86.70, and mRec 73.58. The best direct-input DMP configuration is the Original disk setting with mIoU 65.47, mF1 77.70, mPrec 86.57, and mRec 71.76. The corresponding hybrid DMP model improves to mIoU 65.90, mF1 78.22, mPrec 85.98, and mRec 72.66, but does not reach the baseline.

Per-class observations clarify where DMP helps. Direct-input DMP often excels on geometry-dominated classes. Roundabout gives the best IoU for direct-input in SegFormer and EfficientViT, with SegNeXt direct-input also competitive. Bridge and basketball court are often best or second best for direct-input variants. The paper attributes this to the simple, high-contrast shapes of these classes and to the effect of forcing the model to rely on grayscale plus DMP.

Hybrid DMP mitigates many direct-input weaknesses and helps classes requiring both color and shape. Swimming pool is a prominent example: for SegFormer, the baseline is 50.88, direct-input 28.00, and hybrid 51.47; for SegNeXt, the baseline is 45.62, direct-input 31.15, and hybrid 51.65. Hybrid can also slightly improve small-vehicle and large-vehicle IoU relative to direct-input. By contrast, the baseline remains strongest on texture-rich or color-rich classes such as baseball diamond, ground track field, and plane/helicopter, indicating continued dependence on texture and color cues that the direct-input stream lacks.

6. Interpretation, ablations, and limitations

The ablation findings emphasize that DMP effectiveness is architecture-dependent and class-dependent (Huangal et al., 4 Sep 2025). Forward absolute differences between successive scales are the only differential scheme used. Broad multi-scale sets do not guarantee better segmentation: the best direct-input setting for SegNeXt uses the small Original set, while SegFormer’s best direct-input configuration uses Evo. 2. Disk structuring elements generally outperform square structuring elements for SegFormer and SegNeXt; EfficientViT shows mixed results but still favors disks in some configurations. More scales increase channel count and computation, but the paper’s best overall result, SegNeXt hybrid, uses the smaller Original set, indicating that per-backbone tuning is necessary.

The fusion results show that simple channel-wise concatenation consistently improves upon direct-input DMP, even though no attention-based fusion is used. The paper explicitly suggests attention-based fusion as future work. The injection point also matters: DMP as input underperforms, whereas DMP through a dedicated encoder fused at feature level is more reliable.

The analysis section links performance patterns to remote-sensing failure modes. DMP’s multi-scale differences accentuate structures at multiple object sizes, from thin lines and small vehicles to large fields and planes. Opening differentials highlight bright, convex structures such as aircraft fuselages and storage tanks, while closing differentials emphasize dark concavities and shadows such as bridge decks and vehicles against asphalt. DMP tends to boost recall for certain geometric classes by making shape contrast more explicit, although direct-input recall still lags overall without color.

The reported error modes are specific. Direct-input failures occur on color-dependent classes, including camouflaged aircraft and red nose cones, and in texture-rich scenes, due to loss of spectral cues. Hybrid models retain weaknesses of their own: channel-wise concatenation sometimes fails to exploit complementary strengths, as in a roundabout example where direct-input exceeds baseline and baseline exceeds hybrid. A plausible implication is that selective or attention-mediated fusion could better preserve the conditions under which pure geometric cues dominate.

The computational trade-offs are also explicit. Direct-input increases input channels from 3 to 7 or 15, adding cost to the highest-resolution stage and disrupting pretrained input weights. Hybrid roughly doubles encoder compute and parameters, with corresponding increases in memory and latency. Despite this overhead, hybrid delivers the strongest segmentation among DMP variants and, for SegNeXt, surpasses the RGB baseline.

7. Relation to prior DMPNet work and practical implications

The paper situates itself as the first study of DMPNets for semantic segmentation and characterizes prior DMPNet research as focused on classification and detection (Huangal et al., 4 Sep 2025). Those earlier studies showed that shape features can help specific classes but do not universally beat RGB baselines, with performance sensitive to encoder choice, structuring-element sets, and pretraining. The semantic-segmentation study extends that line of work and reaches three practical conclusions.

First, directly replacing RGB with grayscale plus DMP is insufficient for dense prediction. Second, a hybrid dual-stream architecture can match or surpass a strong RGB baseline, as demonstrated by SegNeXt-S on mIoU, mF1, and mRecall. Third, disk structuring elements and careful per-architecture tuning are important.

The paper’s practical guidance follows these observations closely. Hybrid DMP is preferred when additional compute is acceptable and robust gains are desired, because it preserves color and consistently outperforms direct-input. Direct-input DMP can still be useful when target classes are geometry-dominated, such as roundabouts and some bridges, or when memory constraints rule out dual encoders, but an overall mIoU drop relative to RGB baselines should be expected. Disk structuring elements are recommended as the default choice. The Original set is a reasonable starting point when classes contain thin structures or when compute is limited, since three differentials yield seven channels. Larger multi-scale sets may be considered when objects span wide size ranges, but transfer from detection-oriented ECA-found sizes to segmentation is not guaranteed and should be validated per backbone.

For large images, the paper recommends tiling large scenes, as done with $\vee$9 crops and stride 512, and precomputing DMP per tile. For very large structuring elements, efficiency tricks such as downsampling before morphology or iterative application of smaller structuring elements are proposed as future work rather than evaluated methods.

Taken together, the study establishes DMP as a non-learnable, multi-scale morphological representation that can strengthen semantic segmentation when fused with RGB features. Its contribution is not a universal replacement for standard spectral pipelines, but a structured geometric complement whose value depends on class morphology, encoder family, structuring-element design, and the point at which fusion is introduced into the network.