LMFNet: Lightweight Multi-scale SOD Network

• LMFNet is a lightweight salient object detection network that leverages the LMF layer to fully connect multi-scale features via depthwise separable dilated convolutions.
• The encoder-decoder architecture integrates the LMF-based encoder without an external backbone, achieving competitive performance with only 0.81M parameters and 3.8G FLOPs.
• Ablation studies highlight that optimized dilation schedules and a hybrid loss function (SSIM, BCE, IoU) are key to balancing efficiency and detection accuracy.

LMFNet is a lightweight salient object detection (SOD) network built around the LMF (Lightweight Multi-scale Feature) layer, a multi-scale feature extraction layer that employs depthwise separable dilated convolutions in a fully connected structure (Shi et al., 10 Aug 2025). It was proposed to address the tension between efficient inference and effective multi-scale context modeling in lightweight CNNs, especially for SOD, where accurate foreground masks depend on both local detail and large-scale context. In its reported SOD configuration, LMFNet uses 0.81M parameters and 3.8G FLOPs for $256\times256$ input while achieving state-of-the-art or comparable results on DUTS-TE, ECSSD, HKU-IS, PASCAL-S, and DUT-OMRON; related code files are available at https://github.com/Shi-Yun-peng/LMFNet (Shi et al., 10 Aug 2025).

1. Problem setting and design objective

Salient object detection aims to produce a binary mask highlighting the main foreground objects in an image. Strong SOD systems typically depend on multi-scale feature extraction mechanisms such as FPN, ASPP, pyramid pooling, deformable convolution, or nested U-shaped designs, but these modules are often parameter- and FLOP-intensive. LMFNet is explicitly framed as a response to this efficiency bottleneck: it targets efficient multi-scale feature learning in lightweight CNNs, with emphasis on rich context capture, low parameter count, and competitive SOD accuracy (Shi et al., 10 Aug 2025).

The motivating challenges are threefold. First, lightweight networks have a limited receptive field because fewer layers and channels reduce effective spatial coverage. Second, classical multi-scale modules are expensive and therefore poorly suited to strict lightweight regimes. Third, stacked dilated convolutions introduce discontinuous sampling effects when adjacent dilation ratios become too large, causing “holes” and information loss. LMFNet addresses these constraints by replacing heavy multi-scale machinery with a simple fully connected multi-scale layer based on depthwise separable dilated convolutions, then stacking those layers into an encoder-decoder network (Shi et al., 10 Aug 2025).

A recurrent misconception is that LMFNet is a lightweight head attached to a conventional backbone. In the reported design, there is no external backbone like ResNet or VGG; the LMF-based encoder is the backbone. This architectural decision is central to its low complexity.

2. LMF layer and fully connected multi-scale coupling

The LMF layer is the basic computational unit of LMFNet. It takes a vector of feature maps

$$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$

where each $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$, and applies a vector of depthwise separable dilated convolutions

$$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$

governed by a dilation factor vector

$\bm{d} = [d_0, d_1, \dots, d_n].$

Each input feature map is passed through each operator $K_i$, and the resulting responses are concatenated: $$\bm{F_i}= \text{Concat}(\mathit{K_i}(\bm{I_0}), \mathit{K_i}(\bm{I_1}), \dots, \mathit{K_i}(\bm{I_m})).$$ A $1\times1$ convolution $K^c$ then fuses the concatenated response: $\bm{O_i} = \mathit{K^c}(\bm{F_i}).$ This is the paper’s “fully connected structure” in the scale dimension: every branch sees all incoming feature maps, and every output branch is obtained by fusing all per-input responses (Shi et al., 10 Aug 2025).

The efficiency argument is rooted in the use of depthwise separable convolution. A standard convolution with kernel size $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$0, $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$1 input channels, and $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$2 output channels has

$$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$3

whereas a depthwise separable convolution has

$$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$4

For $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$5, $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$6, and $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$7, the paper gives 589,824 parameters for standard convolution and 67,840 for depthwise separable convolution, a ~8.7× reduction. Every convolution in the LMF layer, including the depthwise and pointwise components, is followed by BatchNorm and an activation (Shi et al., 10 Aug 2025).

The dilation schedule is not arbitrary. The paper proposes three rules: use multiple dilation rates per LMF layer; keep the dilation rate ratio between adjacent layers at or below 3 for $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$8 kernels; and retain at least one branch with $$\bm{I} = [\bm{I_0}, \bm{I_1}, \bm{I_2}, \dots, \bm{I_m}],$$9 in each LMF layer. The first LMF layer uses

$\bm{I_j} \in \mathbb{R}^{C \times H \times W}$0

while deeper layers use

$\bm{I_j} \in \mathbb{R}^{C \times H \times W}$1

This design is intended to mix dense local sampling with enlarged receptive fields while reducing discontinuity effects in early high-resolution layers (Shi et al., 10 Aug 2025).

3. Network topology

LMFNet is an encoder-decoder SOD network composed entirely of LMF layers, together with max pooling for downsampling and bilinear interpolation for upsampling (Shi et al., 10 Aug 2025). In the encoder, the input image $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$2 is processed as

$\bm{I_j} \in \mathbb{R}^{C \times H \times W}$3

where $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$4 is the first LMF layer without pooling and $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$5 denotes an LMF layer with additional max pooling. The first encoder layer uses the special schedule $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$6, whereas deeper blocks use the prefix of $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$7 appropriate to the number of operators in that layer.

The model also includes a separate path for mid- and low-level feature fusion: $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$8 where $\bm{I_j} \in \mathbb{R}^{C \times H \times W}$9 denotes pooling-equipped LMF layers and $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$0 denotes upsampling-equipped LMF layers. This creates two auxiliary features, $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$1 and $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$2, which are later merged into the decoder.

The decoder proceeds as

$$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$3

followed by a $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$4 convolution and a Sigmoid to produce the saliency map: $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$5 In functional terms, the network combines deep high-level features, fused mid-level features, and fused low-level features through a lightweight skip-like concatenation strategy rather than attention or deformable alignment (Shi et al., 10 Aug 2025).

4. Optimization, datasets, and evaluation protocol

LMFNet is trained end-to-end with a hybrid loss

$$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$6

where $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$7 is the predicted saliency map and $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$8 is the ground-truth saliency mask. The reported ablation indicates that combining SSIM, binary cross-entropy, and IoU yields better performance than using BCE alone or two-term variants (Shi et al., 10 Aug 2025).

The implementation uses PyTorch with $$\bm{K} = [\mathit{K_0}, \mathit{K_1}, \dots, \mathit{K_n}],$$9 input resolution. Training is performed on DUTS-TR (10,553 images), and testing is conducted on DUTS-TE, ECSSD, HKU-IS, PASCAL-S, and DUT-OMRON. Data augmentation includes random brightness and contrast adjustment, random cropping, and horizontal flipping. Optimization uses Adam with $\bm{d} = [d_0, d_1, \dots, d_n].$0, $\bm{d} = [d_0, d_1, \dots, d_n].$1, and weight decay 0.0001. The initial learning rate is 0.001 with exponential decay by a factor of 0.98 per epoch; batch size is 8; training runs for up to 200 epochs, with convergence at around 146 epochs (Shi et al., 10 Aug 2025).

Evaluation uses MAE, maximum $\bm{d} = [d_0, d_1, \dots, d_n].$2 with $\bm{d} = [d_0, d_1, \dots, d_n].$3, maximum $\bm{d} = [d_0, d_1, \dots, d_n].$4, adaptive $\bm{d} = [d_0, d_1, \dots, d_n].$5 with

$\bm{d} = [d_0, d_1, \dots, d_n].$6

PR curves, F-measure curves, parameter count, and FLOPs. The MAE definition is

$\bm{d} = [d_0, d_1, \dots, d_n].$7

and the paper reports thresholds over $\bm{d} = [d_0, d_1, \dots, d_n].$8 for PR and F-measure curves (Shi et al., 10 Aug 2025).

5. Empirical performance and ablation behavior

The main quantitative result is that LMFNet achieves 0.81M parameters and 3.8G FLOPs at $\bm{d} = [d_0, d_1, \dots, d_n].$9 input while remaining competitive across five SOD benchmarks (Shi et al., 10 Aug 2025). On DUTS-TE, it reports MAE 0.059, $K_i$0, $K_i$1, and $K_i$2. On ECSSD, it reports MAE 0.054 and $K_i$3, with similar results stated for HKU-IS, PASCAL-S, and DUT-OMRON.

The paper positions the model against both heavy and lightweight baselines. Compared with U2Net, which has 44.01M parameters, 58.8G FLOPs, and average $K_i$4, LMFNet has 0.81M parameters, 3.8G FLOPs, and average $K_i$5. The reported interpretation is not that LMFNet matches the very best heavy models in absolute accuracy, but that it occupies a strong efficiency-performance trade-off point. Against lightweight baselines, CSNet is smaller but slightly weaker on most datasets, HVPNet is more accurate on several datasets but has higher parameter count and uses attention and pre-training, and SAMNet is generally the most accurate among lightweights but has more parameters and a more complex architecture (Shi et al., 10 Aug 2025).

The ablation study isolates the effects of dilation schedule, loss composition, model size, and dense-sampling branches. The default dilation schedule

$K_i$6

performs best; $K_i$7 is slightly worse; $K_i$8 is significantly worse; and $K_i$9 causes the largest drop. Removing the $$\bm{F_i}= \text{Concat}(\mathit{K_i}(\bm{I_0}), \mathit{K_i}(\bm{I_1}), \dots, \mathit{K_i}(\bm{I_m})).$$0 branch also decreases performance, supporting the claim that dense local sampling is needed to counteract information loss from large dilations. On the optimization side, BCE+SSIM+IoU outperforms BCE alone and two-term combinations. With respect to scale, a reduced model of about 0.5M parameters loses some accuracy, while increasing to 1.31M yields only marginal improvement, suggesting that the 0.81M configuration is near a favorable efficiency-performance operating point (Shi et al., 10 Aug 2025).

Qualitative comparisons show that LMFNet tends to produce clean and coherent saliency maps with good foreground coverage, though heavier models can still be better on fine boundary precision. Relative to several lightweight baselines, LMFNet is reported to provide better discrimination in cluttered backgrounds and more complete detection of small or low-contrast salient objects.

6. Conceptual significance, extensions, and limitations

LMFNet’s central conceptual contribution is the claim that receptive-field diversity in lightweight networks can be achieved without heavy pyramidal or attention-based machinery. Instead of increasing depth or channel count, the network uses multiple dilated depthwise separable convolutions arranged in a fully connected cross-scale pattern. This suggests a design principle in which efficient receptive-field engineering substitutes for architectural heaviness (Shi et al., 10 Aug 2025).

A second conceptual point is the model’s “fully connected” scale coupling. In many conventional parallel multi-scale modules, each branch operates on a single feature level. In the LMF layer, each dilation branch sees all input feature maps and each output branch fuses those responses through a $$\bm{F_i}= \text{Concat}(\mathit{K_i}(\bm{I_0}), \mathit{K_i}(\bm{I_1}), \dots, \mathit{K_i}(\bm{I_m})).$$1 convolution. A plausible implication is that the layer acts as a compact alternative to more explicit feature pyramid fusion mechanisms while remaining structurally simple.

The paper also tests the transferability of the encoder beyond SOD. Reusing the encoder as a backbone for CIFAR-10/100 classification, followed by Global Average Pooling and two fully connected layers, yields 0.6995 top-1 accuracy on CIFAR-100 with 0.66M parameters and 0.04G FLOPs for OURS, and 0.7135 with 1.34M parameters and 0.10G FLOPs for OURSp. This is presented as evidence that LMF-based encoders can generalize beyond salient object detection (Shi et al., 10 Aug 2025).

The limitations are explicit. LMFNet does not fully match the strongest heavy SOD models such as U2Net or CDMNET in absolute accuracy, especially on fine boundary handling and some complex scenes. The model intentionally avoids advanced attention and deformable modules, which preserves lightness but can limit peak accuracy. The ablation also indicates possible redundancy, since increasing parameter count beyond 0.81M gives only minor gains. Future work proposed in the paper includes model pruning and other optimization techniques, broader application of the LMF layer to image segmentation and other dense prediction problems, and further refinement of dilation scheduling for different domains (Shi et al., 10 Aug 2025).