Scale-Driven Decoder (SDD) in FlexDepth

• The paper introduces SDD, a decoder that removes pre-convolutions, employs dynamic content-aware upsampling, and adapts its architecture based on model scale.
• SDD enhances global semantic preservation and object boundary precision, addressing the multi-scale and dynamic challenges in driving environments.
• The design supports five FlexDepth variants, achieving state-of-the-art accuracy on KITTI and Cityscapes while offering efficiency improvements such as 37.6 FPS on mobile platforms.

Searching arXiv for the primary paper and closely related uses of “SDD” to ground the article in current literature. arXiv search query: (Zhu et al., 1 Jul 2026) The Scale-Driven Decoder (SDD) is the decoder architecture introduced in FlexDepth, a family of self-supervised monocular depth estimation models for driving scenes. In that setting, SDD is the architectural core that removes pre-convolution, introduces dynamic upsampling, and selects decoder components as a function of overall model scale rather than fixing one decoder design across all capacity points. It operates within a two-stage static–dynamic decoupled training pipeline, but the decoder itself is unchanged across stages; what varies is the supervision. The design is motivated by the combination of highly multi-scale geometry, dynamic traffic participants, and structure-sensitive depth discontinuities in road environments (Zhu et al., 1 Jul 2026).

1. Design rationale and problem setting

Self-supervised monocular depth estimation in driving typically relies on monocular videos or stereo pairs together with photometric consistency, without ground-truth depth. The driving setting is described as highly multi-scale, dynamic, and structure-sensitive: very near road surfaces, mid-range vehicles and pedestrians, and far buildings or trees coexist; moving vehicles, cyclists, and pedestrians violate the static-scene assumption behind photometric loss; and safety depends on sharp object boundaries and correct depth for small or distant actors (Zhu et al., 1 Jul 2026).

Most prior self-supervised MDE decoders are described as simple U-Net–style modules in which each scale follows

$$\text{pre-conv} \rightarrow \text{upsample} \rightarrow \text{concat} \rightarrow \text{post-conv}.$$

In the FlexDepth formulation, that pattern is problematic when the same decoder is scaled from tiny to large models. For lightweight models in particular, pre-convolution on low-resolution deep features is said to save FLOPs but to destroy already sparse global semantic information before spatial restoration, leading to blurred object boundaries, missing small or distant objects, and poor behavior when the same decoder design is reused across scales (Zhu et al., 1 Jul 2026).

SDD is presented as the answer to that failure mode. Its defining commitments are architectural rather than loss-specific: it removes pre-conv entirely to preserve global semantics; it uses content-adaptive dynamic upsampling instead of fixed bilinear interpolation; and it changes the post-convolution block and prediction-head ordering according to model size. This design supports a five-model family—Nano, Small, Medium, Large, and X-Large—rather than a single capacity point (Zhu et al., 1 Jul 2026).

2. Position within FlexDepth

FlexDepth takes an input image $I_t \in \mathbb{R}^{H \times W \times 3}$ and passes it through a YOLOv11-based depth encoder, the SDD, a ResNet-18 pose network with a small decoder, and a self-supervised warping stage. The encoder uses “11n/s/m/l-seg” variants and produces multi-scale features $\mathbf{F}_{enc,i}$ at resolutions $\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$ of the input. SDD then performs top–down decoding from the coarsest feature (Zhu et al., 1 Jul 2026).

At decoder level $i$, SDD receives a decoder feature $\mathbf{F}_{i+1}$ and the corresponding encoder skip feature $\mathbf{F}_{enc,i}$. The decoding step is

$$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$

where $\mathcal{U}$ is a scale-dependent upsampling operator, $\mathcal{C}$ is the post-convolution block, and $I_t \in \mathbb{R}^{H \times W \times 3}$0 is the prediction head. The output is a refined decoder feature for the next higher resolution and, optionally, a depth prediction at scale $I_t \in \mathbb{R}^{H \times W \times 3}$1 (Zhu et al., 1 Jul 2026).

The decoder is “scale-driven” because its internal configuration depends on the overall model scale. The paper specifies the following choices:

Model scale	Decoder choices
Nano / Small	HEB, dynamic upsampling, traditional head
Medium / Large / X-Large	HPB, dynamic upsampling, inverted head

For the five FlexDepth variants, the width multipliers are $I_t \in \mathbb{R}^{H \times W \times 3}$2, $I_t \in \mathbb{R}^{H \times W \times 3}$3, $I_t \in \mathbb{R}^{H \times W \times 3}$4, $I_t \in \mathbb{R}^{H \times W \times 3}$5, and $I_t \in \mathbb{R}^{H \times W \times 3}$6. The paper explicitly states that there is no per-pixel or per-region routing inside the decoder. The selection is static at the architecture level, while the upsampling itself is dynamic and content-aware through learned offsets $I_t \in \mathbb{R}^{H \times W \times 3}$7 (Zhu et al., 1 Jul 2026).

3. Internal components

The post-convolution operator $I_t \in \mathbb{R}^{H \times W \times 3}$8 is instantiated as either the High-Efficiency Bottleneck (HEB) or the High-Performance Bottleneck (HPB). HEB is used for Nano and Small models and is described as lightweight but favorable to gradient flow. Given an input feature $I_t \in \mathbb{R}^{H \times W \times 3}$9, it applies an initial convolution and channel split,

$\mathbf{F}_{enc,i}$0

then a sequence of bottlenecks

$\mathbf{F}_{enc,i}$1

followed by dense concatenation and a final convolution,

$\mathbf{F}_{enc,i}$2

The intended effect is dense connectivity and shortened gradient paths, which the paper identifies as critical for very small models such as Flex-Nano (Zhu et al., 1 Jul 2026).

HPB is used for Medium, Large, and X-Large models. It replaces simple bottlenecks with CSP-based ones: $\mathbf{F}_{enc,i}$3 The stated effect is to split feature paths, process one branch through a stack of bottlenecks, concatenate it with a shortcut branch, and fuse the result with a final convolution. HPB is heavier than HEB but more expressive, and is enabled only where the added representational power is considered affordable (Zhu et al., 1 Jul 2026).

The upsampling operator $\mathbf{F}_{enc,i}$4 is a dynamic, content-aware module inspired by deformable sampling: $\mathbf{F}_{enc,i}$5 where $\mathbf{F}_{enc,i}$6 is the low-resolution feature map, $\mathbf{F}_{enc,i}$7 is a base uniform sampling grid, $\mathbf{F}_{enc,i}$8 is a learned offset predicted from $\mathbf{F}_{enc,i}$9, $\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$0 is a coordinate normalizing factor, $\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$1 is the upsampling factor, $\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$2 is pixel shuffle, and $\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$3 is grid-sample. The paper’s interpretation is that the module learns where to sample from the low-resolution feature map for each high-resolution location, which improves precision at object borders and thin structures relative to uniform bilinear interpolation (Zhu et al., 1 Jul 2026).

The prediction head $\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$4 is also scale-dependent. Small models use a conventional head that predicts depth before upsampling. Medium and larger models use an inverted head that upsamples features first and predicts on high-resolution, high-channel features afterward. The stated rationale is that at large scale, predicting at high resolution improves local detail and depth discontinuities, while for small models the conventional head avoids extra full-resolution convolutions (Zhu et al., 1 Jul 2026).

4. Supervision and static–dynamic decoupled training

SDD is used identically in both stages of FlexDepth training. In Stage 1, the pose network and the SDD-based depth network are trained jointly using standard self-supervised losses. The warping equation is

$\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$5

the photometric error is

$\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$6

minimum reprojection with auto-masking is

$\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$7

and edge-aware smoothness is

$\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$8

All SDD depth predictions at scales $\frac{1}{2}, \frac{1}{4}, \dots, \frac{1}{32}$9 are supervised through

$i$0

This makes decoder quality directly visible to the self-supervised objective even though there is no SDD-specific regularizer (Zhu et al., 1 Jul 2026).

Stage 2 introduces static–dynamic decoupling without changing the decoder architecture. Two checkpoints from Stage 1—an early checkpoint $i$1 and a late checkpoint $i$2—are compared. The mask separating static and dynamic pixels is

$i$3

where $i$4 is a learned adaptive threshold. Pixels with stable predictions are labeled static; unstable regions are treated as dynamic (Zhu et al., 1 Jul 2026).

In Stage 2, PoseNet is frozen and the depth network with the same SDD is retrained using a masked normal distribution loss,

$i$5

a geometric depth smoothing term,

$i$6

and the final loss

$i$7

A common misconception is that SDD itself performs dynamic-scene reasoning. The paper’s formulation is narrower: dynamic-scene handling comes from the two-stage training and mask-based losses, whereas the decoder is reused unchanged across both stages (Zhu et al., 1 Jul 2026).

5. Efficiency, accuracy, and robustness

The FlexDepth results are reported on KITTI and Cityscapes, with SDD identified as central to the accuracy–efficiency trade-off. On KITTI at $i$8, the five model variants are reported as follows:

Model	Params / FLOPs	Abs Rel
Flex-Nano	1.52M / 0.718G	0.110
Flex-Small	6.06M / 2.78G	0.104
Flex-Medium	12.72M / 9.99G	0.096
Flex-Large	15.20M / 11.52G	0.095
Flex-X-Large	32.27M / 24.56G	0.093

On a Snapdragon 8 Elite at $i$9 and batch size 1, the reported speeds are 37.6 FPS for Flex-Nano, 18.6 FPS for Flex-Small, 5.8 FPS for Flex-Medium, 5.2 FPS for Flex-Large, and 3.0 FPS for Flex-X-Large. The paper also states that Flex-Nano requires only 0.7 GFLOPs and achieves 37.6 FPS on mobile platforms (Zhu et al., 1 Jul 2026).

The main benchmark claims attach specific SDD-enabled variants to specific accuracy points. On KITTI, Flex-Nano at 0.7G FLOPs matches or beats Lite-Mono-Tiny at similar AbsRel while using about one quarter of the FLOPs; Flex-Small at 2.8G outperforms Lite-Mono-Small and PuriLight; and Flex-X-Large at 24.6G achieves $\mathbf{F}_{i+1}$0 AbsRel and $\mathbf{F}_{i+1}$1 at $\mathbf{F}_{i+1}$2. On Cityscapes at $\mathbf{F}_{i+1}$3, Flex-X-Large reaches AbsRel $\mathbf{F}_{i+1}$4 and $\mathbf{F}_{i+1}$5. The authors describe these results as state-of-the-art across arbitrary scales without auxiliary information and with minimal computational overhead (Zhu et al., 1 Jul 2026).

Ablation studies isolate the contribution of the decoder. For X-Large on KITTI, HEB and HPB both yield $\mathbf{F}_{i+1}$6 AbsRel, but HPB improves SqRel from $\mathbf{F}_{i+1}$7 to $\mathbf{F}_{i+1}$8 and RMSE from $\mathbf{F}_{i+1}$9 to $\mathbf{F}_{enc,i}$0. Replacing dynamic upsampling with bilinear changes AbsRel from $\mathbf{F}_{enc,i}$1 to $\mathbf{F}_{enc,i}$2 while reducing FLOPs from $\mathbf{F}_{enc,i}$3G to $\mathbf{F}_{enc,i}$4G. Replacing the inverted head with a non-inverted head changes AbsRel from $\mathbf{F}_{enc,i}$5 to $\mathbf{F}_{enc,i}$6. Removing Stage 2 produces a larger degradation to $\mathbf{F}_{enc,i}$7 AbsRel, while a fixed-threshold mask recovers $\mathbf{F}_{enc,i}$8 and the adaptive dynamic mask achieves the best SqRel and RMSE at the same AbsRel (Zhu et al., 1 Jul 2026).

The robustness claims are not limited to whole-image metrics. On Cityscapes dynamic regions, Flex-X-Large is reported to achieve dynamic-region AbsRel $\mathbf{F}_{enc,i}$9 and $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$0 with $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$1G FLOPs. On KITTI moving-object analysis, FlexDepth improves AbsRel from $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$2 to $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$3 versus DSI-MonoViT, a $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$4 gain, and improves SqRel by $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$5. KITTI-trained Flex-X-Large also reaches the best Make3D metrics among lightweight baselines, with AbsRel $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$6 versus $$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$7–$$D_i = \mathcal{P}\Big\{\mathcal{C}\big(\mathrm{Concat}[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}]\big)\Big\},$$8, and zero-shot evaluation on Cityscapes and DDAD is reported as stable (Zhu et al., 1 Jul 2026).

6. Limitations, interpretation, and disambiguation

The paper explicitly limits the intended domain of SDD and FlexDepth to outdoor driving. Indoor environments with irregular textures, close-range clutter, and complex occlusions are not the focus, and performance there may be suboptimal. It also notes that the scale-driven mechanism is architecture-scale selection rather than per-image adaptive routing: there is no learned per-sample choice of HEB versus HPB or conventional versus inverted heads (Zhu et al., 1 Jul 2026).

Several extensions are proposed in the source material. These include a Transformer-based SDD that integrates lightweight transformers into HPB while preserving dynamic upsampling; content-driven decoder routing that activates different decoder branches according to depth range or scene statistics; depth-range-aware heads for near, mid, and far ranges; multi-task extensions that share SDD with semantic segmentation or optical flow; and retuning for indoor or non-driving domains. These are presented as possible extensions rather than demonstrated properties (Zhu et al., 1 Jul 2026).

The acronym SDD is also used elsewhere in the arXiv literature for unrelated concepts. In LLMs, Scale-Distribution Decoupling denotes a re-parameterization of fully connected layers for stable Transformer training (Wang et al., 21 Feb 2025). In knowledge distillation, Scale Decoupled Distillation denotes a logit distillation framework based on multi-scale local logits (Luo, 2024). In anti-UAV detection, SDD-YOLO refers to a scale-driven detection design centered on a P2 high-resolution detection head (Chen et al., 26 Mar 2026). In theoretical computer science, SDD commonly abbreviates symmetric diagonally dominant matrices and the associated parallel linear-system solvers [(Peng et al., 2013); (Blelloch et al., 2011)]. Within FlexDepth, however, SDD has the narrower and explicit meaning of a Scale-Driven Decoder for self-supervised monocular depth estimation in driving scenes (Zhu et al., 1 Jul 2026).